In last post of this series, we learnt about Transport Zones and why we need them. We also discussed about Transport Node profiles and created a TN profile and couple of Transport Zones. 

This post is focussed on components involved in data plane and how to configure the same in NSX-T.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this blog series:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

Let’s get started.

What is meant by Data Plane in NSX-T?

The data plane is where all packet forwarding takes place based on tables created by the control plane. Packet level stats are found here as well as topology info which is then reported from the data plane up to the control plane.

Data plane in NSX-T comprises of 2 components: Hosts and the Edge nodes. Depending upon the environment, hosts can be vSphere hosts or KVM hosts. Data plane setup involves configuring NSX-T modules on hosts and deploying edge nodes to setup logical routing. 

What Happens During Data Plane Setup?

When ESXi hosts are prepped for NSX-T, various vibs are pushed on to hosts, which enables functions like distributed routing, overlay networking etc. All Esxi host that are configured as Transport Nodes, have NSX-T modules installed and are registered with the NSX-T management plane.

N-VDS that have been defined in transport zones, are installed on ESXi host & Edge nodes during host prep and Edge deployment process. ESXi hosts & Edges are then ready to participate in overlay networking.

When configuring NSX-T modules on ESXi host, we have 2 methods. Either define things like Transport Zone, IP pool for TEP, Uplink profiles etc on every ESXi host that are part of cluster which has been selected for NSX-T installation or use Transport Node profiles to define these settings just once and apply the profile on cluster selected for installation. 

In my lab I am using Transport Node profiles to simplify my NSX-T installation.

Time to jump into lab and see things in action. We will start with configuring NSX-T on ESXi host first.

Configure ESXi Transport Nodes

Login to NSX-T manager and navigate to System > Fabric > Nodes > Host Transport Node and select the vCenter server instance under Managed by drop down list.

Select the cluster which you want to prep for NSX-T and click on Configure NSX.

I am following a collapsed design in my lab so I only have one cluster created in vCenter.

Select Transport Node profile which you have created earlier and click on Apply button.

Configure NSX task starts installing NSX-T vibs on ESXi host. This process typically takes 5-10 minutes (depending on number of hosts in a cluster).

Once all hosts are prepped with NSX-T bits, NSX configuration status reports as success and node status as up. Number of N-VDS installed depends on the transport zone which we define in TN profile. I had only 1 N-VDS defined in my TN profile. 

Don’t get disheartened by Tunnel status not available as we have not deployed any workloads in this cluster yet. 

During host prep, you will see vmkernel adapters created on ESXi hosts and type as vxlan. Although its reads vxlan, it carries GENEVE traffic. IP address configured on these vmkernel portgroups are ESXi host TEP IP’s. 

NSX-T vibs that gets installed on host, also enables a new namespace called nsxcli. To see list of all NSX vibs that are installed on host, run command: esxcli software vib list | grep -i nsx

we are now done with ESXi Transport Node configuration.

Configure Edge Transport Nodes

Before deploying Edge nodes, we need to create couple of portgroups on VDS. These portgroups should be created as trunk portgroup as Edge nodes uses these portgroups to connect with upstream ToR switches via VLAN backed segments.

Below diagram will kinda explain which I just talked about.

I haved 2 portgroups created in my environment and both portgroups are configured for VLAN trunking. 

To deploy an Edge node, navigate to Edge Transport Nodes and click on Add Edge VM.

Provide name & hostname for the edge vm and select appropriate form factor. Certain features such as load balancing works only when edges are deployed in large form factor.

Optionally you an adjust CPU/Memory reservations for the edge vm.

Punch in credentials for the Admin, Root and Audit user. Also if you wish, you can enable ssh login for the edge vm.

Select the placement options for the vm.

Note: As a best practice, Edge nodes should be placed under a dedicated Resource Pool with appropriate shares configured.

Since this is a lab deployment, I did not created any custom resource pool for edges.

Configure Management IP/Interface for the edge. This is the network over which you will connect to edge appliances over ssh etc.

Punch in your DNS & NTP details as well. Make sure you have forward & reverse records created in advance for edge FQDN. 

On configure NSX page, specify following:

• Edge-Switch Name: The name which you provide here, installs NVDS on the edge node. 
• Transport Zone: Edge nodes will be part of same overlay transport zone which ESXi hosts are part of. Also Edges are part of VLAN TZ, so that it can connect to upstream ToR using VLAN backed segments. 
• Uplink Profile: This you might have created earlier. If not, it can be created on the fly. Uplink profile dictates how edge NIC’s connect with physical networks.
• IP Assignment: You can either leverage IP pools or static ip’s for Edge TEP IP assignment. In case of Static IP list, you have to specify gateway address and subnet mask as well.

Under teaming policy, map the edge uplinks with the trunked distributed portgroups that we created earlier before edge deployment.

Hit finish button to start deploying the first edge vm.

For redundancy & high availability, I have deployed 2 edge nodes in my environment. Configuration state for both edge vm’s is reported as success and node status as up. Also examine the TEP IP’s to ensure it has got correct TEP address from IP Pool/Static IP List.

Create Edge Cluster

To create an edge cluster, navigate to Edge Clusters tab and click on Add button.

Specify a name for the edge cluster and leave the profile selected to default. 

Moe the edge vm’s under selected page by clicking on arrow button and hit Add to complete the edge cluster creation wizard.

And there we go. We have successfully setup the data plane now.

In next post of this series, I will demonstrate logical routing setup.

I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing 🙂