In the last post of this series, we learned about transport zones and why we need them. We also discussed transport node profiles and created a TN profile and a couple of transport zones. This post is focused on setting up the data plane in NSX-T.
If you have landed directly on this post by mistake, I would recommend reading previous articles from this blog series:
1: NSX-T Management & Control Plane Setup
3: Transport Zones & Transport Node Profiles
What is meant by the data plane in NSX-T?
The data plane is where all packet forwarding takes place based on tables created by the control plane. The packet-level stats are found here, as well as topology info, which is then reported from the data plane up to the control plane.
The data plane in NSX-T comprises 2 components: hosts and the edge nodes. Depending upon the environment, hosts can be vSphere hosts or KVM hosts. Data plane setup involves configuring NSX-T modules on hosts and deploying edge nodes to set up logical routing.
What Happens During Data Plane Setup?
When ESXi hosts are prepped for NSX-T, various vibs are pushed onto hosts, which enables functions like distributed routing, overlay networking, etc. All ESXi hosts that are configured as transport nodes have NSX-T modules installed and are registered with the NSX-T management plane.
The N-VDS defined in transport zones is installed on ESXi hosts & edge nodes during the host prep and edge deployment process. ESXi hosts & edges are then ready to participate in overlay networking.
When configuring NSX-T modules on an ESXi host, we have 2 methods. Either define the transport zone, IP pool for TEP, uplink profiles, etc., on every ESXi host that is part of the cluster selected for NSX-T installation or use Transport Node profiles to define these settings just once and apply the profile on the cluster selected for installation.
In my lab, I am using transport node profiles to simplify my NSX-T installation.
Time to jump into the lab and see things in action. We will start with configuring NSX-T on the ESXi host.
Configure ESXi Transport Nodes
Login to NSX-T Manager and navigate to System > Fabric > Nodes > Host Transport Node, and select the vCenter Server instance under the Managed by drop-down list.
Select the cluster that you want to prep for NSX-T and click Configure NSX.
I am following a collapsed design in my lab, so I only have one cluster created in vCenter.
Select the transport node profile that you created earlier and click the Apply button.
NSX starts installing the VIBs on the ESXi host. This process typically takes 5-10 minutes (depending on the number of hosts in a cluster).
Once all hosts are prepped with NSX-T VIBs, NSX configuration status reports success and node status as up. The number of N-VDS installed depends on the transport zone, which we define in the Transport Node profile.
The tunnel status shows as not available, as we have not deployed any workloads in this cluster yet.
During host prep, you will see vmkernel adapters created on ESXi hosts. Although it reads VXLAN, it carries GENEVE traffic. IP addresses configured on these vmkernel port groups are ESXi host TEP IPs.
The NSX-T VIBs that get installed on the host enable a new namespace called nsxcli. To see the list of all NSX vibs that are installed on the host, run the command: esxcli software vib list | grep -i nsx
We are now done with ESXi transport node configuration.
Configure Edge Transport Nodes
Before deploying edge nodes, we need to create a couple of port groups on VDS. These port groups should be created as trunk port groups, as edge nodes use these to connect with the upstream ToR switches via VLAN-backed segments.
The diagram explains the topology.
I have 2 port groups created in my environment, and both are configured for VLAN trunking.
To deploy an Edge node, navigate to Edge Transport Nodes and click on Add Edge VM.
Provide a name & hostname for the edge VM and select the appropriate form factor. Certain features work only when edges are deployed in a large form factor.
Optionally, you can adjust CPU and memory reservations for the edge VM.
Enter credentials for the admin, root, and audit users. Also, if you wish, you can enable SSH login for the edge VM.
Select the placement options for the vm.
Note: As a best practice, edge nodes should be placed under a dedicated resource pool with appropriate shares configured.
Since this is a lab deployment, I did not create any custom resource pool for edges.
Configure Management IP/Interface for the edge. This is the network you will connect to when connecting edge appliances over SSH.
Enter your DNS & NTP details as well. Make sure you have forward & reverse records created in advance for the edge FQDN.
On the Configure NSX page, specify the following:
- Edge-Switch Name: The name which you provide here installs NVDS on the edge node.
- Transport Zone: Edge nodes will be part of the same overlay transport zone that the ESXi hosts are part of. Also, edges are part of the VLAN TZ so that they can connect to upstream ToR using VLAN-backed segments.
- Uplink Profile: An uplink profile dictates how edge NIC’s connect with physical networks.
- IP Assignment: You can either leverage IP pools or static IPs for Edge TEP IP assignment. In case of a static IP list, you have to specify the gateway address and subnet mask as well.
Under the teaming policy, map the edge uplinks to the trunked distributed port groups that we created earlier before edge deployment.
Hit the Finish button to start deploying the first edge VM.
For redundancy & high availability, I have deployed 2 edge nodes in my environment. The configuration state for both edge VMs is reported as “success,” and node status is reported as “up.” Also examine the TEP IPs to ensure they have the correct TEP address from the IP Pool/Static IP List.
Create Edge Cluster
To create an edge cluster, navigate to the Edge Clusters tab and click the Add button.
Specify a name for the edge cluster and leave the profile selected as default.
Move the edge VMs under the selected page by clicking the arrow button, and click Add to complete the edge cluster creation wizard.
And there we go. We have successfully set up the data plane now.
In the next post of this series, I will demonstrate logical routing setup.
I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.