In the last post of this series, we learned about transport nodes and how to setup a data plane. Now my NSX-T environment is ready for setting up logical routing and eventually starts flowing packets across the network.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this blog series:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

Let’s get started. 

What is Logical Routing?

NSX logical routing, enable us to connect both virtual and physical endpoints that are located in different logical Layer 2 networks. This is made possible by the separation of physical network infrastructure from logical networks that network virtualization provides.

Logical routing is provided by Logical Routers that get created on Edge Nodes when we configure routing. Logical Routers are responsible for handling East-West & North-South traffic across the datacenter.

Components of Logical Routing

Components that constitute logical routing in NSX-T are:

• Logical Routers ( Distributed Router & Service Router)
• Router Ports
• Segments

 The logical router has 2 components:

• Distributed Component: It runs as a kernel module in the hypervisor and is responsible for handling east-west routing among segments.
• Centralized Component: This component takes care of centralized functions like NAT, DHCP, and LB and provides connectivity to physical infrastructure.

In NSX-T we have two types of logical routers:

• Tier-0 Gateway aka T0-Router: A Tier 0 router is used to connect NSX networking with traditional physical networking. A Tier 0 router forwards layer 3 IP packets and typically peers with a traditional physical router using BGP or static routing.
• Tier-1 Gateway aka T1-Router: A tier 1 logical router is often used for tenants, users, and applications. Tier-1 logical routers have downlink ports to connect to NSX-T logical switches and uplink ports to connect to NSX-T tier-0 logical routers.

I will not discuss logic routing anymore in this post because I intend to cover this in greater detail in an upcoming post. 

In my lab, I am using Two-Tier Routing and my setup looks like as shown in the below diagram.

Create Uplink For T0 Gateway

T0 Gateway connects to the physical world using VLAN backed segments. These segments are part of the VLAN Transport Zone that is created specifically for Edge Nodes. 

In my lab, the Edge node connects to ToR using segments that are in VLAN 600 & 700 respectively. I have a couple of ToRs in my lab and edge nodes have connections to both ToRs.

Create & Configure T0 Gateway

To deploy T0 Gateway, navigate to Networking > Connectivity > Tier-0 Gateways and click on Add Gateway button.

Provide a name for the T0 GW and select HA mode. Select the edge cluster of which this T0 will be part and click on the save button.

T0 GW has been created now. Click on Yes to start the configuration of T0.

First, we have to attach uplink interfaces on T0. This is done under the Interfaces option within the T0 console.

Provide a name for the interface and select the type as External. 

Also provides an IP address that will be configured on this interface. This IP address acts as a BGP neighbor for the upstream ToR switch if you are planning to use BGP for routing. 

Under Connected To, select the VLAN backed segment which you created earlier. We have to add 2 interfaces per edge node. So if you have 2 edge nodes, and both edges connect to ToR using 2 distinct uplinks, there will be a total of 4 interfaces on T0. 

As of now, I have only added 2 interfaces and both interfaces is in VLAN 600. One interface connects with one edge node and another interface connects to the second edge node.

Further configuration in my case will be to add 2 interfaces selecting segment marked for VLAN 700 and both interfaces will be connected to their respective edges. 

Create & Configure T1 Gateway

Switch to the Tier-1 Gateways tab and click on Add gateway button.

Provide a name for the gateway and select the T0 GW to which this T1 GW will connect to. Also, select the edge cluster for T1 GW placement and Failover type. 

Click on the Save button to finish the Tier-1 GW creation wizard.

T1 gateway is deployed now.

Before further configuration of T0, I want to first setup BGP between Edge Nodes and the Physical network.

BGP Configuration for Dynamic Routing

NSX-T Edges can connect to the physical world via static routes or dynamic routing. NSX-T supports configuring BGP as routing protocol on the Edge nodes (T0). 

To configure BGP, edit the settings of the T0 gateway and expand the BGP section.

• Enter the local AS number for T0. This AS number is used as Remote AS for physical upstream devices (ToR or Router) where BGP might be already configured. 
• Toggle BGP, Inter SR iBGP, and ECMP options to enable them.
• Select Graceful Restart & helper and set the timer to 600.

If BGP is already configured on the upstream device, we can go ahead and set BGP Neighbors by clicking on the Set option. 

A new wizard will be open. Click on Add BGP Neighbors button to start configuring the neighbors. 

IP Address field is the IP address of your upstream Tor or Router device with which T0 will form BGP peering. Also, punch in the Remote As of ToR/Router.

Under Source Addresses, specify the IP addresses configured on uplink interfaces of T0. 

Note: If you are using any BGP password on your physical devices, punch in the password under Timers & Password field. 

Note: You can leave Hold Down Timers & Keep Alive Time to default values if you wish. Otherwise, change the values as per your infrastructure design. 

Click on Save to finish the BGP neighbor addition wizard. 

Enable Route Re-distribution

Route Re-distribution provides the capability of publishing routes from T0 GW to the upstream device. To know more about this, please see this Article

To enable Route Re-distribution, edit settings of T0 GW and expand Route Re-distribution, and toggle the Route status button to enable it. 

Clicking on the Set button opens Add Route Re-Distribution wizard.

Provide a name for the rule and click Set to specify which Tier-0 subnets will be advertised to the upstream devices. 

Select subnets to advertise and click on Apply.

Click on Apply button again to finish the wizard. 

Click on save to save the Route Re-distribution settings. 

Once we are done with configuring T0 & T1 GW, we can view the topology by navigating to Networking > Network Topology.

Verify Routing

We are done with setting up logical routing. It’s time to test whether routing is working as per design or not.

First I verified that my ToR can see my T0 as a BGP neighbor

I created a new segment named App-NW and attached it to my T1 GW.

On checking the route table on my ToR (vyos in my lab), I can see subnet 192.168.15.0 being learned via BGP via neighbor 172.16.60.2 & 60.3 (uplink interfaces of T0)

Also, I am able to ping the gateway of my overlay segment from my ToR.

Next, I connected to the SR component of T0 and verified that T0 is able to learn the routes which I have advertised from the ToR switch.

Next, I tried pinging one of the machine which is out there on the physical network, from my edge device and I am able to reach out to that machine.

And this concludes routing verification. I am able to route packets in both directions. 

And that it for this post. I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing 🙂