In the last post of the NSX-T series, I demonstrated East-West packet flow and discussed various cases around that. In this post, I will explain how packets are forwarded in northbound/southbound traffic. 

Before you start reading this article, please ensure you have a fair understanding of NSX-T routing architecture and how the SR & DR components of the logical router work together. Also, knowledge of TEP/MAC/ARP table formation is handy when trying out packet flow in lab/prod.

Here is the lab topology that I am going to use to demonstrate N-S packet walk.

Note: The topology below is a single-tier routing topology.

Egress to Physical Network

Here is how a packet traverses when VM 1, connected to the App-LS logical segment, tries to communicate with VM 2, which is on the physical network.

Step 1: VM 1 sends a Layer 2 packet to its default gateway (192.168.10.1), which is a LIF on the DR component on the hypervisor node.

Step 2: The DR component checks its forwarding table to see if it has a route to the 10.196.88.2 subnet. Since the route to the 10.196.88.x subnet doesn’t exist in the forwarding table, the DR has to send the packet to its default gateway 169.254.0.2, which is the SR component on the edge node.

Step 3: Before sending the packet to the edge node, the DR component on the ESXi host encapsulates the packet with a Geneve header.

Step 4: The encapsulated packet is sent to the edge node via the TEP tunnel (overlay transport network)

Step 5: Upon receiving the packet, the edge node decapsulates the packet and sends it to its SR component. The SR component checks its routing table and finds a route to the 10.196.88.x network (learned via BGP) over the uplink segment.

Step 6: The SR component sends the packet to its upstream physical router, which routes the packet to its destination vm (10.196.88.4)

Ingress from Physical Network

Step 1: The Source VM (10.196.88.4) sends the return packet to its default gateway, which is an IP on the physical router. The router checks its routing table and determines it has a route to the 192.168.10.0/24 network via 172.16.60.2 (edge uplink). The packet is then routed to the edge node.

Step 2: The SR component checks its routing table and finds a directly connected route to 192.168.10.0/24 via interface downlink-412.

The downlink-412 is nothing but an interface on the DR component.

The packet will be sent to the remote host using the DR interface.

Step 3: Before sending the packet from the edge node to the hypervisor, it is encapsulated with a Geneve header.

Step 4: The encapsulated packet is sent over the overlay network.

Step 5: The transport node then decapsulates the packet and routes it to its destination vm (192.168.10.2)

And this concludes the N-S packet walk in NSX-T single-tier routing architecture.

In the next post, I will explain the N-S packet walk in a multi-tier routing topology.

I hope you enjoyed reading the post. Feel free to share this on social media if it is worth sharing.