In my last post, I explained Egress/Ingress packet flow in a single-tier routing topology where logical segments are attached directly to T0 gateway.

In this article I will explain the same for a multi-tier routing topology in NSX-T.

Here is the topology which I have used in my lab.

Egress to Physical Network

Scenario: VM 1 with IP 192.168.10.2 is connected to logical segment App-LS and wants to communicate with a VM with IP 10.196.88.2 which is out there on physical network.

Step 1: VM 1 sends packet to its default gateway (192.168.10.1) which is LIF IP on T1-DR. 

Step 2: T1 DR checks its forwarding table to make a routing decision. Since route to network 10.196.88.x doesn’t exist in forwarding table, T1-DR sends the packet to its default gateway (100.64.0.0) which is the DR instance of Tier-0 on the same hypervisor.

Step 3: The packet is sent to the T0 DR instance over internal segment (Router-Link). … Read More

In last post of NSX-T series, I demonstrated East-West packet flow and discussed various cases around that. In this post I will explain how packets are forwarded in case of northbound/southbound traffic. 

Before you start reading this article, please ensure you have fair understanding of NSX-T routing architecture and how SR & DR component of logical router work together. Also knowledge of TEP/MAC/ARP table formation is handy when trying out packet flow in lab/prod.

Here is the lab topology that I am going to use to demonstrate N-S packet walk.

Note: Below topology is single-tier routing topology.

Egress to Physical Network

Here is how a packet traverse when VM 1 which is on App-LS logical segment tries to communicate with VM 2 which is out there on physical network.

Step 1: VM 1 sends layer 2 packet to its default gateway (192.168.10.1) which is a LIF on DR component on hypervisor node.… Read More

In my last post on the NSX-T series, I explained how VTEP, MAC & ARP table is constructed. This knowledge is needed to understand packet flow.

In this post, I will demonstrate how packet forwarding is performed for East-West traffic.  

NSX-T has an inbuilt tool called Traceflow, which is very handy when analyzing packet flow within/across segments. This tool is located under Plan & Troubleshoot > Troubleshooting tools > Traceflow in NSX-T UI.

This tool is very easy to use and you just need to select the source vm & destination vm and click on trace to start packet flow analysis.

There are 2 deployment models available for T1 gateways. We can instantiate the T1 gateway on an edge cluster or we can choose not to associate with any edge cluster. If we need stateful services on the T1 gateway, we go with the first deployment model. 

In part-1 of this post, I will demonstrate packet walk when T1 is associated with edge-cluster.… Read More

In this post I will explain how NSX-T create and maintains various table which forms the building block of logical switching.  Basically I will discuss about formation of below tables:

• VTEP Table
• MAC Table
• ARP Table

These tables are continuously updated and modified as we provision new workloads and create new segments. 

VTEP Table

This table holds the VNI to TEP IP mapping. A couple of points before we start.

• Each segment has a unique identifier called VNI. 
• Each transport node in that TZ will have a TEP IP. 

Lets understand TEP table creation with the help of below diagram.

Step 1: As soon as a segment is created in a TZ, all transport node of that TZ updates its local TEP table and registers VNI of the created segment against its TEP IP. Each transport node then send this info to Local Control Plane (LCP).

Note: VTEP can be viewed by logging into ESXi host and running command: get logical-switch <ls-uuid> vtep-table

Step 2: Each transport nodes then send their VNI-TEP entry from its LCP to CCP (running on NSX-T Manager).… Read More

In my last post I discussed about single-tier routing architecture and demonstrated how T0 gateway is able to handle both East-West & North-South routing. In this post I will explain two-tier (aka multi-tier) routing architecture. 

If you are new to NSX-T, I would recommend reading previous blog post from my NSX-T 3.0 series to gain some understanding. 

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

5: Configure Logical Routing in NSX-T

Let’s get started.

Introduction

Two-Tier architecture is most common deployment method in production environment. It lays the foundation of Multi-Tenancy by separating T0 gateway (provider construct) from the T1 gateway (tenant construct).

In a multi-tenant environment, its the service provider who takes cares of deploying & configuring T0 gateway. Tenants are responsible for creating & managing their respective T1 gateways.

Logical Routing Connectivity

Let’s do a quick recap about components of  T0 & T1 gateway and how they interact with each other.… Read More

In my NSX-t 3.0 series, I wrote an article on setting up Logical Routing so that traffic can start flowing through the SDDC.

If you have missed my NSX-T 3.0 series, here are the links to the same:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

5: Configure Logical Routing in NSX-T

Let’s do a quick recap on routing capabilities in NSX-T. 

Logical Routing in NSX-T has the ability to provide connectivity for both virtual and physical workloads that are in different logical L2 networks. Workloads get connected to each other via segments and these segments can in turn attach to a T0/T1 GW for East-West & North-South communication.

T0/T1 gateways have Service Router (SR) & Distributed Routers (DR). The DR component is embedded at the hypervisor level and ‘abstracted’ from the underlying physical network.… Read More

In my last Post i briefly talked about transit subnets that gets created when T1 gateway is attached to a T0 gateway. In this post we will learn in depth working of the SR components that gets deployed when we set up Logical Routing in NSX-T.

In this post we will learn about following:

• Inter-SR Architecture
• How to Enable Inter-SR routing
• Ingress/Egress traffic patterns
• Failure scenarios & remediation when an edge node losts northbound connectivity with upstream router

If you are new to NSX-T, then I would recommend reading my NSX-T series from below links:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

5: Configure Logical Routing in NSX-T

Let’s get started.

What is Tier-0 Inter-SR Routing?

Tier-0 gateway in active-active mode supports inter-SR iBGP. In active-active mode, the SR components form an internal connection between each other over a pre-defined NSX managed subnet 169.254.0.X/25. … Read More

In last post of my NSX-T 3.0 series, I briefly talked about Route Re-Distribution feature. In this post I will try to explain it in more detail. We will learn when this feature should be used and when not.

If you have missed my NSX-T 3.0 series, here are the links to the same:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

5: Configure Logical Routing in NSX-T

Let’s get started.

When a Tier-1 GW is attached to a Tier-0 GW, a router link between the 2 gateways is created automatically. You can consider this link as transit segment which connects T1 GW with T0.

Default address space that get assigned on this transit subnet is 100.64.0.0/16. Router ports on T0 & T1 get IP address 100.64.0.0/31 & 100.64.0.1/31 respectively.

 

A tier-0 gateway in active-active mode supports inter-SR (service router) iBGP.… Read More

In the last post of this series, we learned about transport nodes and how to setup a data plane. Now my NSX-T environment is ready for setting up logical routing and eventually starts flowing packets across the network.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this blog series:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

4: NSX-T Data Plane Setup

Let’s get started. 

What is Logical Routing?

NSX logical routing, enable us to connect both virtual and physical endpoints that are located in different logical Layer 2 networks. This is made possible by the separation of physical network infrastructure from logical networks that network virtualization provides.

Logical routing is provided by Logical Routers that get created on Edge Nodes when we configure routing. Logical Routers are responsible for handling East-West & North-South traffic across the datacenter.… Read More

In last post of this series, we learnt about Transport Zones and why we need them. We also discussed about Transport Node profiles and created a TN profile and couple of Transport Zones. 

This post is focussed on components involved in data plane and how to configure the same in NSX-T.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this blog series:

1: NSX-T Management & Control Plane Setup

2: Uplink Profiles in NSX-T

3: Transport Zones & Transport Node Profiles

Let’s get started.

What is meant by Data Plane in NSX-T?

The data plane is where all packet forwarding takes place based on tables created by the control plane. Packet level stats are found here as well as topology info which is then reported from the data plane up to the control plane.

Data plane in NSX-T comprises of 2 components: Hosts and the Edge nodes.… Read More