Network IO Control in vSphere 6

In this post we will discuss about what is NIOC and why we need it. We will also configure NIOC in lab.

What is Network IO Control (NIOC)?

Network I/O Control (NIOC) was first introduced with vSphere 4.1 and it is a vDS feature that allows a vSphere administrator to prioritize different type of network traffic by making use of Resource pools and shares/limits etc. NIOC does the same for network tarffic which SIOC does for storage traffic.

What problem NIOC is solving?

In old days physical servers were equipped with as many as 8 (or more) ethernet cards and administrators (as a best practice) configured vSphere to use dedicated NIC for passing various network traffic like management traffic or vMotion or fault tolerance. Managing these many physical cables were a bit cumbersome.

Modern day servers addressed this issue by introducing servers with support for 10 GBPS/40 GBPS network speed and these servers have only 2 NIC’s and all the traffic is passed via these 2 NIC’s. Since all traffic types do not have same priority and at the time of contention, administrator needs the critical network traffic is not impacted by contention. NIOC enabled admins to address this issue by defining the priority of different traffic.

When enabled, Network NIOC categorizes traffic into network resource pools, which use resource allocation policies to control bandwidth for various traffic types. By default, a shares value of 50 is given to each pool, except for VM traffic, which gets a default of 100.

The administrator can adjust the shares value to each pool to determine how much bandwidth a pool gets to fulfill specific service levels. Pools with a higher shares number receive more physical network resources.

NIOC resource pools include:

Fault tolerance traffic
iSCSI traffic
vMotion traffic
Management traffic
vSphere Replication traffic
Network file system (NFS) traffic
Virtual machine (VM) traffic
vSphere Data Protection traffic
User-defined traffic

These resource pools are automatically applied to their corresponding traffic type when NIOC is enabled. NIOC utilizes the same type of sharing mechanism that resource pools utilize.

In vsphere 6, NIOC v3 was introduced which brought below enhancements with it

Enables bandwidth to be guaranteed at the virtual network interface on a virtual machine
Reservation set on the vNIC in the virtual machine properties
Applied at a Distributed Port Group
Enables bandwidth to be guaranteed to a specific VMware Distributed Switch port group
Reservation set on the VDS port group
Enables multi-tenancy on one VDS by guaranteeing bandwidth usage from one tenant won’t impact another

Requirements for using NIOC?

You need to have Enterprise plus license required because it uses vSphere Distributed Switch.
Network I/O control can be enabled only on VDS

Note: NIOC v3 is only supported on vDS v 6.0. If you have upgraded your infrastructure from vSphere 5.x, you have to upgrade the vDS to Version 6.0. Also NIOC v3 can be configured only from Web Client.

How to enable NIOC?

With vSphere 6, NIOC is enabled by default when creating new VDS. To manage/configure NIOC, login to vSphere Web Client and switch to networking view and select vDS > Manage > Resource Allocation > System traffic

You can see in screenshot that all traffic types are set to 50 shares except the VM traffic.

In my lab, I have total 1 GBPS bandwidth available, out of which max reservation allowed is 0.75Gbps. To modify share values of a traffic type, select the traffic type and click on pencil button to edit the share value.

You can set shares value as Low, Normal, High or Custom. No reservation or limits are set by default, but you can set it if you want that X amount of network bandwidth is always guranteed to the selected network traffic type.

Bandwidth Allocation for Virtual Machine Traffic

NIOC v3 enables administrators to configure bandwidth requirements for individual virtual machines. We can also use network resource pools to assign a bandwidth quota from the aggregated reservation for the virtual machine traffic and then allocate bandwidth from the pool to individual virtual machines.

To configure bandwidth allocation for individual VMs, edit the VM settings and expand network adapter settings to define the share/reservation/limit etc.

Shares: Shares, from 1 to 100, reflect the relative priority of a system traffic type against the other system traffic types that are active on the same physical adapter. The amount of bandwidth available to a system traffic type is determined by its relative shares and by the amount of data that the other system features are transmitting.
Reservation: The minimum bandwidth, in Mbps, that must be guaranteed on a single physical adapter. The total bandwidth reserved among all system traffic types cannot exceed 75 percent of the bandwidth that the physical network adapter with the lowest capacity can provide.
Limit: The maximum bandwidth, in Mbps or Gbps, that a system traffic type can consume on a single physical adapter.

Creating User-Defined Network Resource Pools

To guarantee bandwidth, Network I/O Control implements a traffic placement engine that becomes active if a virtual machine has bandwidth reservation configured. The distributed switch attempts to place the traffic from a VM network adapter to the physical adapter that can supply the required bandwidth and is in the scope of the active teaming policy. The total bandwidth reservation of the virtual machines on a host cannot exceed the reserved bandwidth that is configured for the virtual machine system traffic.

Custom resource pool can be created to reserve part of the aggregated bandwidth for VMs system trafic on all the physical adapters connected to the VDS.

To create new resource pool, select Network Resource Pools and clcik on green + button.

Provide a name and optional description for this pool and set reservation (if needed)

Once new network resource pool is created, you can new distributed port group or can edit existing portgroup to attach it to the newly created network resource pool and then assign VM’s to that portgroup so that bandwidth can be allocated to the VMs.