What is Multipathing?

Multipathing is having more than one path to storage devices from your server. At a given time more than one paths are used to connect to the LUN’s on storage device. It provides the ability to load-balance between paths when all paths are present and to handle failures of a path at any point between the server and the storage.

The vSphere host supports multipathing in order to maintain a constant connection between the server and the storage device, in case of failure that results in an outage of an HBA, fabric switch, storage controller, or Fibre Channel cable.

Multipathing support does not require specific failover drivers. However, in order to support path switching, the server does require two or more HBAs, from which the storage array can be reached by using one or, typically, multiple fabric switches.

Virtual machine I/O might be delayed for up to sixty seconds while path failover takes place. These delays allow the SAN to stabilize its configuration after topology changes. In general, the I/O delays might be longer on active-passive arrays and shorter on active-active arrays.

Multipathing Policies

There are three Multipathing Policies available in ESXi server:

1: Most Recently Used (MRU): Selects the first working path discovered at system boot time. If this path becomes unavailable, the ESXi host switches to an alternative path and continues to use the new path while it is available. There is no failback in this technique. If the original path which has failed, comes back online, Esxi host doesn’t use it. This is the default policy for LUNs presented from an Active/Passive array.

2: Fixed (Fixed): In this technique, Esxi host uses the designated preferred path (if it has been configured), Otherwise it uses the first working path discovered at system boot time. If the ESXi host cannot use the preferred path, it selects a random alternative available path. The ESXi host automatically reverts back to the preferred path as soon as the path becomes available. This is the default policy for LUNs presented from an Active/Active array.

3: Round Robin (RR): In this technique, Esxi host can use all available paths and thus enabling load distribution among configured paths. This technique can be used for both Active/Active or Active/Passive type storage array.

• For Active/Active all paths are used.
• For Active/Passive array, only those paths which are connecting to active controller can be used.

Native Multipathing Plugin (NMP)

This is the default multipathing plugin which provided by VMware and is included in Esxi host VMkernel.  NMP performs the following functions:

• Registers logical devices with the PSA framework
• Receives input/output (I/O) requests for logical devices it registered with the PSA framework
• Selecting the physical path to which it sends the I/O requests
• Handling failure conditions encountered by the I/O requests
• Handles task management operations, such as aborts/resets

NMP has 2 sub-plugins:

• Storage Array Type Plugin (SATP) : SATP keeps information about the available paths. It monitors the hardware state of the physical paths to the storage array. SATP initiates path failover when a path  is failed.
• Path Selection Policy (PSP): PSP dictates which path will be selected based on the multipathing technique used.

Storage Array Type Plug-Ins (SATPs) run in conjunction with the VMware NMP and are responsible for array-specific operations. ESXi offers a SATP for every type of array that VMware supports. It also provides default SATPs that support non-specific active-active and ALUA storage arrays, and the local SATP for direct-attached devices.

NMP communicates with SATPs for the following operations:

• Setting up a new logical device—claiming a physical path
• Updating the hardware states of the physical paths (for example, active, standby, dead)
• Activating the standby physical paths of an active/passive array (when the active path’s state is dead or unavailable)
• Notifying the plug-in that an I/O is about to be issued on a given path
• Analyzing the cause of an I/O failure on a given path

Pluggable Storage Architecture (PSA)

PSA is a special VMkernel module which gives Esxi hosts the ability to use third party multipathing software. Storage vendor provides their own multipathing plugin (MPP) which when installed on Esxi host, works together with NMP so that failover and load balancing for that storage array can be optimized.

The following diagram shows the architecture of PSA

Note: When NMP is used, Esxi host identifies the type of array by checking it against /etc/vmware/esx.conf file and then associates the SATP to that array based on the make and model.

Viewing and Changing Multipathing Policy 

To view and change the multipathing policy for a datastore, login to vSphere Web Client and navigate to storage view and  Select datastore > Setting > Connectivity and multipathing > host > Edit multipathing

As you can see in above screenshot, current multipathing policy that is being used by Esxi host in my lab is Fixed (VMware) and NMP is the owner of this multipathing policy. Also the default SATP for this VMW_SATP_DEFAULT_AA.

To view full list of VMware NMP SATPs that is loaded on Esxi host run command: esxcli storage nmp satp list

Display Multipathing Modules

Use esxcli storage core plugin list command to list all multipathing modules loaded into the system. Multipathing modules manage physical paths that connect your host with storage.

Changing Default Path Selection Policy

To change the path selection policy use command:

Listing Claim Rules

When an ESXi host boots or administrator rescans storage adapter, the Esxi host discovers all physical paths to storage devices available to the host. Based on a set of claim rules, the host determines which multipathing plug-in (MPP) should claim the paths to a particular device and become responsible for managing the multipathing support for the device.

To list all Multipathing Claim Rules for an Esxi host, run command : esxcli storage core claimrule list

In my lab, NMP is claiming all paths connected to storage devices.

To add a new claimrule please see this article

To remove claimrule from system, please see this article

Listing Device and Path Information

To list all devices present in system run command: esxcli storage nmp device list

Alternatively you can narrow down the list by specifying the storage device:

So you can see that above command is displaying same info about SATP and Path Selection Policy, which we saw in GUI earlier.

Listing Path information

To view all paths for an Esxi host to the storage device use command: esxcli storage core path list.  Use the ‘-d’ switch to specify a device for which you want to list paths

The above output shows that there are two paths to storage device and that both are active.  This matches with what we saw earlier in vSphere Web Client

I hope you find this post informational. Feel free to share this on social media if it is worth sharing. Be sociable 🙂