In a shared storage environment, when multiple hosts access the same VMFS datastore, specific locking mechanisms are used. These locking mechanism prevent multiple hosts from concurrently writing to the metadata and ensure that no data corruption occurs. This type of locking is called distributed locking.

To counter the situation of Esxi host crashing, distributed locks are implemented as lease based. An Esxi host that holds lock on datastore has to renew the lease for the lock time and again and via this Esxi host lets the storage know that he is alive and kicking. If an Esxi host has not renewed the locking lease for some time, then it means that host is probably dead.

If the current owner of the lock do not renews the lease for a certain period of time, another host can break that lock.  If an Esxi host wants to access a file that was locked by some other host, it looks for the time-stamp of that file and if it finds that the time-stamp has not been increased for quite a bit of time, it can remove the stale locks and can place its own lock.… Read More

What is Datastore Cluster?

A datastore cluster is a collection of datastores with shared resources and a shared management interface. Datastore clusters are to datastores what clusters are to hosts.

When you add a datastore to a datastore cluster, the datastores resources become part of the datastore cluster’s resources. Datastore clusters are used to aggregate storage resources, which enables you to support resource allocation policies at the datastore cluster level. Also datastore cluster provides following benefits:

• Space utilization load balancing
• I/O latency load balancing
• Anti-affinity rules

We will talk about these in greater details a bit later in this post.

Datastore Cluster Requirements

Before creating a datastore cluster, one should keep following important points in mind:

1: Datastore clusters must contain similar or interchangeable datastores: A datastore cluster can contain a mix of datastores with different sizes and I/O capacities, and can be from different arrays and vendors. However, the following types of datastores cannot coexist in a datastore cluster.… Read More

What is vSphere Flash Read Cache aka vFlash?

Flash Read Cache helps in accelerating virtual machine performance through the use of flash devices residing in Esxi host as a cache.

vFlash was first introduced in vSphere 5.5. It allows you to use local SSD disks of Esxi host to create a caching layer for your virtual machines. By using host local SSD’s, you can offload some of the IO from your SAN storage to these local SSD disks.

vFlash aggregates local flash devices into a pool and this pool is called “Virtual flash resource” (vFRC). For example if you have 3 x 60 GB SSD you end up with a 180 GB virtual flash resource. Each local SSD configured for vFRC is formatted with a filesystem called VFFS  aka “Virtual Flash File System”.

 

vFRC helps reducing lowering application latency as the read IO don’t have to go all the way down to SAN across all the physical network controllers/storage controllers etc and instea they just go to vFRC.… Read More

My first interaction with iSCSI and port binding was back in 2013 when we introduced an iSCSI based SAN (Dell MD3200i) in our environment. We were a small SMB entity and introduction of SAN for our vSphere environment was a very big thing for me as an administrator.

This was our architecture back then

I clearly remember that before starting the SAN implementation, I was contacted by Dell engineer to do some pre-work which included creating 2 vmkernel portgroup and each with only one vNIC as active and one as unused so as to achieve multipathing with iSCSI.

At that time I was aware of multipathing and what it does, but I was confused on Active/Unused adapter configuration (as I was still learning) and when the actual implementation started, the implementation guy explained it and that was first time when I heard the word Port Binding.

Its time to refresh the concepts now as I am going through my VCAP preparations.… Read More

When you create a new datastore in vSphere, each VMFS volume is assigned a unique identifier (UUID) and this UUID info is stored in a metadata file as unique hexadecimal number.

You can see these UUID via ssh console of Esxi host.  In the /vmfs/volumes directory, each VMFS volume has a long string and a human readable names (which we configure from GUI while creating a datastore) as links to the UUID.

The UUID is comprised of four components. Lets understand this by taking example of one of the vmfs volume’s UUID : 591ac3ec-cc6af9a9-47c5-0050560346b9

• System Time (591ac3ec)
• CPU Timestamp (cc6af9a9)
• Random Number (47c5)
• MAC Address – Management Port uplink of the host used to re-signature or create the datastore (0050560346b9)

In my example, the mac address was of the Management NIC of my first Esxi host

When a LUN is replicated or its snapshot is taken on the storage side, the copied LUN is identical to original one.… Read More

Raw Device Mapping aka RDM is a way for providing virtual machine direct access to LUN on the SAN storage. The LUN presented to VM can be then formatted with any filesystem like NTFS or FAT for Windows OS and thus there is no need to format the LUN with VMFS filesystem and then place a vmdk on it.

RDM can be think of as a symbolic link from a VMFS volume to the Raw LUN. When an RDM is mapped to a virtual machine, a mapping file is created.This mapping file acts as a proxy for the physical device and contains metadata used for managing and redirecting access to the raw disk.

When the virtual machine tries to access the LUN, the mapping file is read to obtain the reference to the raw LUN and then the reads and writes go directly to the raw LUN rather than going through the mapping file.… Read More

VMware vSphere hot add is a feature that allows vSphere administrators to increase the RAM and CPU capacity of a running virtual machine on the fly. This feature is one of my favourites (second to vMotion). Hot add of CPU is usually referred as hot-plug, but I will be using term hot add for both RAM & CPU.

What so special about hot-add?

Think of older times when a physical server used to run out of resources (cpu/memory) and the administrator has to do shutdown the server to increase the capacity of server. Shutting down of server itself was a big headache in those time as the administrator has to get approval from the application owner, create a change request for maintenance etc etc.

At that time if someone would have explained me that you can add CPU/RAM into a server in running condition, I would have laughed hard on him and may be I would have given him titles like mad,crazy and what not.… Read More

When a virtual machine is created, there are number of files that are created alongwith. The most important among them is the .vmx file which is the configuration file of the VM. Whatever settings we chose during deployment of virtual machine, is written in this file along with many other configuration settings.

Although most of the virtual machine settings can be edited via web client, we can modify the vmx file directly (not recommended though) also to change configuration settings or to add some extra configuration items.

When we edit the virtual machine settings from Web Client, those settings are written in vmx file of the vm. To do so the VM needs to be in powered-off state.

To modify the settings of a virtual machine, login to Web Client and select VM from the inventory and navigate to Manage > Settings > VM Options > Advanced settings and click on Edit button to make changes to the current configuration.… Read More