Last week I was facing a serious issue in my home lab where my esxi host is getting disconnected from my vCenter Server randomly. Whenever I am doing any configuration changes like enabling ssh or creating a new vSwitch the host got disconnected immediately. I was damn frustrated and was looking for a solution because it was very hard for me to work.

So I started troubleshooting by going through my vCenter log files and found following:

So I guess something wrong was happening related to heartbeat exchange between my host and vCenter server. I started my troubleshooting by following below steps:

1: Checked whether Esxi is able to reach my vCenter server or not by pinging and doing a telnet from Esxi host to vCenter Server on port 902

Note: Telnet command wont work in Esxi so you have to use “nc -z” command

So as you can see I  was able to reach my vCenter from my Esxi host successfully.

2: Next I checked whether or not my Esxi host is listening on port 902 (heartbeat port)

The above command verified yes my host is listening on port 902

4: I added the host disconnection timeout string in Advance Settings of vCenter and increased the value to 120

I verified once again that value has been added.

4: Next I check my vCenter Server for “Managed IP Setting”. Sometimes if the vCenter IP is not listed then also you can face this issue.

In my case I manually entered IP under Run Time Settings as shown in above image.

5: I checked the same settings on my Esxi host.

So from above image it is pretty clear that my Esxi host is configured to managed by correct vCenter server.

6: Next I checked for Heartbeat Port Value on my Esxi host by running the command:

# grep -i serverport /etc/vmware/vpxa/vpxa.cfg

The output which I got was something strange as my Esxi host was using port 922 for heartbeats exchange instead of using default port 902.

According to VMware KB Article 2040630

This issue is caused by dropped, blocked, or lost heartbeat packets between the vCenter Server and the ESXi/ESX host. If there is an incorrect configuration of the vCenter Server managed IP address, the host receives the heartbeat from vCenter Server but cannot return it.

It is important to remember that the default heartbeat port is UDP 902, and these packets must be sent between vCenter Server and the ESXi/ESX host for the host to stay connected and remain in the vCenter Server inventory.

I changed the port to 902 by editing the vpxa.cfg file and removed and added back my Esxi host to vCenter Server and hoped that my issue is now resolved. But surprisingly I was still getting the disconnection problem. Once again I connected my Esxi host using ssh and checked vpxa.cfg file and found the port has been again changed to 922. This was strange.

On digging more I found that this is happening because of heartbeat port specified as 922 in the registry key of vCenter server. I got this clue from one of the issue 2437489 posted in VMware Community group.

The full registry key is :

HKEY_LOCAL_MACHINESOFTWAREVMware, Inc.VMware VirtualCenter

As you can see in above image the heartbeat port is 922 which is causing all the troubles. I changed it to 902 and restarted my vCenter Server and bingo my issue is resolved.

Hit Like and share on social media if above information is helpful to you. Happy Learning!!!