In a VCF environment, lifecycle operations are expected to be performed through SDDC Manager/VCF Operations. However, in real-world scenarios, operational mistakes occur—especially when components are modified directly in vCenter Server rather than through the VCF control plane.
In this blog, I will walk through a failure scenario involving NSX Advanced Load Balancer (Avi) in a VCF 9 environment, where Avi controller VMs were accidentally deleted from vCenter, leaving behind stale inventory references in SDDC Manager. I will cover the issue, impact, recovery approach, and key lessons.
The Problem Scenario
In a VCF-9 environment, Avi was deployed via SDDC Manager (integrated with NSX) and was running fine. During a troubleshooting session, Avi controller nodes were deleted directly from vCenter, leaving stale entries in the SDDC Manager UI and thus preventing the redeployment of Avi.
Result: SDDC Manager still believed Avi was deployed, and NSX integration references remained. The deletion option was grayed out in the SDDC manager UI.… Read the rest