Those who are familiar with VMware Cloud Foundation (VCF) are aware that deploying the management domain requires a minimum of 4 ESXi hosts. In a production environment, this is not a problem, but in resource-crunched Lab/PoC environments, it is very difficult to deploy the full-fledged management domain.

In this blog post, I am going to demonstrate how to deploy a VCF management domain with just a single ESXi host in a nested lab. This tidbit will be very helpful for the folks who want to test VCF but don’t have adequate resources available in the lab.

I am following VCF 5.1.1 BOM for my deployment. I will cover resource requirements as well as touch base on the nested ESXi configuration. Let’s get started!!!

Nested ESXi Configuration

1: Resource Allocation

The nested ESXi was deployed with 96 GB Memory and 14 CPUs, 2 network adapters, and 1000 GB SSD storage. Although 96 GB is a bit less especially if you plan to deploy NSX Edges later after the SDDC bringup. At the end of this post, I will discuss another workaround you can follow for deploying the edge VMs.

2: Configure Services

Ensure that you configure NTP settings on the ESXi host and use the same NTP server that you have configured in the cloud builder machine.

Note: You deploy your cloud builder outside of the nested ESXi where you want to deploy the management domain. 

Set the NTP server startup policy to “Start and stop with host”

Start the SSH service and set the service startup policy to “Start and stop with host”

3: Network Configuration

Navigate to the Networking settings of the host and set the VLAN tag of your infrastructure management network on the “VM Network” portgroup. 

Note: Your ESXi management network can be different than the infra network. In that case, your ESXi management network should be routable to the infra network. 

Optionally you can set MTU of the vSwitch to 9000 (if your environment supports jumbo frames). Also, ensure that vSwitch0 has only one active physical adapter and the other network adapter is free and both network adapters are connected to the Trunk portgroup on your physical ESXi host.

4: Certificate Configuration

By default, when you deploy an ESXi host, the self-signed certificate common name (CN) points to localhost. local domain. The cloud builder expects ESXi fqdn configured in the Common name and if it doesn’t find the fqdn, the SDDC bringup validation fails. You can execute the following commands to regenerate the ESXi SSL certificates.

After re-generating the SSL certs, validate that the ESXi host certificates reflect the FQDN in its Common Name.

5: Modify Cloud Builder

There is a configuration change in Cloud Builder that allows deploying a management domain with a single ESXi host and setting up a one-node vSAN cluster. This configuration change relaxes the requirement of adding additional ESXi nodes for a full-fledged cluster to provide high availability and redundancy. 

Also, you can instruct the cloud builder to deploy a single NSX manager node instead of 3 nodes, and that too in a small form factor by configuring the following parameters and restarting the bringup service.

Note: You can also control the number of NSX nodes and their size in the json file used for the deployment. The next steps show how to generate the json file.

6: Generate a Custom JSON file

After you have filled the VCF deployment configuration workbook, upload it to the cloud builder vm and invoke the sos command to generate the corresponding json file for it.

Note: You need to run the SOS command as root user.

7: Modify the Generated JSON

Download the json file that you generated in the previous step and modify the following sections:

Remove the references to the additional ESXi hosts under the “hostSpecs” section.

Next, append the following entry “hostFailuresToTolerate”: 0 under the clusterSpec section.

Finally, modify the “nsxtSpec” section to specify the nsx manager size and number of nodes (if you did not do this previously in Step 5)

And you are good to go now.

For the reference, I have pasted the full json file from my environment in the below section

Show More

8: Initiate SDDC Bringup

Login to the Cloud Builder appliance UI and select the VMware Cloud Foundation under Supported Platform.

Review the prerequisites for the SDDC deployment and click next.

Upload your modified json file and click Next.

Cloud Builder will initiate the validation against the json file that you uploaded.  In my lab, there was one warning and I ignored it by clicking on the Acknowledge button as I knew it was a non-fatal warning.

Once the validation is completed and all tests are passed, you can initiate SDDC deployment.

After the cloud builder has completed the SDDC deployment, click on the finish button.

Cloud Builder will then prompt for the launch of the SDDC manager. 

After logging in to the SDDC manager, you can perform further configuration of the SDDC under the Guided Setup page.

This is what a one-node VCF management domain looks like.

Final Words

You can further tweak this setup to allow deployment of NSX Edge nodes. You can follow the following steps:

1: Edit the NSX Manager node and remove the memory and CPU reservations to free up some resources.

2: Power off the SDDC manager VM and deploy NSX Edge nodes directly from the NSX manager UI.

3: Deploy NSX Edge nodes in small form-factor and remove CPU/memory reservations during the deployment.

This is how my lab looks finally. I was even able to deploy Avi Controller post tweaking the lab. 

A big shout out to my friend Roshan for lending resources from his lab to test this setup. Roshan also provides paid on-demand labs and if you have any lab requirements, feel free to contact him using this url: https://virtualrove.com/tag/on-demand-labs/

I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.