Reset Cloud Builder DB for a Fresh Bringup (VCF)

Posted on 16-06-2020 by Manish Jha

Many of us might have encountered this situation where Cloud Builder UI doesn’t provide a back button after a successful SDDC bringup. I have written a blog Post in the past where I showed the API method of resetting CB for a new bringup.

This post is an extension of my earlier post. Here I will demonstrate the database hack method to initiate a fresh bringup.

Step 1: Login to Cloud Builder via SSH and switch to root user.

Step 2: Connect to the Database

# psql -U postgres -d bringup -h /home/postgresql/

1	# psql -U postgres -d bringup -h /home/postgresql/

Step 3: Nuke the Bringup related tables

bringup=# delete from execution;

bringup=# delete from "Resource";

bringup=# \q

bringup=# delete from execution;

bringup=# delete from "Resource";

bringup=# \q

Refresh the CB UI and you will be returned to the checklist page.… Read More

Retry Failed Bringup with Modified Input Spec in VCF

Posted on 04-06-2020 by Manish Jha

Those who are experienced with VCF might have seen this situation in past where a SDDC Bringup task fails because of an incorrect parameter in the json/xls file used for bringup.

Cloud Builder UI don’t provides a mechanism to go back and change the input file with correct values and retry bringup with modified file. Don’t loose heart if you are caught in this situation as it is possible to do this via API and in this post I will walk through steps of doing so.

By default cloud builder places the sddc-bringup json in /opt/vmware/sddc-support/cloud_admin_tools/Resources/vcf-ems/ directory by name vcf-ems.json.

Note: Even if you are using xls to feed cloud builder, it is internally converted in json file.

You can either modify this json directly and fix the incorrect value that was supplied in your xls/json file or you can generate a new json for bringup.

Step 1: Login to cloud builder vm via ssh and switch to root user.… Read More

Troubleshooting vRSLCM Deployment Failure in VCF

Posted on 04-06-2020 by Manish Jha

Last week while working in my VCF lab, I faced an issue with vRSLCM deployment. Deployment failed in step where SDDC Manager tries to configure vRSLCM NTP settings.

Started my troubleshooting with checking domainmanager.log on sddc manager appliance. I saw following log entries:

2020-05-30T14:44:37.013+0000 INFO [847f3151af714e35,a502] [c.v.e.s.v.v.ImportNtpSettingInVrslcm,dm-exec-4] Importing NTP Server 10.84.x.x in vRSLCM with name VCF NTP Server 1

2020-05-30T14:44:47.189+0000 DEBUG [847f3151af714e35,a502] [c.v.e.s.r.c.LoggingHttpRequestInterceptor,dm-exec-4] Request URI: https://vrslcm.vstellar.local/lcm/lcops/api/settings/productntpsetting
Request method: POST
Request body: {"name":"VCF NTP Server 1","hostName":"10.84.x.x"}
Response code: 400 BAD_REQUEST

2020-05-30T14:45:17.923+0000 ERROR [847f3151af714e35,f1d1] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-10] [L5UVJG] INTERNAL_SERVER_ERROR Invocation of prefix '' part of task ImportNtpSettingInVrslcm in plugin VrslcmPlugin failed with exception.
com.vmware.evo.sddc.common.core.error.InternalServerErrorException: Invocation of prefix '' part of task ImportNtpSettingInVrslcm in plugin VrslcmPlugin failed with exception.

Caused by: org.springframework.web.client.HttpClientErrorException$BadRequest: 400 : [{"status":"ERROR","statusCode":"BAD_REQUEST","message":"Please check if the provided IP/FQDN is of an NTP server and is reachable.","resourceIdentifier":null,"errorStackTrace":["com.vmware.vrealize.lc... (13471 bytes)]

2020-05-30T14:44:37.013+0000 INFO [847f3151af714e35,a502] [c.

Dealing With MyVMware Credentials shenanigan in VCF

Posted on 30-05-2020 by Manish Jha

Recently while working in my lab, I deployed a proxy server and configured my SDDC Manager to use that proxy server to talk to internet.

SDDC Manager was happily talking to internet, but when I tried configuring MYVMware credentials to download product binaries, operation was failing with error “Updating depot user credentials failed”

Tried configuring credential via API (as UI sometime acts weird), but operation failed again.

curl http://localhost/lcm/depot/user -H ‘Content-Type: application/json’ -X PUT -d ‘{“userId”:”xxx@vmware.com”, “password”:”<PASSWORD>”}’

and got error 500 in response

{
"status":500,
"code":"VMWARE_DEPOT_CONNECT_FAILURE",
"message":"Depot Not Available"
}

{

"status":500,

"code":"VMWARE_DEPOT_CONNECT_FAILURE",

"message":"Depot Not Available"

}

On further investigation I found that the issue was with proxy server itself as it was not trusting certificate returned by depot.vmware.com and thus causing lcm-bundle-transfer to fail.

Google search returned nothing for this issue and I had to reach out to VCF engineering team.… Read More

Deploy Edge Cluster With Custom Form Factor in VCF via API

Posted on 30-05-2020 by Manish Jha

Recently while working in my lab, I was facing an issue with edge cluster deployment in workload domain. Edge Cluster deployment was failing with below error:

2020-05-30T05:37:17.920+0000 ERROR [9691f5ba228f6be5,531e] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-5] [F50Q9E] INPUT_PARAM_ERROR Some hosts in cluster(s) 81bc0275-1a09-455d-91fd-3fce591cf500 do not support 8 vCPUs for edge form-factor LARGE
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Some hosts in cluster(s) 81bc0275-1a09-455d-91fd-3fce591cf500 do not support 8 vCPUs for edge form-factor LARGE

2020-05-30T05:37:17.920+0000 ERROR [9691f5ba228f6be5,531e] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-5] [F50Q9E] INPUT_PARAM_ERROR Some hosts in cluster(s) 81bc0275-1a09-455d-91fd-3fce591cf500 do not support 8 vCPUs for edge form-factor LARGE

com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Some hosts in cluster(s) 81bc0275-1a09-455d-91fd-3fce591cf500 do not support 8 vCPUs for edge form-factor LARGE

Since my lab is a nested lab, I bumped cpu on Esxi hosts to 10 and retriggered the task, but deployment was still failing with same error.… Read More

How to Reuse VCF Cloud Builder For Another Bringup

Posted on 14-05-2020 by Manish Jha

Those who are working on VCF are aware of the fact that, after a successful bringup, the only option you get is to launch SDDC Manager. There is no option in UI to go back to cloud builder initial screen where you can upload configuration workbook xls/json to start a fresh bringup.

One work around to this problem is to take snapshot of cloud builder before bringup and revert snapshot after succesful bringup to start again.

But this task seems very tedious to me. If you are running a huge infrastructure and performs bringup almost every day, this will not be an ideal situation.

In this post I will explain how to get rid of this annoying problem.

This can be achieved by running an internal one liner API call on cloud builder vm as root user.

curl -X GET http://localhost:9080/bringup-app/bringup/sddcs/test/deleteAll

1	curl -X GET http://localhost:9080/bringup-app/bringup/sddcs/test/deleteAll

Just refresh the cloud builder ui after running above API call and job done. … Read More

How To Verify if SDDC Bringup is AVN Enabled in VCF?

Posted on 11-05-2020 by Manish Jha

Recently I saw an interesting thread on slack where one of the consultant was seeing deploy button grayed out for VRSLCM deployment. Consultant was not aware if the management domain bringup was AVN enabled or not.

AVN was first introduced in VCF 3.9 and it enabled deploying vRealize components on logical segments instead of VLAN backed portgroup. AVN brought many enhancements with Cloud Foundation. Please read this Article to know more about AVN.

Since AVN enablement is optional and you can chose not to use it during SDDC bringup This is controlled via a parameter in VCF configuration workbook and ca be enabled later also.

Now the question is how to verify whether or not a SDDC bringup is AVN enabled bringup. There are couple of ways to verify this.

1: Through PowerVCF

PowerVCF is a PowerShell module to interact with SDDC Manager and the VCF stack. If you have PowerVCF is installed, you can connect to sddc-manager using below command:

# Connect-VCFManager <sddc-manager-fqdn> <sso-credentials>

and then run command: Get-VCFApplicationVirtualNetwork

PS C:UsersAdministrator> Connect-VCFManager sddc-manager.vstellar.local

Deploying vSphere with Kubernetes via VCF 4.0

Posted on 09-05-2020 by Manish Jha

In this post I will walk through how to deploy a Kubernetes cluster in a workload domain in VCF. This is a new feature that is introduced in VCF 4.0. vSphere with Kubernetes is also known as Project Pacific and Cormac Hogan did a great job in explaining nuances of Kubernetes in his Article

Before deploying a Kubernetes cluster, there are few prerequisites that must be met:

1: A NSX-T backed workload domain deployed.

2: Dedicated Edge Cluster deployed for workload domain. I have covered steps of deploying edge cluster Here

3: All Esxi hosts that are part of workload domain are licensed with “VMware vSphere 7 Enterprise Plus with Add-on for Kubernetes” license.

4: Subnets for Kubernetes cluster egress/ingress traffic created on your ToR.

Once above prerequisites, we are good to go with deployment. Let’s jump into lab and walk through deployment steps.

To deploy Kubernetes cluster, login to SDDC Manager and navigate to Home > Solutions.… Read More

How To Safely Remove Edge Cluster from Workload Domain in VCF

Posted on 07-05-2020 by Manish Jha

Recently I deployed an edge cluster in my lab for NSX-T based workload domain. In VCF 4.0, a new feature was added for automated deployment of edge cluster. If you haven’t tried this yet, you can follow my previous Post to learn more about this.

My edge cluster encountered some routing issues because of incorrect BGP information passed during edge cluster deployment workflow and I was looking for how to remove it cleanly from sddc manager.

While searching on internet, I came across VMware KB-78635 which lists the instructions of doing so. In this post I will walk through steps from the above KB.

Edge Cluster removal workflow is not baked into sddc manager yet and is carried out by python scripts provided by VMware. May be in next release of VCF, we will see this option added to sddc manager 😉

Edge cluster python script is named “edge_cluster_cleaner” and you will find this script as tar file in attachment section of KB-78635.… Read More

Automated NSX-T Edge Cluster Deployment in VCF 4.0

Posted on 06-05-2020 by Manish Jha

NSX-T integration with VCF is there for quite some time. Till VCF 3.9, SDDC Manager was not deploying NSX-T edge cluster in workload domains as part of VI WLD creation. Process of edge cluster deployment was manual and very well documented Here

Also this article by Cormac Hogan is worth checking to understand things in greater details.

In VCF 4.0, edge cluster deployment task is automated and can be kicked directly from SDDC Manager itself. No more juggling around configuring various things in NSX-T.

Below are high level steps for Automated Edge Cluster Deployment in VCF 4.0.

Step 1: Login to SDDC Manager and navigate to Workload Domains tab and click on environment where you want to deploy NSX-T Edge Cluster.

Form Actions Menu, click on Add Edge Cluster

Step 2: A list of prerequisites will pop up. Make sure your infrastructure has met all these requirements before moving ahead.

Word of caution here, make sure VLAN used for Host and Edge TEP’s are different.… Read More

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love