I encountered an issue while playing with Container Service Extension 3.1.1 in my lab where I was unable to construct TKGm clusters. During troubleshooting, I discovered that the Rights Bundle “cse:nativeCluster Entitlement” was missing certain critical rights that are newly added with CSE 3.1.1.
On attempting to delete the failed clusters, the clusters stuck in the state “DELETE:IN_PROGRESS”.
On attempting to delete the failed cluster via vcd-cli, the operation failed with the error “RDE_ENTITY_NOT_RESOLVED“
|
1 2 3 4 5 6 |
[root@cse~]# vcd cse cluster delete mj-tkc01 --yes Usage:vcdcseclusterdelete[OPTIONS][NAME] Try'vcd cse cluster delete -h'forhelp. Error:[0870ce65-3c19-4faa-a483-9ff52f4e9f06]RDE_ENTITY_NOT_RESOLVED[556ce860-1b72-4365-b327-3b38af622eca] |
Further investigation revealed that the stuck clusters can only be removed via API. The instructions below demonstrate how to delete stuck clusters.
Step 1: Find Cluster ID
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
GEThttps://{{vcd_host}}/cloudapi/1.0.0/entities/interfaces/vmware/k8s/1.0.0 Headers:Accept:application/json;version=36.0,Content-Type:application/json Authorization:BearerToken:<X-VMWARE-VCLOUD-ACCESS-TOKEN> ResponseOutput: ----------snipped--------------------- { "id":"urn:vcloud:entity:cse:nativeCluster:009931a6-8ff6-4418-bb8a-25d596d2084b", "entityType":"urn:vcloud:type:cse:nativeCluster:2.0.0", "name":"tkg14-tkc01", "externalId":"https://vcd.manish.lab/api/vApp/vapp-11643552-6776-4485-8e71-70c8093f8f61" } |
Make a note of the field ‘Id’ in the output. That is the cluster id.
Alternatively, Cluster-ID can be fetched from VCD UI by clicking on the stuck cluster and taking a note of the urn.
Step 2: Set the Stuck Cluster Status to Resolve
|
1 2 3 4 5 6 7 8 9 |
Method:POST URL:https://{{vcd_host}}/cloudapi/1.0.0/entities/<cluster-id>/resolve Examle:https://{{vcd_host}}/cloudapi/1.0.0/entities/urn:vcloud:entity:cse:nativeCluster:556ce860-1b72-4365-b327-3b38af622eca/resolve Headers:Accept:application/json;version=36.0 Authorization:BearerToken:<X-VMWARE-VCLOUD-ACCESS-TOKEN> |
Step 3: Delete Stuck Cluster
|
1 2 3 4 5 6 7 |
Method:DELETE URL:https://{{vcd_host}}/cloudapi/1.0.0/entities/<cluster-id>?invokeHooks=false Headers:Accept:application/json;version=36.0 Authorization:BearerToken:<X-VMWARE-VCLOUD-ACCESS-TOKEN> |
When you refresh the VCD UI, the stuck cluster will be gone.
I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.