I encountered an issue while playing with Container Service Extension 3.1.1 in my lab where I was unable to construct TKGm clusters. During troubleshooting, I discovered that the Rights Bundle “cse:nativeCluster Entitlement” was missing certain critical rights that are newly added with CSE 3.1.1.
On attempting to delete the failed clusters, the clusters stuck in the state “DELETE:IN_PROGRESS”.
On attempting to delete the failed cluster via vcd-cli, the operation failed with the error “RDE_ENTITY_NOT_RESOLVED“
|
1 2 3 4 5 6 |
[root@cse ~]# vcd cse cluster delete mj-tkc01 --yes Usage: vcd cse cluster delete [OPTIONS] [NAME] Try 'vcd cse cluster delete -h' for help. Error: [ 0870ce65-3c19-4faa-a483-9ff52f4e9f06 ] RDE_ENTITY_NOT_RESOLVED [ 556ce860-1b72-4365-b327-3b38af622eca ] |
Further investigation revealed that the stuck clusters can only be removed via API. The instructions below demonstrate how to delete stuck clusters.
Step 1: Find Cluster ID
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
GET https://{{vcd_host}}/cloudapi/1.0.0/entities/interfaces/vmware/k8s/1.0.0 Headers: Accept: application/json;version=36.0, Content-Type: application/json Authorization: Bearer Token: <X-VMWARE-VCLOUD-ACCESS-TOKEN> Response Output: ----------snipped--------------------- { "id":"urn:vcloud:entity:cse:nativeCluster:009931a6-8ff6-4418-bb8a-25d596d2084b", "entityType":"urn:vcloud:type:cse:nativeCluster:2.0.0", "name":"tkg14-tkc01", "externalId":"https://vcd.manish.lab/api/vApp/vapp-11643552-6776-4485-8e71-70c8093f8f61" } |
Make a note of the field ‘Id’ in the output. That is the cluster id.
Alternatively, Cluster-ID can be fetched from VCD UI by clicking on the stuck cluster and taking a note of the urn.
Step 2: Set the Stuck Cluster Status to Resolve
|
1 2 3 4 5 6 7 8 9 |
Method: POST URL: https://{{vcd_host}}/cloudapi/1.0.0/entities/<cluster-id>/resolve Examle: https://{{vcd_host}}/cloudapi/1.0.0/entities/urn:vcloud:entity:cse:nativeCluster:556ce860-1b72-4365-b327-3b38af622eca/resolve Headers: Accept: application/json;version=36.0 Authorization: Bearer Token: <X-VMWARE-VCLOUD-ACCESS-TOKEN> |
Step 3: Delete Stuck Cluster
|
1 2 3 4 5 6 7 |
Method: DELETE URL: https://{{vcd_host}}/cloudapi/1.0.0/entities/<cluster-id>?invokeHooks=false Headers: Accept: application/json;version=36.0 Authorization: Bearer Token: <X-VMWARE-VCLOUD-ACCESS-TOKEN> |
When you refresh the VCD UI, the stuck cluster will be gone.
I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.