Welcome to yet another troubleshooting post for tmc self-managed operation in VCD. In the last post, I discussed the tmc self-managed deployment issue and how I fixed it. In this post, I will discuss another issue that I encountered with the solution.
After successfully deploying TMC Self-Managed, you must publish the solution to tenants so that they can attach their TKG clusters to TMC. When the publishing operation is performed, the TMC Self-Managed Add-On solution creates a temporary VM known as the solution agent vm, which is subsequently destroyed once the task is complete.
In my lab, the publishing task was completed for a couple of tenants and later when I tried publishing it to another tenant, the task got stuck (VCD was acting cranky at that time).
This behavior is encountered when the solutions process in the VCD cell gets killed or there is network interruption between the cell and the VCD public address during the operation execution.
My previous blog post discussed the VCD Extension for Tanzu Mission Control and covered the end-to-end deployment steps. In this post, I will cover how to troubleshoot a stuck TMC self-managed deployment in VCD.
I was deploying TMC self-managed in a new environment, and during configuration, I made a mistake by passing an incorrect value for the DNS zone, leading to a stuck deployment that did not terminate automatically. I waited for a couple of hours for the task to fail, but the task kept on running, thus preventing me from installing it with the correct configuration.
The deployment was stalled in the Creating phase and did not fail.
On checking the pods in the tmc-local namespace, a lot of them were stuck in either ‘CreateContainerConfigError” or “CrashLoopBackOff” states.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
root@jumpbox:~# kubectl get po -n tmc-local | grep CreateContainerConfigError
In VCD, when I checked the failed task ‘Execute global ‘post-create’ action,” I found the installer was complaining that the tmc package installation reconciliation failed.… Read More
VMware Tanzu Mission Control is a centralized hub for simplified, multi-cloud, multi-cluster Kubernetes management. It helps platform teams take control of their Kubernetes clusters with visibility across environments by allowing users to group clusters and perform operations, such as applying policies, on these groupings.
VMware launched Tanzu Mission Control Self-Managed last year for customers running their Kubernetes (Tanzu) platform in an air-gapped environment. TMC Self-Managed is designed to support organizations that prefer to maintain complete control over their multi-cluster management hub for Kubernetes to take full advantage of advanced capabilities for cluster configuration, policy management, data protection, etc.
