Deploying TKG 2.0 Clusters in TKGS on vSphere 8 to Trust Private CA Certificates

In a vSphere with Tanzu environment, when you enable Workload Management, the Supervisor cluster that gets deployed operates as the management cluster. After supervisor cluster is deployed, you can provision two types of workload clusters

  • Tanzu Kubernetes clusters.
  • Clusters based on a ClusterClass (aka Classy Cluster). 

TKG on vSphere 8 provides different sets of APIs to deploy a TKC or a Classy cluster:

When you deploy a cluster using v1beta1 API, you get a Classy Cluster or TKG 2.0 cluster which is based on a default ClusterClass definition.

By default workload cluster don’t trust any self-signed certificates. Prior to TKG 2.0 clusters, the easiest way to make Tanzu Kubernetes Clusters (TKCs) to trust any self-signed CA certificate was to edit tkgserviceconfigurations and define your Trusted CAs there. This setting was then enforced on any newly deployed TKCs.

For TKG 2.0 clusters, the process has changed a bit and in this post I will walk through configuring the same. I am demonstrating example of trusting self-signed certificate of internal harbor registry in this example.

Step 1: Create a Kubernetes secret definition YAML file

Create a yaml file to store your kubernetes secret for the additional CA certificates that your workload cluster should trust.

You have to customize the yaml file as per below: 

  • namespace: This is the name of the vSphere Namespace in which you will deploy your workload cluster.
  • name: This is a very important field. Any mistake in the name will lead to the cluster deployment failure. The name of the secret should include your workload cluster name followed by a fix value ‘user-trusted-ca-secret’. For e.g. if the name of your workload cluster that you will deploy is tkc01, the corresponding secret name will be ‘tkc01-user-trusted-ca-secret’
  • data: This name can be any user defined name whose value is a double base64-encoded certificate. 

Important: If the contents of the data map value are not double base6-encoded, the resulting PEM file cannot be processed.

So what do we mean by double encoding here? It simply means that you have to encode your CA certificate twice. You can run a simple command to do double encoding: cat ca.crt | base64 | base64 where ca.crt is the file that contains certificate contents of your root CA.

Step 2: Switch the context to the vSphere namespace and create the secret.

Step 3: Create a TKG 2.0 cluster deployment yaml

You can refer to the TKGS product documentation to see the examples listed for TKG 2.0 cluster creation. A sample yaml file is shown below for reference.

Important: Name of the additional trusted CA should match with the name that you have specified under the field ‘data’ in the Kubernetes secret definition file created in step1.

Step 4: Create Workload cluster

Step 5: Create a registry secret for kapp-controller

Kapp-controller is set up automatically when you deploy TKG 2.0 workload clusters in Supervisor running on vSphere 8. The kapp-controller must also trust the CA certificate of the internal harbor repo in order to deploy packages and applications from it. To activate trust for kapp, follow the instructions below.

5.1: Login to the TKG 2.0 workload cluster

5.2: Get the configMap name of the kapp-controller

5.3: Create secret for kapp-controller

Before executing the below command, create a file named registry_certs‘ and paste the certificate contents of your harbor CA certificate.

Important: Don’t use any other name for the certificate file, otherwise the operation would fail.

Now that the workload cluster has been installed, you can deploy applications and tanzu packages from your private repository.

I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.

Leave a ReplyCancel reply