Install Container Service Extension 4.2 in an Airgap Environment

16-02-2024 Manish Jha

Introduction

VMware Container Service (CSE) is an extension of VMware Cloud Director that enables cloud providers to offer Kubernetes as a service to their tenants. CSE helps tenants quickly deploy the Tanzu Kubernetes Grid clusters in their virtual data centers with just a few clicks directly from the tenant portal. Tenants can manage their clusters utilizing Tanzu products and services, such as Tanzu Mission Control, in conjunction with the VMware Cloud Director Container Service Extension.

Until CSE 4.0, the deployment of TKG clusters depended on internet connectivity to get the necessary installation binaries from the VMware public image registry. There was no support for the airgap environment.

With CSE 4.1, VMware introduced support for deploying CSE in an Airgap environment. Before diving into the nitty-gritty of configuring CSE, let’s look at the CSE airgap architecture.

CSE Airgap Architecture

The image below is from the CSE product documentation and depicts the high-level architecture and service provider workflow of CSE in an airgap setup.… Read More

Container Service Extension 4.0 on VCD 10.x – Part 4: Tenant Operations

22-03-202322-03-2023 Manish Jha

In the previous post in this series, I discussed the CSE configuration options that a service provider can use to provide Kubernetes-as-a-service to their tenants. In this post, I’ll go over how tenants can use the Container Service Extension plugin for Kubernetes cluster deployment in a self-service manner.

If you haven’t read the previous posts in this series, you can do so by clicking on the links provided below.

1: CSE Introduction & Architecture

2: NSX Advanced Load Balancer Configuration & VCD Integration

3: Container Service Extension Configuration by Service Provider

Log in to the tenant’s org to deploy a Kubernetes cluster. The user should be assigned the “Kubernetes Cluster Author” role. To begin with the cluster creation wizard, navigate to Home > More > Kubernetes Container Clusters and click the New button.

Select the Kubernetes runtime for the cluster. CSE 4.0 only supports Tanzu Kubernetes Grid runtime.

Choose the Kubernetes version and give the Kubernetes cluster a name.… Read More

Container Service Extension 4.0 on VCD 10.x – Part 3: Service Provider Configuration

03-02-2023 Manish Jha

The first two posts in this series covered CSE architecture and NSX ALB deployment/configuration. This post focuses on the steps taken by a service provider to set up a CSE deployment.

You can read the previous posts in this series by clicking on the links provided below.

1: CSE Introduction & Architecture

2: NSX Advanced Load Balancer Configuration & VCD Integration

At this time, it is assumed that the Service Provider has completed the following configurations in VCD:

vCenter is registered in VCD.
NSX-T is registered in VCD.
A Geneve-backed network pool is created in VCD.
Provider VDC has been created.

The service provider workflow for CSE deployment includes the following tasks:

Import Tier-0 gateway/VRF that is created for CSE in NSX-T.
Create an organization in VCD. This is a Service Provider managed organization that hosts the Container Service Extension server and any other extensions in the future. This is known as a Service/Solutions organization.

Container Service Extension 4.0 on VCD 10.x – Part 2: NSX Advanced Load Balancer Configuration

27-01-202302-02-2023 Manish Jha

In part 1 of this blog series, I discussed Container Service Extension 4.0 platform architecture and a high-level overview of a production-grade deployment. This blog post is focused on configuring NSX Advanced Load Balancer and integrating it with VCD.

I will not go through each and every step of the deployment & configuration as I have already written an article on the same topic in the past. I will discuss the configuration steps that I took to deploy the topology that is shown below.

Let me quickly go over the NSX-T networking setup before getting into the NSX ALB configuration.

I have deployed a new edge cluster on a dedicated vSphere cluster for traffic separation. This edge cluster resides in my compute/workload domain. The NSX-T manager managing the edges is deployed in my management domain.

On the left side of the architecture, you can see I have one Tier-0 gateway, and VRFs carved out for NSX ALB and CSE networking.… Read More

Container Service Extension 4.0 on VCD 10.x – Part 1: Introduction & Architecture

26-01-202327-01-2023 Manish Jha

Introduction

VMware Container Service is an extension to VMware Cloud Director which enables cloud providers to offer Kubernetes-as-a-Service to their tenants. CSE helps tenants quickly deploy the Tanzu Kubernetes Grid clusters in their virtual data centers in just a few clicks directly from the tenant portal. By using VMware Cloud Director Container Service Extension, customers can also use Tanzu products and services such as Tanzu Mission Control to manage their clusters.

Container Service Extension (CSE) has come a long way and with each release, the product is getting better and better. Folks who have worked on the older versions of CSE knows how painful it the setup process was and involved too many manual steps. With CSE 4.0, the provider workflow is simplified and the installation can be done in less than 30 minutes. Kudos to the CSE engineering team.

CSE 4.0 Benefits

I want to list a few benefits that CSE 4.0 offers before getting into the architecture.… Read More

Error Deploying Container Service Extension 3.1.1 – No module named ‘_sqlite3’

28-12-202127-02-2022 Manish Jha

Container Service Extension 3.1.1 was released a few days back with new enhancements. The release announcements were made here and here.

Although the deployment procedure hasn’t changed much, mine was not smooth and I faced a couple of hiccups. This blog post discusses the problem I experienced and how I resolved it.

After installing VCD-CLI using pip, I was unable to execute any VCD command. The command was throwing an error as shown below:

[root@cse ~]# vcd version

Traceback (most recent call last):
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 18, in <module>
    from pysqlite2 import dbapi2 as sqlite3
ModuleNotFoundError: No module named 'pysqlite2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/bin/vcd", line 5, in <module>
    from vcd_cli.vcd import vcd
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/vcd.py", line 121, in <module>
    from vcd_cli import login  # NOQA
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/login.py", line 24, in <module>
    from vcd_cli import browsercookie
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 20, in <module>
    import sqlite3
  File "/usr/local/lib/python3.7/sqlite3/__init__.py",

Native Kubernetes in VCD using Container Service Extension 3.0

20-09-202127-02-2022 Manish Jha

Introduction

VMware Container Service is an extension to Cloud Director which enables VCD cloud providers to offer Kubernetes-as-a-Service to their tenants. CSE integration with VCD has allowed CSPs to provide true developer-ready cloud offering to VCD tenants. Tenants can quickly deploy the Kubernetes cluster in just a few clicks directly from the VCD portal.

Cloud Providers upload customized Kubernetes templates in public catalogs which tenants leverage to deploy K8 clusters in self-contained vApps. Once the K8 cluster is available, developers can use their native Kubernetes tooling to interact with the cluster.

To know more about the architecture and interaction of CSE components, please see my previous blog on this topic.

Container Service Extension 3.x went GA earlier this year and brought several new features/enhancements and one of them is supporting Tanzu Kubernetes Grid multi-cloud (TKGm) for K8 deployments and thus unlocking the full potential of consistent upstream Kubernetes in their VCD powered clouds.… Read More

VCD Container Service Extension Series-Part 4: Tenant Onboarding & K8 Cluster Deployment

01-07-202017-03-2022 Manish Jha

In last post of this series, we learn how to install and integrate CSE plugin with VCD for easier management of Kubernetes container. In this post we will learn how tenants can leverage CSE plugin to deploy K8 clusters.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this series.

1: Container Service Extension Introduction & Architecture

2: CSE Server Installation

3: CSE Plugin Integration With VCD

Onboarding Tenants

Before a tenant can start provisioning K8 cluster from CLI or UI (via CSE plugin), we need to enable the tenant to do so. This can be done directly from CSE server or login to any machine where vcd-cli utility is installed. To onboard a tenant, use following commands:

Note: These commands needs to be run as VCD system admin.

# vcd login vcd.vstellar.local system admin -iw

# vcd right add -o <org-name> “{cse}:CSE NATIVE DEPLOY RIGHT”

Example: # vcd right add -o cse_org “{cse}:CSE NATIVE DEPLOY RIGHT”

Rights added to the Org ‘cse_org’

Note: At this point of time, if we run command vcd cse ovdc list, it will show us no K8 provider has been configured for the tenants.… Read More

VCD Container Service Extension Series-Part 3: CSE Plugin Integration With VCD

01-07-202017-03-2022 Manish Jha

In last post of this series, I explained how to set up CSE server. In this post we will look at steps of integrating CSE plugin in VMware Cloud Director, so that tenants can spin K8’s cluster from VCD portal.

If you have landed directly on this post by mistake, I would recommend reading previous articles from this series.

1: Container Service Extension Introduction & Architecture

2: CSE Server Installation

Latest and greatest version of CSE plugin can be downloaded from Here

CSE plugin installation is taken care by Cloud Provider. Post installation, provider can choose to publish plugin to all/specific tenants.

Click on upload button to start the wizard. Clicking on Select Plugin File allow you to browse to location where plugin file is downloaded.

Select the scope of publishing CSE plugin. Service provider can publish this plugin to all or subset of tenants. … Read More

VCD Container Service Extension Series-Part 1: Introduction & Architecture

30-06-202026-01-2023 Manish Jha

I was working on VMware Container Service Extension (CSE) for the last 2 weeks and it was a great learning opportunity for me. My CSE deployment did not go smoothly and I faced many issues with very little or no idea on how to fix them. But kudos to Joe Mann for lending a helping hand to fix all infra-related issues.

Through this blog series, I want to pen down my experience of working with CSE and the challenges which I encountered, and how those issues were resolved.

What is VMware Container Service Extension?

VMware Container Service is an extension to Cloud Director which enables cloud providers to offer Kubernetes-as-a-Service (on top of VCD) to their tenants. Kubernetes as a service helps tenants to quickly deploy the Kubernetes cluster in just a few clicks directly from the VCD portal.

Cloud Providers upload customized Kubernetes templates in public catalogs which tenants leverage to deploy K8 clusters in self-contained vApps.… Read More