How to make NSX ALB 21.1.3 work with TKGm 1.5.1

Posted on 14-03-2022 by Manish Jha

To test TKGm 1.5.1 against the latest version of nSX ALB, I upgraded my ALB deployment to 21.1.3. The deployment of the TKG management and workload cluster went smoothly.

However, when I deployed a sample load balancer application that uses a dedicated SEG and VIP network, the service was waiting for an external IP assignment.

root@linjump [ ~ ]# kubectl get all
NAME                                   READY   STATUS    RESTARTS   AGE
pod/http-deployment-785df98964-dhvqj   1/1     Running   0          104s
pod/http-deployment-785df98964-xf6hk   1/1     Running   0          104s

NAME             TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service/l4-svc   LoadBalancer   100.70.116.153   <pending>     80:31082/TCP   103s

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/http-deployment   2/2     2            2           104s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/http-deployment-785df98964   2         2         2       104s

root@linjump [ ~ ]# kubectl get all

NAME READY STATUS RESTARTS AGE

pod/http-deployment-785df98964-dhvqj 1/1 Running 0 104s

pod/http-deployment-785df98964-xf6hk 1/1 Running 0 104s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/l4-svc LoadBalancer 100.70.116.153 <pending> 80:31082/TCP 103s

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.

NSX ALB Signed Certificates and TKGm Installation Gotcha

Posted on 13-03-2022 by Manish Jha

The Problem

I recently replaced the self-signed NSX-ALB certificates with a CA-signed (Microsoft CA) certificate, which caused a new unanticipated issue with TKGm deployment.

The TKGm installer wizard was complaining about the certificate validity. I knew there was nothing wrong with the certificate validity on NSX ALB because it was replaced just a few hours ago. Nonetheless, I double-checked the certificate expiration date, which is set to 2024.

After some jiggling, I investigated the bootstrap machine CLI terminal, where I issued the tanzu management-cluster create command, and spotted the main problem right away.

This is the error shown in the CLI.

E0313 12:55:20.639848 1549280 avisession.go:666] Client error for URI: login. Error: Post "https://alb.vstellar.local/login": x509: certificate signed by unknown authority

1	E0313 12:55:20.639848 1549280 avisession.go:666] Client error for URI: login. Error: Post "https://alb.vstellar.local/login": x509: certificate signed by unknown authority

Since the certificate is not signed by a Public CA, the bootstrapper machine has no idea about the CA server who signed this cert.… Read More

Replacing NSX ALB Certificates with Signed Certificates

Posted on 13-03-2022 by Manish Jha

In this post, I will walk through the steps of replacing NSX ALB self-signed certificates with a CA-signed certificate. For the purpose of this demonstration, I am using Active Directory Certificate Service in my lab. I have a windows server 2019 deployed and additional roles configured for AD integrated Certificate Service.

Please follow the below procedure for replacing NSX ALB certificates.

Step 1: Generate Certificate Signing Request (CSR)

CSR includes information such as domain name, organization name, locality, and country. The request also contains the public key/private key, which will be associated with the certificate generated. A CSR can be generated directly from the NSX ALB portal, but that requires configuring a Certificate Management Profile or using the OpenSSL utility.

To generate a CSR via the NSX ALB portal, go to Templates > Security > SSL/TLS Certificates and click on the Create button, then select controller certificate from the drop-down menu.… Read More

NSX ALB Integration with VCD-Part 5: Load Balancing in Action

Posted on 26-12-2021 by Manish Jha

Welcome to the last post of this series. I am sure if you are following this blog series, then you have got yourself familiar with how NSX ALB integrates with VCD to provide “Load Balancing as a Service (LBaaS)”

In this post, I will demonstrate how tenants can leverage NSX ALB to create load balancer constructs (Virtual Services, Pools, etc)

If you haven’t read the previous posts in this series, I recommend you do so using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

3: Implementing Dedicated Service Engine Groups Design

4: Implementing Shared Service Engine Groups Design

Tenant vStellar has deployed a couple of servers that are connected to a routed network “Prod-GW” and have got IP addresses 192.168.40.5 and 192.168.40.6 respectively.

Both servers are running an HTTP web server and are accessible via their local IP.

The tenant is looking for load balancing these web servers by leveraging NSX ALB.… Read More

NSX ALB Integration with VCD-Part 4: Shared Service Engine Groups

Posted on 26-12-2021 by Manish Jha

Welcome to the 4th part of the NSX Advanced Load Balancer Integration with VMware Cloud Director series. The first post in this series covered Service Engine design topologies, while the second covered the processes for enabling “Load Balancing as a Service” in VCD. The deployment of the Dedicated Service Engine design was demonstrated in the third post.

This post will talk about the implementation of the Shared Service Engines design.

If you haven’t read the previous posts in this series, I recommend you do so using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

3: Implementing Dedicated Service Engine Groups Design

In Shared Service Engine Group design, tenant’s Edge Gateways can leverage a common Service Engine Group for the load balancer and virtual services placement. Since VCD tenants can have overlapping org networks implemented in their respective org’s, data traffic segregation is achieved by implementing VRF’s in NSX ALB. … Read More

NSX ALB Integration with VCD-Part 3: Dedicated Service Engine Groups

Posted on 11-12-2021 by Manish Jha

I discussed the supported design for NSX ALB integration with the VMware Cloud Director in the first post of this series. Part 2 of this series described how to enable “Load Balancing as a Service” in VCD.

If you missed any of the previous posts in this series, I recommend that you read them using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

This blog post is focused on implementing the Dedicated Service Engine Groups design.

The below diagram shows the high-level overview of Dedicated SEG in VCD.

In this design, the management network of Service Engine (eth0) is attached to the tier-1 gateway dedicated for NSX ALB management and provisioned by the service provider. When a Virtual Service is created by the tenant, a logical segment corresponding to the VIP network is automatically created and gets attached to the tenant’s tier-1 gateway.… Read More

NSX ALB Integration with VCD-Part 2: NSX ALB & Infra Configuration

Posted on 28-11-2021 by Manish Jha

In the first post of this series, I discussed the design patterns that are supported for NSX ALB integration with VCD.

In this post, I will share the steps of the NSX ALB & Infra configuration, before implementing the supported designs.

Step 1: Configure NSX-T Constructs

1a: Deploy a couple of new Edge nodes to place the Tier-0 gateway that you will be creating for the NSX ALB consumption.

Associate the newly deployed edge nodes with the existing Edge Cluster.

1b: Create a Tier-0 and configure BGP. Also, ensure that Tier-1 connected segments are allowed to be redistributed via BGP.

1c: Create a Tier-1 gateway and associate it with the Tier-o gateway that you created in the previous step.

Ensure that the tier-1 gateway is configured to redistribute connected routes to the tier-0 gateway.

1d: Create a DHCP-enabled logical segment for the Service Engine management and connect the segment to the tier-1 gateway which you created in the previous step.… Read More

NSX ALB Integration with VCD-Part 1: Design Patterns

Posted on 28-11-2021 by Manish Jha

Overview

NSX Advanced Load Balancer provides multi-cloud load balancing, web application firewall, application analytics, and container ingress services from the data center to the cloud. It is an Intent-based software load balancer that provides scalable application delivery across any infrastructure. NSX ALB provides 100% software load balancing to ensure a fast, scalable and secure application experience. It delivers elasticity and intelligence across any environment.

With the release of VCD 10.2, NSX Advanced Load Balancer integration is available for use by the tenants. Service Provider configured NSX ALB and exposes load balancing functionality to the tenants so that tenants can deploy load balancers in a self-service fashion.

The latest release of VCD (10.3.1) supports NSX ALB versions up to 21.1.2. Please check the VMware product interop matrix before planning your deployment.

In this blog post, I will be talking about the NSX ALB design patterns for VCD and the ALB integration steps with VCD.… Read More

Tanzu Kubernetes Grid Ingress With NSX Advanced Load Balancer

Posted on 08-09-2021 by Manish Jha

NSX ALB delivers scalable, enterprise-class container ingress for containerized workloads running in Kubernetes clusters. The biggest advantage of using NSX ALB in a Kubernetes environment is that it is agnostic to the underlying Kubernetes cluster implementations. The NSX ALB controller integrates with the Kubernetes ecosystem via REST API and thus can be used for ingress & L4-L7 load balancing solution for a wide variety of Kubernetes implementation including VMware Tanzu Kubernetes Grid.

NSX ALB provides ingress and load balancing functionality for TKG using AKO which is a Kubernetes operator that runs as a pod in the Tanzu Kubernetes clusters and translates the required Kubernetes objects to Avi objects and automates the implementation of ingresses/routes/services on the Service Engines (SE) via the NSX ALB Controller.

The diagram below shows a high-level architecture of AKO interaction with NSX ALB.

AKO interacts with the Controller & Service Engines via API to automate the provisioning of Virtual Service/VIP etc.… Read More

NSX ALB Upgrade Breaking AKO Integration

Posted on 20-08-2021 by Manish Jha

Recently I upgraded NSX ALB from 20.1.4 to 20.1.5 in my lab and observed weird things whenever I attempted to deploy/delete any Kubernetes workload of type LoadBalancer.

The Issue

On deploying a new K8 application, AKO was unable to create a load balancer for the application. In NSX ALB UI, I can see that a pool has been created and a VIP assigned but no VS is present. I have also verified that the ‘ako-essential’ role has the necessary permission “PERMISSION_VIRTUALSERIVCE” to create any new VS.

On attempting to delete a K8 application, the application got deleted from the TKG side, but it left lingering items (VS, Pools, etc) in the ALB UI. To investigate more on the issue, I manually tried deleting the server pool and captured the output using the browser network inspect option.

As expected the delete operation failed with the error that the object that you are trying to delete is associated with ‘L4PolicySet’

Request URL: https://192.168.15.20/api/pool/pool-18629d29-535a-49e5-93b5-dc2a4589374b
Request Method: DELETE

{
   "error":"Cannot delete, object is referred by: ['L4PolicySet tkc-wld01--default-my-service']",
   "obj_name":"tkc-wld01--default-my-service--8080"
}

Request URL: https://192.168.15.20/api/pool/pool-18629d29-535a-49e5-93b5-dc2a4589374b

Request Method: DELETE

{

"error":"Cannot delete, object is referred by: ['L4PolicySet tkc-wld01--default-my-service']",

"obj_name":"tkc-wld01--default-my-service--8080"

}

But the l4policyset was empty

[admin:172-16-10-11]: > show l4policyset
No results.

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Spread the Love

Overview

Spread the Love

Spread the Love

Spread the Love