Backing Up Stateful Applications using TMC Data Protection

Introduction

Kubernetes is frequently thought of as a platform for stateless workloads because the majority of its resources are ephemeral. However, as Kubernetes grows in popularity, enterprises are deploying more and more stateful apps. Because stateful workloads require permanent storage for application data, you can no longer simply reload them in the event of a disaster.

As businesses invest extensively in Kubernetes and deploy more and more containerized applications across multi-clouds, providing adequate data protection in a distributed environment becomes a challenge that must be addressed.

Data Protection in Tanzu Mission Control (TMC) is provided by Velero which is an open-source project. Velero backups typically include application and cluster data like config maps, custom resource definitions, secrets, and so on, which would then be re-applied to a cluster during restoration. The resources that are using a persistent volume, are backed up using Restic

In this post, I’ll show how to backup and recover a stateful application running in a Tanzu Kubernetes cluster. I am using MinIO S3 storage for storing the backups.

The App

Before I jump into the lab, I want to give you an overview of the application that I will be using for demonstration. I am using the Acme Fitness demo app which is publically available on Github.

The below image shows the high-level overview of the application architecture. 

There are various components that make this app. This page explains the components in detail. 

App Deployment

The below steps provide the steps of deploying this app in a Tanzu Kubernetes cluster.

Step 1: Clone the repository

I have imported the application into my personal GitHub repository and made some changes to fit my environment. If you want to use my repo, clone the same on a jumpbox from where you can access your Tanzu Kubernetes clusters.

Step 2: Create a namespace for deploying the app

Connect to your Tanzu Kubernetes cluster and create a namespace for the app

Step 3: Create a registry secret

I have modified the deployment YAML and I am using a secret named regcred. Using this secret, the required pod images will be pulled from the docker repository. You need to create this secret under the namespace that you have created in the previous step.

Replace the docker-username, docker-password, and docker-email with your values.

Verify that you can see the secret in the namespace by running the command: kubectl get secrets

Step 4: Deploy the application

To deploy the application, go to the acme-fitness/kubernetes-manifests folder and execute the deployacme.sh file with the execute permission set. When you run the file, you’ll notice that pods, services, and pvcs, among other things, are generated.

Step 5: List Deployment Items

List PVC’s

List Pods

List Services

The ‘frontend‘ service has an external IP address for accessing the application. You should be able to view the below portal if you go to the frontend service’s external IP.

You can login to the application using the credentials eric/vmware1!

Step 6: Add data into the application

You can do some practice shopping and place an order after logging into the app. The order details are kept in the order-postgres pod, which has a physical volume attached.

 

User manish has placed an order as shown below.

 

When the order is placed, you will see an order id ‘8d33—‘. This order is stored in the database which we will examine in the next step.

To go inside the order-postgres pod, execute the below commands:

verify the order details by executing the command as shown below:

When a backup and restore of the acmefitness app is performed, you should be able to see the same data in the restored app.

Step 7: Annotate the order-postgres pod.

In order for Restic to perform the backup of the stateful application, the pods need to be annotated using the key backup.velero.io/backup-volumes. Currently, TMC doesn’t have this functionality.

You should be knowing the name of the volume which is tied to the pod that you are annotating. To find the name of the volume, do a kubectl describe on the pod and search at the section ‘Volumes’. The existing annotations for the pod can also be seen in the same output.

Once the volume name is discovered, the pod can be annotated using the command shown below:

Now the application is ready for backup/restore. It is assumed that Data protection is already enabled on the Tanzu Kubernetes cluster where the application is running.

Perform Application Backup

Locate your Tanzu Kubernetes cluster by logging into the Tanzu Mission control portal. When data protection is enabled on the cluster, the Create Backup button appears.

Select the ‘acme’ namespace for backup.

Select the location where the backup will be stored. I am using the location which I configured in my on-prem. 

Continue through the create backup wizard and at the last step, provide a name for the backup. 

Allow some time for the backup process status to change to complete.

When the backup is finished, you’ll notice that a restic folder has been created on the S3 storage you chose for backup.

If you have velero installed on your jumpbox, you may use the velero get backups command to receive information about the backups. To get information about the restic backup, use the velero backups describe command as shown below:

If you pass the –details argument to the above command, the output provides you the name of the pod’s backed up by restic.

Perform Application Restore

To restore the application, return to the Tanzu Mission Control portal and select the backup which you want to restore.

Choose the namespace you want to restore and then click the pencil icon under Target Namespace. The target namespace can be the same as the source namespace, however, because my source namespace is healthy, I am restoring to a new namespace in my lab.

Provide the name for the namespace where you want to restore.

Give the restore a name and then click the Restore button.

Wait for the restore task to complete.

A new namespace is created in the Tanzu Kubernetes cluster.

Verify that the pods in the acme-restore namespace are in running state

Execute the commands provided in Step-6 to login to the order-postgres pod. Check that the order-related data is visible in the database.

And that’s it for this post. I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing.

Leave a Reply