In the second post of this series, we discussed the components needed to form a VCAV solution stack, and we saw that Cassandra is an integral part of this solution. In this post, we will learn how to deploy a Cassandra Cluster.
If you are not following along with this series, then I recommend reading earlier posts of this series from the links below:
1: vCloud Availability Introduction
2: vCloud Availability Architecture & Components
4: Install Cloud Proxy for vCD
Before installing Cassandra software, we need to meet the following requirements:
1: Verify that routing, NTP, forward, and reverse DNS resolutions are working correctly.
2: Make sure that SELinux and your firewall are disabled. If they are enabled, then open appropriate ports in the firewall.
I have disabled SELinux and the firewall service in my lab.
|
1 2 3 4 5 6 7 |
[root@mgmt-cass-b ~]# cat /etc/sysconfig/selinux | grep disabled SELINUX=disabled [root@mgmt-cass-b ~]# systemctl disable firewalld [root@mgmt-cass-b ~]# systemctl stop firewalld |
3: Install Python: Make sure Python 2.7 is installed in your CentOS 7 system. The default version of installed Python is 2.7.5
|
1 2 3 |
[root@mgmt-cass-b ~]# python --version Python 2.7.5 |
Note: If Python is not installed, then you can install it via yum by typing: # yum install python2 -y
4: Install Java: JDK 8u152 has been tested for Cassandra, and it works fine. It can be downloaded from here
|
1 |
[root@mgmt-cass-b ~]# rpm -ivh jdk-8u152-linux-x64.rpm |
5: Set JAVA_HOME Variable
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@mgmt-cass-b ~]# echo JAVA_HOME=/usr/java/jdk1.8.0_152/jre/ >> /etc/profile [root@mgmt-cass-b ~]# echo PATH=$PATH:$HOME/bin/:$JAVA_HOME/bin >> /etc/profile [root@mgmt-cass-b ~]# echo export PATH JAVA_HOME >> /etc/profile [root@mgmt-cass-b ~]# source /etc/profile [root@mgmt-cass-b ~]# echo $JAVA_HOME /usr/java/jdk1.8.0_152/jre/ [root@mgmt-cass-b ~]# java -version java version "1.8.0_152" |
6: Install Java Cryptography Extension (JCE): JCE increases the Java encryption level support.
If your Cassandra node has internet access and you have the wget utility installed, then JCE can be downloaded by running the following command:
|
1 |
[root@mgmt-cass-b ~]# wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip |
Unzip the file by running the following command:
|
1 |
[root@mgmt-cass-b ~]# unzip jce_policy-8.zip |
Note: If you get the error “unzip: command not found,” then you can install it by running the command # yum install zip unzip -y
Copy the unzipped JCE file to the respective directory by running the following command:
|
1 |
[root@mgmt-cass-b ~]# cp UnlimitedJCEPolicyJDK8/*.jar /usr/java/jdk1.8.0_152/jre/lib/security/ |
7: Install Cassandra via YUM
Create the /etc/yum.repos.d/cassandra.repo file with the following content:
|
1 2 3 4 5 6 |
[cassandra] name=Apache Cassandra baseurl=https://www.apache.org/dist/cassandra/redhat/311x/ gpgcheck=1 repo_gpgcheck=1 gpgkey=https://www.apache.org/dist/cassandra/KEYS |
Install Cassandra by running the command: yum install cassandra -y
8: Fine-tune Cassandra: We need to make some configuration changes in Cassandra by editing the cassandra.yaml file. Make sure to back up the original file before making any changes.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@mgmt-cass-b run]# sed -i "s/cluster_name: 'Test Cluster'/cluster_name: 'vCD Metric Cluster'/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# NODEIP=$(ip addr show |grep -w inet |grep -v 127.0.0.1|awk '{ print $2}'| cut -d "/" -f 1) [root@mgmt-cass-b run]# sed -i "s/listen_address: localhost/listen_address: ${NODEIP}/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i "s/rpc_address: localhost/rpc_address: ${NODEIP}/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i 's/seeds: "127.0.0.1"/seeds: "mgmt-cass-b"/g' /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# echo auto_bootstrap: true >> /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i "s/endpoint_snitch: 'SimpleSnitch'/endpoint_snitch: 'GossipingPropertyFileSnitch'/g" /etc/cassandra/conf/cassandra.yaml |
# Enable Cassandra Service
|
1 2 3 4 5 |
[root@mgmt-cass-b ~]# systemctl enable cassandra cassandra.service is not a native service, redirecting to /sbin/chkconfig. Executing /sbin/chkconfig cassandra on |
# Start the Cassandra service.
|
1 |
[root@mgmt-cass-b ~]# systemctl start cassandra |
# Check the Cassandra service status.
|
1 |
[root@mgmt-cass-b ~]# systemctl status cassandra |
# Check nodetool status to verify the node is up and has joined the cluster.
|
1 2 3 4 5 6 7 |
[root@mgmt-cass-b ~]# nodetool status Datacenter: dc1 =============== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host ID Rack UN 192.168.109.37 217.06 KiB 256 100.0% ea299769-d942-4ce6-a61e-aeef92db28a3 rack1 |
8: Cassandra Clustering: To add additional nodes to the Cassandra cluster, deploy a new CentOS 7 vm and repeat the above steps. In my lab, I have deployed only one Cassandra node. You can follow the instructions documented in this article for adding additional nodes.
Once the new node is fully configured, edit the cassandra.yaml file on both nodes and change the value of seed under the seed_provider section
|
1 |
Example: - seeds: "ip-Cassandra-node-1,ip-Cassandra-node-2,ip-Cassandra-node-3" |
And then restart the Cassandra service on all nodes. Verify that the node is operational by running the command: nodetool status
9: Cassandra SSL Installation
1: Generate an SSL certificate by running the following command.
|
1 |
# /usr/bin/keytool -keystore /etc/cassandra/conf/.keystore -storepass vmware -validity 1826 -storetype JKS -genkey -keyalg RSA -alias mgmt-cass-b -dname 'cn=mgmt-cass-b, ou=vStellar, o=Alex.Co, c=IN' -keypass vmware |
2: Export the Cassandra certificate to a PEM-formatted file.
|
1 |
# /usr/bin/keytool -export -rfc -keystore /etc/cassandra/conf/.keystore -storepass vmware -file /root/cloud-mgmt-cass-b-node_number.pem -alias mgmt-cass-b |
3: Copy the .pem certificate file of each Cassandra node to the /root directory of the rest of the Cassandra nodes.
4: Import each certificate into the truststore of every Cassandra host.
|
1 |
# /usr/bin/keytool -noprompt -import -trustcacerts -alias mgmt-cass-b -file /root/cloud-mgmt-cass-b-node_number.pem -keystore /etc/cassandra/conf/.truststore -storepass vmware |
5: Enable the server and the client communication with Cassandra over SSL.
Edit /etc/cassandra/conf/cassandra.yaml configuration file and change the following values as shown below
5a: Set the listen_address and rpc_address values to the Cassandra node IP address.
- listen_address: Cass-Node-IP
- rpc_address: Cass-Node-IP
5b: Update the values of the server_encryption_options properties.
Note: The keystore and truststore passwords are the same passwords that you used to create the keystore and the truststore.
|
1 2 3 4 5 6 7 8 9 10 11 |
server_encryption_options: internode_encryption: all keystore: /etc/cassandra/conf/.keystore keystore_password: vmware truststore: /etc/cassandra/conf/.keystore truststore_password: vmware # More advanced defaults below: # protocol: TLS # algorithm: SunX509 store_type: JKS require_client_auth: false |
5c: Update the values of the client_encryption_options properties.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
client_encryption_options: enabled: true # If enabled and optional is set to true encrypted and unencrypted connections are handled. optional: true keystore: /etc/cassandra/conf/.keystore keystore_password: vmware require_client_auth: true # Set trustore and truststore_password if require_client_auth is true truststore: /etc/cassandra/conf/.keystore truststore_password: vmware # More advanced defaults below: # protocol: TLS # algorithm: SunX509 store_type: JKS |
5d: Restart the Cassandra node and make sure there are no errors reported in /var/log/cassandra/system.log. Last line of this log file usually reads as:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
INFO [main] 2018-09-06 22:07:49,872 StorageService.java:1446 - JOINING: Finish joining ring INFO [main] 2018-09-06 22:07:50,033 StorageService.java:2289 - Node /192.168.109.37 state jump to NORMAL INFO [main] 2018-09-06 22:07:50,059 Gossiper.java:1692 - Waiting for gossip to settle... INFO [main] 2018-09-06 22:07:58,061 Gossiper.java:1723 - No gossip backlog; proceeding INFO [main] 2018-09-06 22:07:58,447 NativeTransportService.java:70 - Netty using native Epoll event loop INFO [main] 2018-09-06 22:07:58,556 Server.java:140 - Enabling optionally encrypted CQL connections between client and server INFO [main] 2018-09-06 22:07:58,612 Server.java:156 - Starting listening for CQL clients on /192.168.109.37:9042 (encrypted)... |
10: Enable cqlsh with SSL encryption
10a: Import the Cassandra keystore into a new PKC12 keystore.
|
1 2 3 |
[root@mgmt-cass-b ~]# /usr/bin/keytool -importkeystore -srckeystore /etc/cassandra/conf/.keystore -srcstorepass vmware -alias mgmt-cass-b -destkeystore /tmp/keystore.p12 -deststorepass vmware -deststoretype PKCS12 Importing keystore /etc/cassandra/conf/.keystore to /tmp/keystore.p12... |
10b: Extract the certificate from the new PKC12 keystore.
|
1 2 3 |
[root@mgmt-cass-b ~]# openssl pkcs12 -in /tmp/keystore.p12 -nokeys -out /etc/cassandra/conf/CLIENT.cer.pem -passin pass:vmware MAC verified OK |
10c: Extract the certificate key from the new PKC12 keystore.
|
1 2 3 |
[root@mgmt-cass-b ~]# openssl pkcs12 -in /tmp/keystore.p12 -nodes -nocerts -out /etc/cassandra/conf/CLIENT.key.pem -passin pass:vmware MAC verified OK |
10d: Create a /root/.cassandra/cqlshrc file with the following contents:
|
1 2 3 4 5 6 7 8 9 10 |
[connection] hostname = 192.168.109.37 port = 9042 factory = cqlshlib.ssl.ssl_transport_factory [ssl] certfile = /opt/cassandra/conf/certs/CLIENT.cer.pem validate = false userkey = /etc/cassandra/conf/CLIENT.key.pem usercert = /etc/cassandra/conf/CLIENT.cer.pem |
10e: Verify that you can use the cqlsh command
|
1 2 3 4 5 |
[root@mgmt-cass-b ~]# cqlsh Connected to vCloud Director Metric Cluster at 192.168.109.37:9042. [cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. cqlsh> |
And that’s it for this post. In the next post of this series, we will deploy a RabbitMQ Cluster.
I hope you enjoyed reading this post. Feel free to share this on social media if it’s worth sharing.