In second post of this series we looked at the components which are needed to form a VCAV solution stack and we saw cassandra is an integral part of this solution.
To know about cassandra in detail, you can refer an old blogpost written by me.
If you are not following along this series, then I recommend reading earlier posts of this series from below links:
1: vCloud Availability Introduction
2: vCloud Availability Architecture & Components
4: Install Cloud Proxy for vCD
In this post we will deploy a Cassandra Cluster. Before installing cassandra software we need to meet following requirements:
1: Verify that routing, NTP, forward, and reverse DNS resolutions are working correctly.
2: Make sure that SELinux and your firewall are in disabled state. If they are enabled then make sure to open appropriate ports in firewall.
I have disabled SELinux and firewall service in my lab.
1 2 3 4 5 6 7 |
[root@mgmt-cass-b ~]# cat /etc/sysconfig/selinux | grep disabled SELINUX=disabled [root@mgmt-cass-b ~]# systemctl disable firewalld [root@mgmt-cass-b ~]# systemctl stop firewalld |
3: Install Python: Make sure Python 2.7 is installed in your CentOS 7 system. Default version of installed python is 2.7.5
[root@mgmt-cass-b ~]# python –version
Python 2.7.5
Note: If python is not installed, then you can install it via yum by typing: # yum install python2 -y
4 : Install Java: JDK 8u152 has been tested for cassandra and it works fine. It can be downloaded from here
[root@mgmt-cass-b ~]# rpm -ivh jdk-8u152-linux-x64.rpm
5: Set JAVA_HOME Variable
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@mgmt-cass-b ~]# echo JAVA_HOME=/usr/java/jdk1.8.0_152/jre/ >> /etc/profile [root@mgmt-cass-b ~]# echo PATH=$PATH:$HOME/bin/:$JAVA_HOME/bin >> /etc/profile [root@mgmt-cass-b ~]# echo export PATH JAVA_HOME >> /etc/profile [root@mgmt-cass-b ~]# source /etc/profile [root@mgmt-cass-b ~]# echo $JAVA_HOME /usr/java/jdk1.8.0_152/jre/ [root@mgmt-cass-b ~]# java -version java version "1.8.0_152" |
6: Install Java Cryptography Extension (JCE): JCE increase the java encryption level support.
If your cassandra node has access to internet and you have wget utility installed then JCE can be downloaded by running the following command:
[root@mgmt-cass-b ~]# wget -c –header “Cookie: oraclelicense=accept-securebackup-cookie” http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip
Unzip the file by running the following command:
[root@mgmt-cass-b ~]# unzip jce_policy-8.zip
Note: If you get error “unzip: command not found” then you can install the unzip by running command # yum install zip unzip -y
Copy the unzipped JCE file to the respective directory by running the following command:
[root@mgmt-cass-b ~]# cp UnlimitedJCEPolicyJDK8/*.jar /usr/java/jdk1.8.0_152/jre/lib/security/
7: Install Cassandra via YUM
Create the /etc/yum.repos.d/cassandra.repo file with the following content:
1 2 3 4 5 6 |
[cassandra] name=Apache Cassandra baseurl=https://www.apache.org/dist/cassandra/redhat/311x/ gpgcheck=1 repo_gpgcheck=1 gpgkey=https://www.apache.org/dist/cassandra/KEYS |
Install Cassandra by running command: yum install cassandra -y
8: Fine Tune cassandra : We need to make some configuration changes in cassandra by editing cassandra.yaml file. make sure to backup original file before making any changes.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@mgmt-cass-b run]# sed -i "s/cluster_name: 'Test Cluster'/cluster_name: 'vCD Metric Cluster'/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# NODEIP=$(ip addr show |grep -w inet |grep -v 127.0.0.1|awk '{ print $2}'| cut -d "/" -f 1) [root@mgmt-cass-b run]# sed -i "s/listen_address: localhost/listen_address: ${NODEIP}/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i "s/rpc_address: localhost/rpc_address: ${NODEIP}/g" /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i 's/seeds: "127.0.0.1"/seeds: "mgmt-cass-b"/g' /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# echo auto_bootstrap: true >> /etc/cassandra/conf/cassandra.yaml [root@mgmt-cass-b run]# sed -i "s/endpoint_snitch: 'SimpleSnitch'/endpoint_snitch: 'GossipingPropertyFileSnitch'/g" /etc/cassandra/conf/cassandra.yaml |
# Enable Cassandra Service
1 2 3 4 5 |
[root@mgmt-cass-b ~]# systemctl enable cassandra cassandra.service is not a native service, redirecting to /sbin/chkconfig. Executing /sbin/chkconfig cassandra on |
# Start the Cassandra service.
[root@mgmt-cass-b ~]# systemctl start cassandra
# Check the Cassandra service status.
[root@mgmt-cass-b ~]# systemctl status cassandra
# Check nodetool status to verify node is up and had joined cluster
1 2 3 4 5 6 7 |
[root@mgmt-cass-b ~]# nodetool status Datacenter: dc1 =============== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host ID Rack UN 192.168.109.37 217.06 KiB 256 100.0% ea299769-d942-4ce6-a61e-aeef92db28a3 rack1 |
8: Cassandra Clustering: To add additional nodes to cassandra cluster, deploy a new CentOS 7 vm and repeat the above steps. In my lab I have deployed only one cassandra node at the moment, but you can follow instructions documented in this article for adding additional nodes.
Once new node is fully configured, edit the cassandra.yaml file on both node and change the value of seed under seed_provider section
e.g: – seeds: “ip-Cassandra-node-1,ip-Cassandra-node-2,ip-Cassandra-node-3”
And then restart the Cassandra service on all nodes. Verify that the node is operational by running command: nodetool status
9: Cassandra SSL Installation
1: Generate an SSL certificate by running the following command.
# /usr/bin/keytool -keystore /etc/cassandra/conf/.keystore -storepass vmware -validity 1826 -storetype JKS -genkey -keyalg RSA -alias mgmt-cass-b -dname ‘cn=mgmt-cass-b, ou=vStellar, o=Alex.Co, c=IN’ -keypass vmware
2: Export the Cassandra certificate to a PEM-formatted file.
# /usr/bin/keytool -export -rfc -keystore /etc/cassandra/conf/.keystore -storepass vmware -file /root/cloud-mgmt-cass-b-node_number.pem -alias mgmt-cass-b
3: Copy the .pem certificate file of each Cassandra node to the /root directory of the rest of the Cassandra nodes.
4: Import each certificate to the truststore of every Cassandra host.
# /usr/bin/keytool -noprompt -import -trustcacerts -alias mgmt-cass-b -file /root/cloud-mgmt-cass-b-node_number.pem -keystore /etc/cassandra/conf/.truststore -storepass vmware
5: Enable the server and the client communication with Cassandra over SSL.
Edit /etc/cassandra/conf/cassandra.yaml configuration file and change following values as shown below
a: Set the listen_address and rpc_address values to the Cassandra node IP address.
- listen_address: Cass-Node-IP
- rpc_address: Cass-Node-IP
b: Update the values of the server_encryption_options properties.
Note: The keystore and truststore passwords are the same passwords that you used to create the keystore and the truststore.
1 2 3 4 5 6 7 8 9 10 11 |
server_encryption_options: internode_encryption: all keystore: /etc/cassandra/conf/.keystore keystore_password: vmware truststore: /etc/cassandra/conf/.keystore truststore_password: vmware # More advanced defaults below: # protocol: TLS # algorithm: SunX509 store_type: JKS require_client_auth: false |
c: Update the values of the client_encryption_options properties.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
client_encryption_options: enabled: true # If enabled and optional is set to true encrypted and unencrypted connections are handled. optional: true keystore: /etc/cassandra/conf/.keystore keystore_password: vmware require_client_auth: true # Set trustore and truststore_password if require_client_auth is true truststore: /etc/cassandra/conf/.keystore truststore_password: vmware # More advanced defaults below: # protocol: TLS # algorithm: SunX509 store_type: JKS |
d: Restart the Cassandra node and make sure there are no errors reported in /var/log/cassandra/system.log. Last line of this log file usually reads as:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
INFO [main] 2018-09-06 22:07:49,872 StorageService.java:1446 - JOINING: Finish joining ring INFO [main] 2018-09-06 22:07:50,033 StorageService.java:2289 - Node /192.168.109.37 state jump to NORMAL INFO [main] 2018-09-06 22:07:50,059 Gossiper.java:1692 - Waiting for gossip to settle... INFO [main] 2018-09-06 22:07:58,061 Gossiper.java:1723 - No gossip backlog; proceeding INFO [main] 2018-09-06 22:07:58,447 NativeTransportService.java:70 - Netty using native Epoll event loop INFO [main] 2018-09-06 22:07:58,556 Server.java:140 - Enabling optionally encrypted CQL connections between client and server INFO [main] 2018-09-06 22:07:58,612 Server.java:156 - Starting listening for CQL clients on /192.168.109.37:9042 (encrypted)... |
10: Enable cqlsh with SSL encryption
a: Import the Cassandra keystore into a new PKC12 keystore.
1 2 3 |
[root@mgmt-cass-b ~]# /usr/bin/keytool -importkeystore -srckeystore /etc/cassandra/conf/.keystore -srcstorepass vmware -alias mgmt-cass-b -destkeystore /tmp/keystore.p12 -deststorepass vmware -deststoretype PKCS12 Importing keystore /etc/cassandra/conf/.keystore to /tmp/keystore.p12... |
b: Extract the certificate from the new PKC12 keystore.
1 2 3 |
[root@mgmt-cass-b ~]# openssl pkcs12 -in /tmp/keystore.p12 -nokeys -out /etc/cassandra/conf/CLIENT.cer.pem -passin pass:vmware MAC verified OK |
c: Extract the certificate key from the new PKC12 keystore.
1 2 3 |
[root@mgmt-cass-b ~]# openssl pkcs12 -in /tmp/keystore.p12 -nodes -nocerts -out /etc/cassandra/conf/CLIENT.key.pem -passin pass:vmware MAC verified OK |
d: Create a /root/.cassandra/cqlshrc file with the following contents:
1 2 3 4 5 6 7 8 9 10 |
[connection] hostname = 192.168.109.37 port = 9042 factory = cqlshlib.ssl.ssl_transport_factory [ssl] certfile = /opt/cassandra/conf/certs/CLIENT.cer.pem validate = false userkey = /etc/cassandra/conf/CLIENT.key.pem usercert = /etc/cassandra/conf/CLIENT.cer.pem |
e: Verify that you can use cqlsh command
1 2 3 4 5 |
[root@mgmt-cass-b ~]# cqlsh Connected to vCloud Director Metric Cluster at 192.168.109.37:9042. [cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. cqlsh> |
And that’s it for this post. In next post of this series we will deploy RabbitMQ Cluster.
I hope you enjoyed reading this post. Feel free to share this on social media if it is worth sharing. Be sociable