February 2, 2017
Setting up client and cluster SSL transport for a Cassandra cluster
This articles is a Cassandra tutorial on Cassandra setup for SSL and CQL clients, as well as installing Cassandra with SSL configured on a series of Linux servers.
Cassandra allows you to secure the client transport
(CQL) as well as the cluster transport (storage transport).
SSL/TLS have some overhead. This is especially true in the JVM world which is not as performant for handling SSL/TLS unless you are using Netty/OpenSSl integration.
Understanding SSL/TLS support for Cassandra is important for developers, DevOps and DBAs.
If possible, use no encryption for the cluster transport
(storage transport
), and deploy your Cassandra Database nodes in a private subnet, and limit access to this subnet to the client transport
. Also if possible avoid using TLS/SSL on the client transport and do client operations from your app tier, which is located in a non-public subnet.
However, that is not always possible. You may work in an industry that requires the use of encrypted transports like the U.S. Health Insurance Portability and Accountability Act (HIPAA), Germany’s Federal Data Protection Act, The Payment Card Industry Data Security Standard (PCI DSS), or U.S. Sarbanes-Oxley Act of 2002. Or you might work for a bank or other financial institution. Or it just might be a corporate policy to encrypt such transports.
Another area of concern is for compliance is authorization, and encrypted data at rest. Cassandra’s has essential security features: authentication, role-based authorization, transport encryption (JMX, client transport, cluster transport), as well as data at rest encryption (encrypting SSTables).
This article will focus just on setting up encryption for the Cassandra client transport
(CQL) and the cluster transport
. Later articles will cover various aspects of compliance and encryption.
Encrypting the transports
Data that travels over the client transport
across a network could be accessed by someone you don’t want accessing said data with tools like wire shark. If data includes private information, SSN number, credentials (password, username), credit card numbers or account numbers, then we want to make that data unintelligible (encrypted) to any and all 3rd parties. This is especially important if we don’t control the network. You can also use TLS to make sure the data has not been tampered with whilst traveling the network. The Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols are designed to provide these features (SSL is the old name for what became TLS but many people still refer to TLS as SSL).
Cassandra is written in Java. Java defines the JSSE framework which in turn uses the Java Cryptography Architecture (JCA). JSSE uses cryptographic service providers from JCA.
If any of the above is new to you, please take a few minutes to read through the TLS/SSL Java guide.
This article picks up right after this one – Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant.
DevOps/DBA Setting up Cassandra Database client config
The client transport encryption protects data as it moves from clients to server nodes in the cluster.
The client_encryption_options
are stored in the cassandra.yaml
.
Here is an example config.
Sample Cassandra config
# enable or disable client/server encryption.
client_encryption_options:
enabled: false
# If enabled and optional is set to true encrypted and unencrypted connections are handled.
optional: false
keystore: conf/.keystore
keystore_password: cassandra
require_client_auth: false
# Set trustore and truststore_password if require_client_auth is true
# truststore: conf/.truststore
# truststore_password: cassandra
protocol: TLS
algorithm: SunX509
store_type: JKS
cipher_suites: [TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_anon_WITH_AES_256_CBC_SHA]
DevOps/DBA task: Setup SSL keys for Cassandra Database
Before we go into the details of setting up the cassandra.yaml
file, let’s create some trust stores, key stores, and export some keys. The following script generates cluster and client keys.
setupkeys-cassandra-security.sh
#!/bin/bash
KEY_STORE_PATH="$PWD/resources/opt/cassandra/conf/certs"
mkdir -p "$KEY_STORE_PATH"
KEY_STORE="$KEY_STORE_PATH/cassandra.keystore"
PKS_KEY_STORE="$KEY_STORE_PATH/cassandra.pks12.keystore"
TRUST_STORE="$KEY_STORE_PATH/cassandra.truststore"
PASSWORD=cassandra
CLUSTER_NAME=test
CLUSTER_PUBLIC_CERT="$KEY_STORE_PATH/CLUSTER_${CLUSTER_NAME}_PUBLIC.cer"
CLIENT_PUBLIC_CERT="$KEY_STORE_PATH/CLIENT_${CLUSTER_NAME}_PUBLIC.cer"
### Cluster key setup.
# Create the cluster key for cluster communication.
keytool -genkey -keyalg RSA -alias "${CLUSTER_NAME}_CLUSTER" -keystore "$KEY_STORE" -storepass "$PASSWORD" -keypass "$PASSWORD" \
-dname "CN=CloudDurable Image $CLUSTER_NAME cluster, OU=Cloudurable, O=Cloudurable, L=San Francisco, ST=CA, C=USA, DC=cloudurable, DC=com" \
-validity 36500
# Create the public key for the cluster which is used to identify nodes.
keytool -export -alias "${CLUSTER_NAME}_CLUSTER" -file "$CLUSTER_PUBLIC_CERT" -keystore "$KEY_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
# Import the identity of the cluster public cluster key into the trust store so that nodes can identify each other.
keytool -import -v -trustcacerts -alias "${CLUSTER_NAME}_CLUSTER" -file "$CLUSTER_PUBLIC_CERT" -keystore "$TRUST_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
### Client key setup.
# Create the client key for CQL.
keytool -genkey -keyalg RSA -alias "${CLUSTER_NAME}_CLIENT" -keystore "$KEY_STORE" -storepass "$PASSWORD" -keypass "$PASSWORD" \
-dname "CN=CloudDurable Image $CLUSTER_NAME client, OU=Cloudurable, O=Cloudurable, L=San Francisco, ST=CA, C=USA, DC=cloudurable, DC=com" \
-validity 36500
# Create the public key for the client to identify itself.
keytool -export -alias "${CLUSTER_NAME}_CLIENT" -file "$CLIENT_PUBLIC_CERT" -keystore "$KEY_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
# Import the identity of the client pub key into the trust store so nodes can identify this client.
keytool -importcert -v -trustcacerts -alias "${CLUSTER_NAME}_CLIENT" -file "$CLIENT_PUBLIC_CERT" -keystore "$TRUST_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
keytool -importkeystore -srckeystore "$KEY_STORE" -destkeystore "$PKS_KEY_STORE" -deststoretype PKCS12 \
-srcstorepass "$PASSWORD" -deststorepass "$PASSWORD"
openssl pkcs12 -in "$PKS_KEY_STORE" -nokeys -out "${CLUSTER_NAME}_CLIENT.cer.pem" -passin pass:cassandra
openssl pkcs12 -in "$PKS_KEY_STORE" -nodes -nocerts -out "${CLUSTER_NAME}_CLIENT.key.pem" -passin pass:cassandra
The keytool utility ships with Java SDK. We use this keytool command to create the cluster key. Let’s break down the script that generates the keys and certificates.
DBA/DevOps Cassandra Task: Create the Cassandra cluster key
keytool -genkey -keyalg RSA -alias "${CLUSTER_NAME}_CLUSTER" -keystore "$KEY_STORE" -storepass "$PASSWORD" -keypass "$PASSWORD" \
-dname "CN=CloudDurable Image $CLUSTER_NAME cluster, OU=Cloudurable, O=Cloudurable, L=San Francisco, ST=CA, C=USA, DC=cloudurable, DC=com" \
-validity 36500
Once we create the cluster key, we will want to export a public key from it.
Export a public key for the Cassandra cluster key.
# Create the public key for the client to identify itself.
keytool -export -alias "${CLUSTER_NAME}_CLIENT" -file "$CLIENT_PUBLIC_CERT" -keystore "$KEY_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
Then we will import the public key into the trust store so that nodes can identify each other.
Import public key for the Cassandra cluster key into the trust store so nodes can identify each other
# Import the identity of the cluster public cluster key into the trust store so that nodes can identify each other.
keytool -import -v -trustcacerts -alias "${CLUSTER_NAME}_CLUSTER" -file "$CLUSTER_PUBLIC_CERT" -keystore "$TRUST_STORE" \
-storepass "$PASSWORD" -keypass "$PASSWORD" -noprompt
We perform the same three tasks for the client keys. Then lastly we create pem files for the client keys by exporting our Java JKS keystore as a PKCS12 trust store.
Cloudurable specialize in AWS DevOps Automation for Cassandra and Kafka
We hope this web page on installing Cassandra SSL keys informative. Check out our Casandra training and Kafka training. We also provide Casandra consulting and Kafka consulting to get you setup fast in AWS with CloudFormation and CloudWatch.
Creating client pem files
keytool -importkeystore -srcalias "${CLUSTER_NAME}_CLIENT" -srckeystore "$KEY_STORE" -destkeystore "$PKS_KEY_STORE" -deststoretype PKCS12 \
-srcstorepass "$PASSWORD" -deststorepass "$PASSWORD"
openssl pkcs12 -in "$PKS_KEY_STORE" -nokeys -out "$KEY_STORE_PATH/${CLUSTER_NAME}_CLIENT.cer.pem" -passin pass:cassandra
openssl pkcs12 -in "$PKS_KEY_STORE" -nodes -nocerts -out "$KEY_STORE_PATH/${CLUSTER_NAME}_CLIENT.key.pem" -passin pass:cassandra
Here are the files that get generated.
Cassandra Cert files, key stores, trust stores, private keys for SSL
$ pwd
~/github/cassandra-image
$ ls resources/opt/cassandra/conf/certs/
CLIENT_test_PUBLIC.cer cassandra.pks12.keystore test_CLIENT.key.pem
CLUSTER_test_PUBLIC.cer cassandra.truststore
cassandra.keystore test_CLIENT.cer.pem
CLIENT_test_PUBLIC.cer
public client key for the test cluster.cassandra.pks12.keystore
PKS12 keystore for client used to generate pemtest_CLIENT.key.pem
private client key in pem format used by csqlshCLUSTER_test_PUBLIC.cer
public cluster key for the test clustercassandra.truststore
Trust store used by cassandracassandra.keystore
Key store used by cassandratest_CLIENT.cer.pem
public client key in pem format used by csqlsh
For the cassandra_image project, these files are copied to /opt/cassandra/conf/cert
. To learn more about our Vagrant project see Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant.
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS.
Use the keys that we setup.
As part of the provision script for cassandra_image(see Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant). We added the following:
cassandra-image/scripts/040-install-certs.sh - DevOps/DBA task provision Cassandra instances with keys
#!/bin/bash
set -e
DESTINATION_DIRECTORY=/opt/cassandra/conf/certs
SOURCE_DIRECTORY="~/resources$DESTINATION_DIRECTORY"
if [ -d "$SOURCE_DIRECTORY" ]; then
mkdir -p "$DESTINATION_DIRECTORY"
cp -r "$SOURCE_DIRECTORY" "$DESTINATION_DIRECTORY"
fi
if [ ! -d "$SOURCE_DIRECTORY" ]; then
echo "UNABLE TO INSTALL CERTS AS THEY WERE NOT FOUND"
fi
This will copy the certs to the right location if you generated a folder in resources (cassandra_image/resources/opt/cassandra/conf/cert), which the last script that we covered does.
Configure Cassandra to use the keys.
/opt/cassandra/conf - DevOps task configure yaml with SSL keystore and trust stores
server_encryption_options:
internode_encryption: all
keystore: /opt/cassandra/conf/certs/cassandra.keystore
keystore_password: cassandra
truststore: /opt/cassandra/conf/certs/cassandra.truststore
truststore_password: cassandra
# More advanced defaults below:
protocol: TLS
client_encryption_options:
enabled: true
# If enabled and optional is set to true encrypted and unencrypted connections are handled.
optional: false
keystore: /opt/cassandra/conf/certs/cassandra.keystore
keystore_password: cassandra
truststore: /opt/cassandra/conf/certs/cassandra.truststore
truststore_password: cassandra
require_client_auth: true
protocol: TLS
Now let’s test it.
DBA/DevOps Testing that our Cassandra nodes can talk to each other
$ vagrant up
# Get a coffee and otherwise relax for a minute.
# Now log into one of the nodes.
$ vagrant ssh node0
# Now check to see if the cluster is formed.
[vagrant@localhost ~]$ /opt/cassandra/bin/nodetool describecluster
Cluster Information:
Name: test
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
86afa796-d883-3932-aa73-6b017cef0d19: [192.168.50.4, 192.168.50.5, 192.168.50.6]
We can see that the servers in the cluster can connect to each other (see Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant).
Setup csql client
We are doing this in OSX. In this example, we have the virtual machines running CentOS 7 with Vagrant on VirtualBox. We can connect to those instances with Cassandra cqlsh
.
Let’s copy cert files so we can access them from the client (MacBook pro / OSX).
DBA/DevOps task: Copy cert files created earlier.
$ cd ~/github/cassandra-image/resources/opt/cassandra/conf/certs
$ mkdir /opt/cassandra/conf/certs
$ cp * /opt/cassandra/conf/certs
Now we will create a cqlshrc which is a file that dictates how we connect to Cassandra.
Create the cqlshrc in ~/.cassandra
$ mkdir ~/.cassandra
$ cd ~/.cassandra
$ touch cqlshrc
# edit this file
~/.cassandra/cqlshrc contents
[connection]
hostname = 192.168.50.4
port = 9042
factory = cqlshlib.ssl.ssl_transport_factory
[ssl]
certfile = /opt/cassandra/conf/certs/test_CLIENT.cer.pem
validate = false
# Next 2 lines must be provided when require_client_auth = true in the cassandra.yaml file
userkey = /opt/cassandra/conf/certs/test_CLIENT.key.pem
usercert = /opt/cassandra/conf/certs/test_CLIENT.cer.pem
Note we specify the nodes and we are using the pem file as our credentials via SSL to prove who we are instead of a username/password. (We could use both.) We need the userkey
and usercert
in the cqlshrc
because we set require_client_auth = true
in the cassandra.yaml
file for the cluster nodes.
Now let’s test that the client connection works with SSL via cqlsh. Testing is an important step of DevOps automation as well as automating Cassandra DBA tasks.
Testing client connection using cqlsh
$ /opt/cassandra/bin/cqlsh --ssl
Connected to test at 192.168.50.4:9042.
[cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4]
Use HELP for help.
Notice we can connect to Cassandra using SSL. Notice that we are connected to the test
cluster, which
is the cluster we setup in Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant.
Review
We setup keys for client and clustering. We deployed keys to three Linux boxes using Vagrant provisioning.
We then setup cqlsh
to use SSL. We then logged into one of the nodes and checked that the network was setup with the nodetool describecluster
. Then we locally setup csqlsh
to connect to the cluster using SSL.
Understanding SSL keystores, truststores, and Cassandra SSL config is essential for doing DevOps operations with Cassandra.
More to come.
Check back with us at the Cloudurable blog to find out more about cassandra-image
and cassandra-cloud
.
About us
Cloudurable™ streamline DevOps/DBA for Cassandra running on AWS provides AMIs, CloudWatch Monitoring, CloudFormation templates and monitoring tools to support Cassandra in production running in EC2. We also teach advanced Cassandra courses which teaches how one could develop, support and deploy Cassandra to production in AWS EC2.
More info
Please take some time to read the Advantage of using Cloudurable™.
Cloudurable provides:
- Subscription Cassandra support to streamline DevOps/DBA (Support subscription pricing for Cassandra and Kafka in AWS)
- Quickstart Mentoring Consulting for Developers and DevOps/DBA
- Architectural Analysis Consulting
- Training and mentoring for Cassandra Database and Kafka
References
- Carpenter, Jeff; Hewitt, Eben (2016-06-29). Cassandra: The Definitive Guide: Distributed Data at Web Scale. O’Reilly Media.
- JKS Cheatsheet
- Java SSL support
- FIPS
- Using Nodetool securely
- Cassandra SSL certificates
- Cassandra SSL transport for clients
- Using cqlsh with SSL
- Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant
- Configuring cqlsh from a file.
- We specialize in AWS Cassandra deployments for organizations that are setting up Cassandra as a Service.
Authors
Written by R. Hightower and JP Azar.
Article Resources
- Source code for this article
- The first article in this series was about setting up a Cassandra cluster with Vagrant
- Cassandra DevOps/DBA Support
- Cassandra Training
- First article on DZone with some additional content DZone DevOps/DBA Setting up a Cassandra Cluster with Vagrant
- The second article in this series: setting up SSL for a Cassandra cluster using Vagrant
- Second article on DZone with additional content: DZone DevOps/DBA Setting up a Cassandra Cluster with SSL
- Third article in this series on doing DevOps/DBA with ansible, vagrant, ssh-agent for Cassandra.
- Fourth article in this series on Cloud DevOps/DBA: Using Packer, Ansible/SSH and AWS command line tools to create and manage EC2 Cassandra instances in AWS
- Cloudurable Cassandra Image for Docker, AWS, Packer and Vagrant
Feedback
We hope you enjoyed this article. Please provide feedback.
About Cloudurable
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS. Cloudurable also provides Kafka training, Kafka consulting, Kafka support and helps setting up Kafka clusters in AWS.
Check out our new GoLang course. We provide onsite Go Lang training which is instructor led.
TweetApache Spark Training
Kafka Tutorial
Akka Consulting
Cassandra Training
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Non-stop Cassandra
Watchdog
Advantages of using Cloudurable™
Cassandra Consulting
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Training
Kafka Consulting
DynamoDB Training
DynamoDB Consulting
Kinesis Training
Kinesis Consulting
Kafka Tutorial PDF
Kubernetes Security Training
Redis Consulting
Redis Training
ElasticSearch / ELK Consulting
ElasticSearch Training
InfluxDB/TICK Training TICK Consulting