February 1, 2017
The cassandra-image
project creates CentOS Cassandra Database images for docker, virtualbox/vagrant and AWS/EC2 using best practices
for Cassandra OS setup. It is nice to use vagrant and/or docker for local development. At this time it is hard to develop systemd services using Docker so we use Vagrant. Since we do a lot of that, we like to use Vagrant.
Vagrant is important for developers and DevOps not to mention Cassandra DBAs.
The cassandra-image
project packages systemd utilities
- to monitor the OS and send metrics to AWS CloudWatch metrics
- to monitor logs from the OS and send them to AWS CloudWatch logs.
- to monitor logs from the Cassandra Database and send them to AWS CloudWatch Logs.
- to monitor the Cassandra Database stats and send them to AWS CloudWatch Metrics
Do you like this article? Please check out our Casandra training and Kafka training. We specialize in AWS DevOps Automation for Cassandra and Kafka.
The cassandra-image project uses the Cassandra cloud project to configure Cassandra running in instances to aid in setting up the cluster.
With this in mind, let’s setup Vagrant to launch a Cassandra Database cluster locally. Consider this a Cassandra tutorial on how to setup a Cassandra Cluster. We cover Cassandra install via Vagrant and Cassandra configuration of Cassandra seed node servers (Cassandra seeds).
We hope this blog post on using the Cassandra image project useful. We find it helps with Cassandra deploys. We also provide Casandra consulting and Kafka consulting to get you setup fast in AWS with CloudFormation and CloudWatch. Check out our Casandra training and Kafka training. Cloudurable specialize in AWS DevOps Automation for Cassandra and Kafka.
We are going to setup three nodes as follows that use our provision scripts to install the Cassandra Database and utilities:
- 192.168.50.4 node0
- 192.168.50.5 node1
- 192.168.50.5 node2
These nodes will about be Cassandra seeds.
Setup network of boxes for DevOps/DBA testing to form the Cassandra Cluster
Vagrant.configure("2") do |config|
# Use CentOS 7
config.vm.box = "centos/7"
# Setup 4 cpus and 3096 MB of memory for each instance
config.vm.provider "virtualbox" do |vb|
vb.memory = "3096"
vb.cpus = 4
end
# Run the provision install scripts
config.vm.provision "shell", inline: <<-SHELL
sudo /vagrant/scripts/000-vagrant-provision.sh
SHELL
config.vm.define "node0" do |node0|
...
# Node 0 is 192.168.50.4
node0.vm.network "private_network", ip: "192.168.50.4"
...
end
config.vm.define "node1" do |node1|
...
# Node 1 is 192.168.50.5
node1.vm.network "private_network", ip: "192.168.50.5"
...
end
config.vm.define "node2" do |node2|
...
# Node 2 is 192.168.50.6
node2.vm.network "private_network", ip: "192.168.50.6"
end
In this example we will use these three servers as Cassandra seed nodes in our Cassandra Cluster.
Cassandra seed nodes are nodes that are first contacted by nodes that join the Cassandra cluster.
When you use Cassandra to add a node, these Cassandra seed nodes are contacted by the Cassandra node you are adding.
It is a good idea to have two or three of them as one would be a SPOF (single point of failure).
In this example we will use the utility cassandra-cloud
to configure the seed nodes.
We will also use cassandra-cloud
to tell the Cassandra Database which address to listen on for clustering (storage network), and which address to listen on for client connections. Vagrant allows us to test DevOps tasks locally.
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS.
Using the Cassandra Cloud from Vagrant to setup Cassandra seeds for Cassandra Cluster
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
...
config.vm.define "node0" do |node0|
...
node0.vm.network "private_network", ip: "192.168.50.4"
### Use Cassandra cloud tool to configure the Cassandra Database before launching it.
### Set the cluster name to test, set the client-address and the cluster-address.
### Also setup the the Cassandra Database seed nodes.
node0.vm.provision "shell", inline: <<-SHELL
sudo /opt/cassandra/bin/cassandra-cloud -cluster-name test \
-client-address 192.168.50.4 \
-cluster-address 192.168.50.4 \
-cluster-seeds 192.168.50.4,192.168.50.5,192.168.50.6
/opt/cassandra/bin/cassandra -R
SHELL
end
config.vm.define "node1" do |node1|
...
node1.vm.provision "shell", inline: <<-SHELL
sudo /opt/cassandra/bin/cassandra-cloud -cluster-name test \
-client-address 192.168.50.5 \
-cluster-address 192.168.50.5 \
-cluster-seeds 192.168.50.4,192.168.50.5,192.168.50.6
/opt/cassandra/bin/cassandra -R
SHELL
end
config.vm.define "node2" do |node2|
...
node2.vm.provision "shell", inline: <<-SHELL
sudo /opt/cassandra/bin/cassandra-cloud -cluster-name test \
-client-address 192.168.50.6 \
-cluster-address 192.168.50.6 \
-cluster-seeds 192.168.50.4,192.168.50.5,192.168.50.6
/opt/cassandra/bin/cassandra -R
SHELL
end
...
end
The utility cassandra-cloud
can read setting from Environment Variables so that it can work well in Mesos, Docker, Heroku, etc. It can also read properties from a config file. Cassandra cloud is all about installing Cassandra configuration.
It can also read properties from the command line. The cassandra-image
creates a cassandra-cloud
config file that can be modified. The cassandra-cloud
utility can setup memory, threads, number of workers, etc. for the Cassandra Database. You can set Cassandra configuration values explicitly or they can be set by looking that the ergonomics of the server. This gives us a flexible, elastic way to resize Cassandra configuration as part of DevOps capacity planning if needed. It is a way to dynamically install Cassandra on Amazon.
Ok. Let’s test our cluster out.
DevOps/DBA Testing of our Cassandra cluster setup
$ vagrant up
$ vagrant ssh node0
[vagrant@localhost ~]$ ps -ef | grep cassandra
root 12414 1 2 19:16 ? 00:00:26 java -Xloggc:/opt/cassandra/bin/../logs/gc.log
...
$ /opt/cassandra/bin/nodetool describecluster
Cluster Information:
Name: test
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
86afa796-d883-3932-aa73-6b017cef0d19: [192.168.50.4, 192.168.50.5, 192.168.50.6]
We can see that we have a Cassandra cluster of three Cassandra seed servers. Now we can use these to test DevOps and DBA tasks against like setting up SSL keys for the Cassandra Database, SSH keys, and using ansible.
You can see the full Vagrantfile on github.
More to come.
Check back with us at the Cloudurable blog to find out more about cassandra-image
and cassandra-cloud
. Cassandra Cloud tools are essential for deploying AWS Cassandra. It makes using Amazon Cassandra instances easier as they can be resized, and the Cassandra install can be customized per EC2 instance type.
About us
Cloudurable™ streamline DevOps and DBA for the Cassandra Database running on AWS provides AMIs, CloudWatch Monitoring, CloudFormation templates and monitoring tools to support the Cassandra Database in production running in Amazon AWS. We also teach advanced Cassandra Database courses which teaches how one could develop, perform DBA tasks, support and deploy Cassandra to production in AWS EC2.
More info
Please take some time to read the Advantage of using Cloudurable™ for Amazon Cassandra deployments.
Cloudurable provides:
- Subscription Cassandra Database Database support to streamline DevOps and DBA tasks (Support subscription pricing for the Cassandra Database and Kafka in AWS)
- Quickstart Mentoring Consulting for Developers and DevOps
- Architectural Analysis Consulting
- Training and mentoring for the Cassandra Database and Kafka
- We specialize in AWS Cassandra deployments for organizations that are setting up Cassandra as a Service.
Authors
Written by R. Hightower and JP Azar.
Article Resources
- Source code for this article
- The first article in this series was about setting up a Cassandra Database cluster with Vagrant
- Cassandra Database DevOps/DBA Support
- Cassandra Training
- First article on DZone with some additional content DZone DevOps/DBA Setting up a Cassandra Cluster with Vagrant
- The second article in this series: setting up SSL for a Cassandra cluster using Vagrant
- Second article on DZone with additional content: DZone DevOps/DBA Setting up a Cassandra Cluster with SSL
- Third article in this series on doing DevOps/DBA with ansible, vagrant, ssh-agent for Cassandra.
- Fourth article in this series on Cloud DevOps/DBA: Using Packer, Ansible/SSH and AWS command line tools to create and manage EC2 Cassandra instances in AWS
- Cloudurable Cassandra Database Image for Docker, AWS, Packer and Vagrant
Feedback
We hope you enjoyed this article. Please provide feedback.
About Cloudurable
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS. Cloudurable also provides Kafka training, Kafka consulting, Kafka support and helps setting up Kafka clusters in AWS.
Check out our new GoLang course. We provide onsite Go Lang training which is instructor led.
TweetApache Spark Training
Kafka Tutorial
Akka Consulting
Cassandra Training
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Non-stop Cassandra
Watchdog
Advantages of using Cloudurable™
Cassandra Consulting
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Training
Kafka Consulting
DynamoDB Training
DynamoDB Consulting
Kinesis Training
Kinesis Consulting
Kafka Tutorial PDF
Kubernetes Security Training
Redis Consulting
Redis Training
ElasticSearch / ELK Consulting
ElasticSearch Training
InfluxDB/TICK Training TICK Consulting