Cloud DevOps: Using Packer, Ansible/SSH and AWS command line tools to create and DBA manage EC2 Cassandra instances in AWS. This article is useful for developers and DevOps/DBA staff who want to create AWS AMI images and manage those EC2 instances with Ansible. Although this article is part of a series about setting up the Cassandra Database images and doing DevOps/DBA with Cassandra clusters, the topics we cover apply to AWS DevOps in general - even if you don’t use Cassandra at all.
Setting up client and cluster SSL transport for a Cassandra cluster This articles is a Cassandra tutorial on Cassandra setup for SSL and CQL clients, as well as installing Cassandra with SSL configured on a series of Linux servers. Cassandra allows you to secure the client transport (CQL) as well as the cluster transport (storage transport). SSL/TLS have some overhead. This is especially true in the JVM world which is not as performant for handling SSL/TLS unless you are using Netty/OpenSSl integration.
Setting up a Cassandra cluster with cassandra image and cassandra cloud project with Vagrant for DevOps
The cassandra-image project creates CentOS Cassandra Database images for docker, virtualbox/vagrant and AWS/EC2 using best practices for Cassandra OS setup. It is nice to use vagrant and/or docker for local development. At this time it is hard to develop systemd services using Docker so we use Vagrant. Since we do a lot of that, we like to use Vagrant. Vagrant is important for developers and DevOps not to mention Cassandra DBAs.
We use systemd unit quite a bit. Getting dependencies correct can be tricky. We use systemd to start up Cloudurable Cassandra config scripts. We use systemd to start up Cassandra/Kafka, and to shut Cassandra/Kafka down nicely. Since systemd is pervasive in all new mainstream Linux distributions, you can see that systemd is an important concept for DevOps. We wrote this little example to try to understand how systemd dependencies work, and explain it to others.
Notes on Cassandra OS setup and optimizations for deploying in EC2/AWS Disk concerns These are important concepts for developers and DevOps who are responsible for developing Cassandra based applications and services. Cassandra writes to four areas commit logs SSTable an index file a bloom filter The compaction process of SSTable data makes heavy use of the disk. LeveledCompactionStrategy may need 10 to 20% overhead. SizeTieredCompactionStrategy worse case is 50% overhead needed to perform compaction.
Understanding what AWS provides for setting up private networks, security groups and more is important for anyone who calls themselves DevOps. AWS allows you to define a software defined network. You do this with Amazon Virtual Private Cloud (Amazon VPC). You can define subnets, ingress rules, security groups, NAT gateways, Internet gateways, and more. Amazon VPC A VPC is a virtual private cloud. You can create multiple Amazon VPCs within a region that spans multiple availability zones.
Understanding what AWS provides for backing up EBS volumes is an important concept for DevOps. Data safety with EBS - Backup/Recovery (Snapshots) Amazon EBS allows you to easily backup data. You do this by taking snapshots. Snapshots are point-in-time backups. Data written to an EBS volume can be periodically used to create a snapshot. Snapshots provide incremental backups of your data. Snapshots just saves the blocks that have changed. Only changed blocks since the last snapshot are saved in the new snapshot.
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Advantages of using Cloudurable™
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Tutorial PDF
ElasticSearch / ELK Consulting
InfluxDB/TICK Training TICK Consulting