AWS Cassandra: Cassandra, NUMA and EC2

March 14, 2017

                                                                           

AWS Cassandra and NUMA

The i3.8xlarge, c4.8xlarge, m4.10xlarge, and above EC2 instance types use more than 1 CPU, which means NUMA controls are available.

A good read on this is from Al Tolbert’s blog post.

The quickest way to tell if a machine is NUMA is to run “numactl –hardware”. -Al Tobey blog post on Cassandra tuning

NUMA stands for Non-Uniform Memory Architecture. Modern x86 CPUs contain an integrated memory controller. Multi-socket system, have two memory controllers. Each CPU gets a share of the memory. If one CPU socket needs memory that another CPU socket has, the memory is transferred. Transferring this memory between CPUs is more expensive than if the memory only existed in one CPUs memory. When a JVM thread only uses memory local to one CPU, things go fast, and if not slower (10 CPU cycles vs. 100 or some order of magnitude).

We hope this information on Cassandra NUMA for AWS. We also provide Casandra consulting and Kafka consulting. Please check out our Casandra training and Kafka training. We specialize in AWS DevOps Automation for Cassandra and Kafka.

The Cassandra startup script sets JVM numactl --interleave.

One is to comment out the numactl --interleave in bin/cassandra and add -XX:+UseNUMA to cassandra-env.sh. –Al Tolbert

This setting allows the Cassandra JVM to handle NUMA directly. The GC (GC G1) will divide GC efforts across domains, which should make GC more efficient.

What about CPU pinning? Rather than using numactl --cpunodebind to pin a JVM to a particular node, you should consider running your EC2 instance on a smaller instance type. Also since Cassandra uses modified SEDA, it should do okay with thread memory boundaries. However, if you are running on a larger EC2 instance and you are running other JVM processes on the same instance (perhaps pairing it with SOLR, Spark or others for some locality/integration benefits), then consider using numactl --cpunodebind. The Cassandra JVM gets bound to NUMA node, so all memory is local, and all threads will execute on the same core. Use with caution, and since the Cassandra uses a modified SEDA, it may not benefit as much as other Java applications.

Please read Al Tobey complete Cassandra tuning guide. It is a great resource.

Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS.

More info about Cassandra and AWS

Read more about Cassandra AWS with this slide deck.

Amazon has a guide that covers Cassandra on AWS that is a must read There is also this Amazon Cassandra guide on High Scalability that is a must read.

About Cloudurable™

Cloudurable™: streamline DevOps/DBA for Cassandra running on AWS. Cloudurable™ provides AMIs, CloudWatch Monitoring, CloudFormation templates and monitoring tools to support Cassandra in production running in EC2. We also teach advanced Cassandra courses which teaches how one could develop, support and deploy Cassandra to production in AWS EC2 for Developers and DevOps/DBA. We also provide Cassandra consulting and Cassandra training.

Follow Cloudurable™ at our LinkedIn page, Facebook page, Google plus or Twitter.

More info about Cloudurable

Please take some time to read the Advantage of using Cloudurable™.

Cloudurable provides:

Authors

Written by R. Hightower and JP Azar.

Feedback


We hope you enjoyed this article. Please provide feedback.

About Cloudurable

Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS. Cloudurable also provides Kafka training, Kafka consulting, Kafka support and helps setting up Kafka clusters in AWS.

Check out our new GoLang course. We provide onsite Go Lang training which is instructor led.

                                                                           

Apache Spark Training
Kafka Tutorial
Akka Consulting
Cassandra Training
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Non-stop Cassandra
Watchdog
Advantages of using Cloudurable™
Cassandra Consulting
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Training
Kafka Consulting
DynamoDB Training
DynamoDB Consulting
Kinesis Training
Kinesis Consulting
Kafka Tutorial PDF
Redis Consulting
Redis Training
ElasticSearch / ELK Consulting
ElasticSearch Training
InfluxDB/TICK Training TICK Consulting