March 14, 2017
AWS Cassandra and NUMA
The i3.8xlarge
, c4.8xlarge
, m4.10xlarge
, and above EC2 instance types use more than 1 CPU, which means NUMA controls are available.
A good read on this is from Al Tolbert’s blog post.
The quickest way to tell if a machine is NUMA is to run “numactl –hardware”. -Al Tobey blog post on Cassandra tuning
NUMA stands for Non-Uniform Memory Architecture. Modern x86 CPUs contain an integrated memory controller. Multi-socket system, have two memory controllers. Each CPU gets a share of the memory. If one CPU socket needs memory that another CPU socket has, the memory is transferred. Transferring this memory between CPUs is more expensive than if the memory only existed in one CPUs memory. When a JVM thread only uses memory local to one CPU, things go fast, and if not slower (10 CPU cycles vs. 100 or some order of magnitude).
We hope this information on Cassandra NUMA for AWS. We also provide Casandra consulting and Kafka consulting. Please check out our Casandra training and Kafka training. We specialize in AWS DevOps Automation for Cassandra and Kafka.
The Cassandra
startup script sets JVM numactl --interleave
.
One is to comment out the
numactl --interleave
inbin/cassandra
and add-XX:+UseNUMA
tocassandra-env.sh
. –Al Tolbert
This setting allows the Cassandra JVM to handle NUMA directly. The GC (GC G1) will divide GC efforts across domains, which should make GC more efficient.
What about CPU pinning? Rather than using numactl --cpunodebind
to pin a JVM to a particular node, you should consider running your EC2 instance on a smaller instance type. Also since Cassandra uses modified SEDA, it should do okay with thread memory boundaries. However, if you are running on a larger EC2 instance and you are running other JVM processes on the same instance (perhaps pairing it with SOLR, Spark or others for some locality/integration benefits), then consider using numactl --cpunodebind
. The Cassandra JVM gets bound to NUMA node, so all memory is local, and all threads will execute on the same core. Use with caution, and since the Cassandra uses a modified SEDA, it may not benefit as much as other Java applications.
Please read Al Tobey complete Cassandra tuning guide. It is a great resource.
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS.
More info about Cassandra and AWS
Read more about Cassandra AWS with this slide deck.
Amazon has a guide that covers Cassandra on AWS that is a must read There is also this Amazon Cassandra guide on High Scalability that is a must read.
About Cloudurable™
Cloudurable™: streamline DevOps/DBA for Cassandra running on AWS. Cloudurable™ provides AMIs, CloudWatch Monitoring, CloudFormation templates and monitoring tools to support Cassandra in production running in EC2. We also teach advanced Cassandra courses which teaches how one could develop, support and deploy Cassandra to production in AWS EC2 for Developers and DevOps/DBA. We also provide Cassandra consulting and Cassandra training.
Follow Cloudurable™ at our LinkedIn page, Facebook page, Google plus or Twitter.
More info about Cloudurable
Please take some time to read the Advantage of using Cloudurable™.
Cloudurable provides:
- Subscription Cassandra support to streamline DevOps (Support subscription pricing for Cassandra and Kafka in AWS)
- Quickstart Mentoring Consulting for Developers and DevOps
- Architectural Analysis Consulting
- Training and mentoring for Cassandra for DevOps/DBA and Developers
- Training and mentoring for Apache Kafka for DevOps and Developers
- We specialize in AWS Cassandra deployments for organizations that are setting up Cassandra as a Service.
Authors
Written by R. Hightower and JP Azar.
Feedback
We hope you enjoyed this article. Please provide feedback.
About Cloudurable
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS. Cloudurable also provides Kafka training, Kafka consulting, Kafka support and helps setting up Kafka clusters in AWS.
Check out our new GoLang course. We provide onsite Go Lang training which is instructor led.
TweetApache Spark Training
Kafka Tutorial
Akka Consulting
Cassandra Training
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Non-stop Cassandra
Watchdog
Advantages of using Cloudurable™
Cassandra Consulting
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Training
Kafka Consulting
DynamoDB Training
DynamoDB Consulting
Kinesis Training
Kinesis Consulting
Kafka Tutorial PDF
Kubernetes Security Training
Redis Consulting
Redis Training
ElasticSearch / ELK Consulting
ElasticSearch Training
InfluxDB/TICK Training TICK Consulting