March 14, 2017
AWS Cassandra and NUMA
m4.10xlarge, and above EC2 instance types use more than 1 CPU, which means NUMA controls are available.
A good read on this is from Al Tolbert’s blog post.
The quickest way to tell if a machine is NUMA is to run “numactl –hardware”. -Al Tobey blog post on Cassandra tuning
NUMA stands for Non-Uniform Memory Architecture. Modern x86 CPUs contain an integrated memory controller. Multi-socket system, have two memory controllers. Each CPU gets a share of the memory. If one CPU socket needs memory that another CPU socket has, the memory is transferred. Transferring this memory between CPUs is more expensive than if the memory only existed in one CPUs memory. When a JVM thread only uses memory local to one CPU, things go fast, and if not slower (10 CPU cycles vs. 100 or some order of magnitude).
We hope this information on Cassandra NUMA for AWS. We also provide Casandra consulting and Kafka consulting. Please check out our Casandra training and Kafka training. We specialize in AWS DevOps Automation for Cassandra and Kafka.
Cassandra startup script sets JVM
One is to comment out the
cassandra-env.sh. –Al Tolbert
This setting allows the Cassandra JVM to handle NUMA directly. The GC (GC G1) will divide GC efforts across domains, which should make GC more efficient.
What about CPU pinning? Rather than using
numactl --cpunodebind to pin a JVM to a particular node, you should consider running your EC2 instance on a smaller instance type. Also since Cassandra uses modified SEDA, it should do okay with thread memory boundaries. However, if you are running on a larger EC2 instance and you are running other JVM processes on the same instance (perhaps pairing it with SOLR, Spark or others for some locality/integration benefits), then consider using
numactl --cpunodebind. The Cassandra JVM gets bound to NUMA node, so all memory is local, and all threads will execute on the same core. Use with caution, and since the Cassandra uses a modified SEDA, it may not benefit as much as other Java applications.
Please read Al Tobey complete Cassandra tuning guide. It is a great resource.
More info about Cassandra and AWS
Read more about Cassandra AWS with this slide deck.
Cloudurable™: streamline DevOps/DBA for Cassandra running on AWS. Cloudurable™ provides AMIs, CloudWatch Monitoring, CloudFormation templates and monitoring tools to support Cassandra in production running in EC2. We also teach advanced Cassandra courses which teaches how one could develop, support and deploy Cassandra to production in AWS EC2 for Developers and DevOps/DBA. We also provide Cassandra consulting and Cassandra training.
More info about Cloudurable
Please take some time to read the Advantage of using Cloudurable™.
- Subscription Cassandra support to streamline DevOps (Support subscription pricing for Cassandra and Kafka in AWS)
- Quickstart Mentoring Consulting for Developers and DevOps
- Architectural Analysis Consulting
- Training and mentoring for Cassandra for DevOps/DBA and Developers
- Training and mentoring for Apache Kafka for DevOps and Developers
- We specialize in AWS Cassandra deployments for organizations that are setting up Cassandra as a Service.
Written by R. Hightower and JP Azar.
We hope you enjoyed this article. Please provide feedback.
Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS. Cloudurable also provides Kafka training, Kafka consulting, Kafka support and helps setting up Kafka clusters in AWS.
Check out our new GoLang course. We provide onsite Go Lang training which is instructor led.Tweet
Apache Spark Training
AWS Cassandra Database Support
Kafka Support Pricing
Cassandra Database Support Pricing
Advantages of using Cloudurable™
Cloudurable™| Guide to AWS Cassandra Deploy
Cloudurable™| AWS Cassandra Guidelines and Notes
Free guide to deploying Cassandra on AWS
Kafka Tutorial PDF
Kubernetes Security Training
ElasticSearch / ELK Consulting
InfluxDB/TICK Training TICK Consulting