AWS EBS Backup and Recovery Guide 2025: Snapshots, DLM, and AWS Backup

What’s New in 2025

Key Updates and Changes

• AWS Backup Integration: Centralized backup management with deletion protection
• Enhanced DLM: Cron expressions, multi-schedule policies (up to 4 per policy)
• Cross-Region Automation: Automated snapshot copying across 3 regions
• Incremental Cross-Region: Optimized data transfer for regional copies
• EventBridge Integration: Real-time snapshot event monitoring

Major Improvements

• Snapshot Locking: Prevent accidental deletion with retention locks
• Fast Snapshot Restore: Pre-warm snapshots for instant recovery
• Archive Tier: 75% cost savings for long-term retention
• Recycle Bin: Recover accidentally deleted snapshots
• Tags-Based Automation: Enhanced tag-based lifecycle management

Understanding what AWS provides for backing up EBS volumes is an important concept for DevOps. In 2025, AWS offers multiple sophisticated options for automating and managing backups.

Continue reading

AWS EC2 Compute Guide 2025: Instance Types and Best Practices

What’s New in 2025

Key Updates and Changes

• 7th Generation Intel Instances: M7i, C7i, and R7i families with Sapphire Rapids processors
• 15% Better Performance: Custom Intel processors exclusive to AWS
• Graviton3 Dominance: Up to 40% better price-performance than Intel counterparts
• Enhanced Networking: Up to 200 Gbps network performance on select instances
• Intel Advanced Features: AMX for ML workloads, QAT, IAA, and DSA accelerators

Instance Family Updates

• Deprecated: M3, C3 instances - migrate to M7i, C7i
• New Compute: C7i with Intel AMX for ML inference
• Memory Optimized: R7i supports up to 24TB memory
• Storage Optimized: I4i with up to 30TB NVMe SSD
• ML/AI Focus: P5 instances with H100 GPUs, Trn1 with Trainium chips

Major AWS Announcements

• EC2 Capacity Blocks: Reserve GPU capacity for ML training
• Spot Instance Integration: Better integration with EKS and ECS
• Windows Server 2025: Full support with optimized AMIs

Understanding what AWS/EC2 provides for provisioning on-demand computing is essential for all DevOps. In 2025, choosing the right instance type can reduce costs by 40% or more.

Continue reading

AWS VPC Guide 2025: From Basic Networking to VPC Lattice

What’s New in 2025

Key Updates and Changes

• VPC Lattice GA: Application-layer networking for microservices across VPCs
• IPAM Tiers: Free tier for single region, Advanced tier ($0.00027/IP/hour) for multi-region
• IPv6 Adoption: Dual-stack configurations now standard practice
• Enhanced Security: VPC Lattice enables zero-trust networking patterns
• Service Network Endpoints: Connect services across accounts without complex routing

Major Service Evolution

• VPC Peering: Still relevant for simple, pairwise connections
• Transit Gateway: Remains the choice for hub-and-spoke architectures
• VPC Lattice: New paradigm for application-layer connectivity
• IPAM Integration: Automated IP management across organizations
• Resource Gateways: New construct for secure resource sharing

Understanding what AWS provides for setting up private networks, security groups and more is important for anyone who calls themselves DevOps. In 2025, the networking landscape has evolved significantly with VPC Lattice and enhanced IPAM capabilities.

Continue reading

Most bang for your buck with AWS Elastic Block Store (EBS) in 2025

What’s New in 2025

Key Updates and Changes

• GP3 Now Standard: GP3 volumes offer 20% cost savings over GP2 with independent IOPS/throughput provisioning
• IO2 Block Express: Supports up to 256,000 IOPS and 4,000 MB/s throughput
• Multi-Attach Support: IO2 volumes can now attach to multiple EC2 instances simultaneously
• Enhanced Durability: IO2 offers 99.999% durability vs IO1’s 99.9%
• Elastic Volumes: Modify volume type, size, or performance without downtime

Migration and Cost Optimization

• GP2 to GP3: Save 20% per GB with better baseline performance (3,000 IOPS, 125 MB/s)
• IO1 to IO2: Get 500 IOPS per GB (vs 50 for IO1) at similar pricing
• Volume Modification: Changes typically complete within 6 hours for 1TB volumes

Getting the most bang for your buck with AWS Elastic Block Store (EBS)

Understanding what AWS/EC2 provides for provisioning on-demand storage is critical for DevOps. Companies waste tons by over provisioning AWS. In 2025, with GP3 volumes and IO2 Block Express, there are more opportunities than ever to optimize costs while improving performance.

Continue reading

Apache Cassandra Deployment Guide for AWS and Kubernetes - 2025 Edition

🚀 What’s New in This 2025 Update

Major Changes Since 2017

• Cassandra 5.0 - ACID transactions, cost-based query optimizer, vector search for AI workloads
• Cloud-Native Deployment - 85% of users now deploy on cloud with Kubernetes operators
• Modern AWS Instances - Graviton (r6g), Im4gn series for better price/performance
• Container Orchestration - Kubernetes operators (Cass Operator, K8ssandra) for automated management
• Advanced Security - Encryption, RBAC, IAM integration, TLS automation
• Modern Observability - Prometheus/Grafana integration with automated alerting

Key Improvements

• ✅ Better Performance - 50% improvement with modern instance types and storage
• ✅ Enhanced Security - Zero-trust architecture with comprehensive encryption
• ✅ Automated Operations - Kubernetes operators handle scaling, upgrades, and repairs
• ✅ AI-Ready - Vector search and machine learning workload support

Modern Cassandra Architecture Overview

Apache Cassandra 5.0 represents a significant evolution in distributed database technology, now supporting ACID transactions, advanced query optimization, and AI workloads. Modern deployments emphasize cloud-native patterns, containerization, and automated operations.

Continue reading

Kafka Broker Startup Scripts and Configuration - 2025 Edition

🚀 What’s New in This 2025 Update

Major Updates and Changes

• KRaft Mode by Default - No ZooKeeper required
• Container-First Approach - Docker and Kubernetes native
• Dynamic Configuration - Minimal restarts needed
• Automated Management - Operators handle lifecycle
• Enhanced Monitoring - Built-in observability
• Cloud-Native Patterns - Auto-scaling and self-healing

Broker Startup Evolution Since 2017

• ✅ No ZooKeeper - KRaft provides native consensus
• ✅ Declarative Config - GitOps and Infrastructure as Code
• ✅ Container Ready - Optimized for Kubernetes
• ✅ Zero-Downtime - Rolling updates standard

Running a Kafka Broker in 2025

Starting Kafka brokers has evolved significantly with KRaft mode and cloud-native practices. This guide covers modern approaches from simple local development to production Kubernetes deployments.

Continue reading

AWS Cassandra Cluster Tutorial 5 (2025): Modern Cassandra Deployment with CDK, EKS, and Infrastructure as Code

Cassandra Cluster Tutorial 5 (2025) - Modern AWS Cassandra Deployment with CDK, EKS, and Infrastructure as Code

This Cassandra tutorial is designed for developers and DevOps/SRE teams who want to deploy production-ready Cassandra clusters in AWS using modern practices and tools available in 2025.

What’s New in 2025

The landscape of deploying Cassandra on AWS has evolved significantly:

1. AWS CDK v2 has become the standard for infrastructure as code, offering type-safe infrastructure definitions
2. Kubernetes operators like K8ssandra provide production-ready Cassandra deployments
3. AWS Graviton3 processors offer 40% better price-performance for Cassandra workloads
4. Container-based deployments are now the norm, with EKS Anywhere for hybrid deployments
5. Service mesh integration with AWS App Mesh provides advanced traffic management
6. AWS Systems Manager replaces bastion hosts for secure access
7. GitOps workflows with AWS CodeCommit and FluxCD for infrastructure management

Cloudurable provides Cassandra training, Cassandra consulting, Cassandra support and helps setting up Cassandra clusters in AWS.

Continue reading

Cassandra 5.0 Cluster Setup 2025: Docker, Vagrant, and Cloud-Native DevOps

What’s New in 2025

Key Updates and Changes

• Cassandra 5.0: Vector search, SAI indexes, unified compaction strategy
• Container-First: Docker and Kubernetes have replaced most Vagrant workflows
• Cloud-Native: Multi-cloud deployment with infrastructure as code
• ARM Support: Native ARM64 support for Apple Silicon and AWS Graviton
• Observability: Enhanced monitoring with OpenTelemetry and Prometheus

Major Platform Evolution

• Docker Compose: Simplified multi-container orchestration
• Kubernetes: Production-ready Cassandra operators
• Testcontainers: Integration testing with ephemeral containers
• Colima/Podman: Docker alternatives for development
• GitOps: Infrastructure managed through Git workflows

The modern approach to Cassandra cluster development has evolved significantly since 2017. While Vagrant remains useful for certain scenarios, container-based development has become the standard for 2025.

Continue reading

Cassandra 5.0 Cluster Tutorial 2025: Ansible Automation for DevOps Tasks

What’s New in 2025

Key Updates and Changes

• Cassandra 5.0: Storage Attached Indexes (SAI), Vector Search, Unified Compaction
• Ansible 2.19: Event-driven automation, enhanced cloud integrations
• VirtualBox Compatibility: Use 6.1.x with Vagrant 2.4.1 for stability
• Security First: Ansible Vault and external secret managers now standard
• Infrastructure as Code: Git-based workflows with Ansible Collections

Deprecated Features

• Cassandra 3.x is end-of-life
• Legacy Ansible inventory formats
• Manual SSH key management (use automation)
• Static inventories for cloud environments

Cassandra Tutorial: Setting up Ansible for our Cassandra Database Cluster for DevOps/DBA tasks

Ansible is a key DevOps/DBA tool for managing backups and rolling upgrades to the Cassandra cluster in AWS/EC2. Ansible uses ssh, so you do not have to install an agent to use it. In 2025, Ansible remains the preferred automation tool with improved event-driven capabilities.

Continue reading

Cassandra Tutorial 3, Part 1: Deploy Cassandra with Ansible and SSH Keys - Modern DevOps Practices 2025

Cassandra Tutorial 3, Part 1: Modern Cassandra Deployment with Ansible - 2025 Edition

What’s New in 2025

This updated tutorial reflects the significant changes in DevOps practices since our original 2017 guide:

• Container-First Approach: Cassandra deployments now primarily use containers (Docker/Podman)
• Kubernetes Native: StatefulSets and operators for managing Cassandra clusters
• GitOps Workflows: Using ArgoCD or Flux for declarative cluster management
• Modern Ansible: Ansible 2.16+ with collections and execution environments
• Enhanced Security: Zero-trust networking, service mesh integration, and secrets management
• Cloud Provider Integration: Native support for AWS, GCP, and Azure Kubernetes services
• Observability: Built-in Prometheus metrics and distributed tracing

Overview

In this modernized tutorial, we’ll deploy a production-ready Cassandra cluster using:

Continue reading

Cassandra Tutorial 3, Part 2: Advanced Ansible Automation and Cloud-Native Operations - 2025 Edition

Cassandra Tutorial 3, Part 2: Advanced Operations and Automation - 2025 Edition

What’s New Since Part 1

Building on Part 1’s foundation, this tutorial covers:

• Multi-Cloud Deployments: Spanning AWS, GCP, and Azure
• Advanced GitOps: Multi-environment promotion workflows
• AI-Powered Operations: Using ML for capacity planning and anomaly detection
• Chaos Engineering: Automated failure testing with Litmus
• FinOps Integration: Cost optimization and resource management
• Zero-Downtime Operations: Rolling upgrades and blue-green deployments

Advanced Ansible Automation

Ansible Execution Environments (2025 Standard)

Create a containerized Ansible environment:

Continue reading

Cloud DevOps 2025: Packer, Ansible, SSH and AWS/EC2

What’s New in 2025

Key Updates and Changes

• New EC2 Instance Types: M7i, C7i, and R7i families now available with up to 15% better price-performance
• Packer Updates: Version 1.11 with predictable plugin loading and HCP integration
• Ansible Best Practices: Enhanced aws_ec2 plugin with improved security and performance features
• EBS Volume Evolution: GP3 volumes now standard, offering 20% cost savings over GP2
• HashiCorp Updates: Terraform AWS Provider 6.0 with multi-region support
• Security Enhancements: AWS Verified Access for SSH/RDP, enhanced IAM with ECR Policy v2

Deprecated Features and Migration Notes

• GP2 to GP3 Migration: GP2 volumes should be migrated to GP3 for cost savings
• EC2 Dynamic Inventory: Old ec2.py script deprecated in favor of aws_ec2 plugin
• Instance Types: Consider upgrading from M6i to M7i instances for better performance
• Packer AWS Builder: Continue using amazon-ebs builder with updated authentication methods

Cloud DevOps: Using Packer, Ansible/SSH and AWS command line tools to create and DBA manage EC2 Cassandra instances in AWS.

This article is useful for developers and DevOps/DBA staff who want to create AWS AMI images and manage those EC2 instances with Ansible. Although this article is part of a series about setting up the Cassandra Database images and doing DevOps/DBA with Cassandra clusters, the topics we cover apply to AWS DevOps in general - even if you don’t use Cassandra at all.

Continue reading

AWS Cassandra Cluster Tutorial 5: Setting up Cassandra Cluster in AWS/EC2

Cassandra Cluster Tutorial 5 - Cassandra AWS Cluster with CloudFormation, bastion host, Ansible and the aws-command line

This Cassandra tutorial is useful for developers and DevOps/DBA staff who want to launch a Cassandra cluster in AWS.

The cassandra-image project has been using Vagrant and Ansible to set up a Cassandra Cluster for local testing. Then we used Packer, Ansible and EC2. We used Packer to create AWS images in the last tutorial. In this tutorial, we will use CloudFormation to create a VPC, Subnets, security groups and more to launch a Cassandra cluster in EC2 using the AWS AMI image we created with Packer in the last article. The next two tutorials after this one, will set up Cassandra to work in multiple AZs and multiple regions using custom snitches for Cassandra.

Continue reading

Part 2 Setting up Ansible and ssh for Cassandra Database Cluster DevOps

Cassandra Cluster Tutorial 3: Part 2 of 2

Setting up Ansible and SSH for our Cassandra Database Cluster for DevOps/DBA Tasks

This tutorial series centers on how DevOps/DBA tasks with the Cassandra Database. As we mentioned before, Ansible and ssh are essential DevOps/DBA tools for common DBA/DevOps tasks whilst working with Cassandra Clusters. Please read part 1 before reading part 2.

In part 1, we set up Ansible for our Cassandra Database Cluster to automate common DevOps/DBA tasks. As part of this setup, we created an ssh key and then set up our instances with this key so we could use ssh, scp, and most importantly ansible. We also created an ansible playbook to install keys on our Cassandra nodes from a bastion host that we set up with Vagrant.

Continue reading

Setting up Ansible/SSH for Cassandra Database Cluster DevOps Part 1

Cassandra Cluster Tutorial 3: Part 1 of 2

Setting up Ansible/SSH for our Cassandra Database Cluster for DevOps/DBA Tasks

Ansible and ssh are essential DevOps/DBA tools for common DBA/DevOps tasks like managing backups, rolling upgrades to the Cassandra cluster in AWS/EC2, and so much more. An excellent aspect of Ansible is that it uses ssh, so you do not have to install an agent to use Ansible.

This article series centers on how DevOps/DBA tasks with the Cassandra Database. However the use of Ansible for DevOps/DBA transcends its use with the Cassandra Database, so this article is good information for any DevOps/DBA or Developer that needs to manage groups of instances, boxes, hosts whether they be on-prem bare-metal, dev boxes, or in the AWS cloud. You don’t need to be setting up Cassandra to get use of this article.

Continue reading

Cloud DevOps: Packer, Ansible, SSH and AWS/EC2

Cloud DevOps: Using Packer, Ansible/SSH and AWS command line tools to create and DBA manage EC2 Cassandra instances in AWS.

This article is useful for developers and DevOps/DBA staff who want to create AWS AMI images and manage those EC2 instances with Ansible. Although this article is part of a series about setting up the Cassandra Database images and doing DevOps/DBA with Cassandra clusters, the topics we cover apply to AWS DevOps in general - even if you don’t use Cassandra at all.

Continue reading

Cassandra Cluster Tutorial: Setting up Ansible for our Cassandra Database Cluster to do DevOps tasks

Cassandra Tutorial: Setting up Ansible for our Cassandra Database Cluster for DevOps/DBA tasks

Ansible is a key DevOps/DBA tool for managing backups and rolling upgrades to the Cassandra cluster in AWS/EC2. Ansible uses ssh, so you do not have to install an agent to use it.

This article series focuses on DevOps/DBA tasks with the Cassandra Database. The use of Ansible for DevOps/DBA goes beyond the Cassandra Database. This article helps any DevOps/DBA or Developer that needs to manage groups of instances, boxes, or hosts. These can be on-prem bare-metal, dev boxes, or in the Cloud. You don’t need to be setting up Cassandra to benefit from this article.

Continue reading

Spark Tutorial: Introduction to BigData Analytics with Apache Spark Part 1

Introduction to BigData Analytics with Apache Spark Part 1

By Fadi Maalouli and R.H.

Spark Overview

Apache Spark, an open source cluster computing system, is growing fast. Apache Spark has a growing ecosystem of libraries and framework to enable advanced data analytics. Apache Spark’s rapid success is due to its power and and ease-of-use. It is more productive and has faster runtime than the typical MapReduce BigData based analytics. Apache Spark provides in-memory, distributed computing. It has APIs in Java, Scala, Python, and R. The Spark Ecosystem is shown below.

Continue reading

Spark Tutorial: Spark SQL from Java and Python with Cassandra

Analytics with Apache Spark Tutorial Part 2 : Spark SQL

Using Spark SQL from Python and Java

Combining Cassandra and Spark

By Fadi Maalouli and R.H.

Spark, a very powerful tool for real-time analytics, is very popular. In the first part of this series on Spark we introduced Spark. We covered Spark’s history, and explained RDDs (which are used to partition data in the Spark cluster). We also covered the Apache Spark Ecosystem.

Continue reading

Spark Tutorial: Spark Streaming with Kafka and MLib

In this part of Spark’s tutorial (part 3), we will introduce two important components of Spark’s Ecosystem: Spark Streaming and MLlib.

Display - Edit

Spark Streaming

By Fadi Maalouli and R.H.

Spark Streaming is a real-time processing tool, that has a high level API, is fault tolerant, and is easy to integrate with SQL DataFrames and GraphX.

On a high level Spark Streaming works by running receivers that receive data from for example S3, Cassandra, Kafka etc… and it divides these data into blocks, then pushes these blocks into Spark, then Spark will work with these blocks of data as RDDs, from here you get your results. The following diagram will demonstrate the process:

Continue reading