Yelp Moved Its Databases Without Copying Data
TL;DR: Yelp migrated its large Cassandra database to a new system by physically re-pointing its data storage volumes instead of slowly streaming data. This novel approach dramatically cut migration time and risk for the company's critical infrastructure.
Key facts
- Category
- Infrastructure
- Impact
- High
- Published
- Source
- InfoQ
Full summary
Yelp migrated its Cassandra database by simply moving its existing data volumes, avoiding a slow and risky data transfer process.
Yelp has successfully migrated its large-scale Cassandra database from a self-managed Kubernetes environment to Amazon's managed Elastic Kubernetes Service (EKS). According to a report from InfoQ, the engineering team accomplished this complex task using a novel and highly efficient method. Instead of following the traditional path of creating a new database cluster and slowly streaming terabytes of data over the network, Yelp’s team physically moved the underlying data storage. They detached the existing Amazon EBS volumes—the virtual hard drives containing all the database information—from the old system and reattached them to the new one. This approach effectively sidestepped the most time-consuming and risky part of a typical database migration, allowing for a much faster and more reliable transition for one of the company's most critical stateful workloads.
The technical brilliance of this strategy lies in treating data storage and computing power as separate, interchangeable components. A standard Cassandra migration involves building a new, empty cluster on the target platform (EKS) and adding it to the existing cluster. The database then begins a long, resource-intensive process of replicating data to the new nodes until they are in sync, after which the old nodes can be decommissioned. This can take days or even weeks, consumes significant network bandwidth, and introduces risk of data inconsistency. Yelp’s method, by contrast, is more like unplugging an external hard drive from one computer and plugging it into another. The team orchestrated a careful shutdown of the Cassandra pods in the old cluster, which released their claim on the persistent EBS volumes. Then, they launched new pods in the EKS cluster, configuring them to claim and mount those exact same volumes, making the data instantly available without any transfer.
This case study highlights a significant trend in modern cloud infrastructure: the growing maturity of container orchestration platforms for managing stateful applications. For years, databases and other systems that need to retain data were considered a poor fit for the ephemeral, stateless nature of containers. Early versions of Kubernetes struggled with persistent storage, making migrations and disaster recovery complex. However, Yelp's success demonstrates how far the ecosystem has come. By leveraging robust cloud provider primitives like AWS's EBS alongside Kubernetes' own storage management capabilities, engineers can now perform sophisticated operations that were once the domain of traditional IT. This decoupling of storage from compute provides immense flexibility, allowing teams to upgrade, replace, or move their application infrastructure without touching the underlying data, a powerful capability for any organization.
The primary takeaway for CTOs, developers, and DevOps teams is to think beyond application-level solutions for infrastructure-level problems. Faced with a massive data migration, the default approach is often to use the database's built-in replication tools. Yelp’s team challenged that assumption and found a more elegant solution at the infrastructure layer, saving countless hours of engineering effort and dramatically reducing the risk of downtime or data loss. This strategy serves as a powerful blueprint for other organizations planning to modernize their Kubernetes platforms or migrate stateful workloads between environments. As cloud-native technologies continue to evolve, looking for creative ways to leverage the underlying platform's capabilities can unlock significant efficiencies and simplify what have historically been some of the most daunting challenges in infrastructure management.
Why it matters
Migrating stateful applications like databases on Kubernetes is a major headache for engineering teams. Yelp's method of re-attaching existing storage volumes offers a practical, faster, and less risky blueprint for any organization running databases on cloud infrastructure, significantly lowering the barrier to platform modernization.
Business impact
This migration strategy drastically reduces downtime and engineering hours typically spent on complex data transfers. For businesses, this means lower operational costs, faster adoption of modern cloud services, and minimized risk of data loss or service disruption during critical infrastructure upgrades.
Tags
Related on Notifire
Related stories
Primary source: InfoQ
