How OpenReplay Ditched Postgres for Massive Scale

OpenReplay replaced Postgres with ClickHouse to ingest billions of events and up to 6 TB of data per day on one server.

Front-end observability platform OpenReplay has successfully migrated its core data infrastructure from PostgreSQL to ClickHouse to handle its rapidly growing data volume. According to a case study published by the ClickHouse team, the company needed a solution that could ingest billions of user interaction events, amounting to as much as six terabytes of new data each day. The previous system, built on Postgres, struggled to keep up with this scale. The migration allowed OpenReplay to manage this immense data firehose on a single, powerful server instance. This new architecture not only handles the high-volume ingestion in real time but also enables analytical queries to return results in under a second, a critical requirement for an observability tool that developers rely on for instant insights. The move highlights a common challenge for fast-growing tech companies: when a general-purpose database is no longer enough.

The technical leap from Postgres to ClickHouse represents a fundamental shift in database philosophy, moving from a row-oriented to a columnar-oriented architecture. PostgreSQL is a transactional database (OLTP) optimized for reading and writing individual rows of data quickly, making it perfect for application backends. However, analytical queries, like those needed for observability, typically scan and aggregate values across millions or billions of rows in just a few columns. A row-based system is inefficient for this task, as it must read entire rows from disk even if only a fraction of the data is needed. ClickHouse, an analytical database (OLAP), stores data by column instead of by row. When a query asks for the average load time across all user sessions, ClickHouse can read only the "load time" column, dramatically reducing the amount of data processed and enabling sub-second query speeds. This efficiency is why OpenReplay can run its entire workload on one vertically-scaled machine, simplifying its architecture and data retention policies.

OpenReplay's story is a prime example of a wider industry trend: the move away from one-size-fits-all databases toward specialized tools for specific workloads. For years, developers relied on relational databases like PostgreSQL and MySQL as the default choice for nearly every application. While these systems are incredibly versatile and reliable, the explosion of data from sources like IoT devices, user analytics, and application logs has exposed their limitations in the analytics domain. This has fueled the rise of specialized databases designed for particular tasks. ClickHouse, along with competitors like Apache Druid and Pinot, excels at real-time analytics on massive datasets. Similarly, time-series databases like TimescaleDB and InfluxDB are built for timestamped data, while vector databases like Pinecone are tailored for AI applications. This "polyglot persistence" approach, where companies use multiple database types, is becoming the new standard for building scalable, high-performance systems.

The primary takeaway for CTOs, architects, and developers is the critical importance of selecting the right database for the workload. While starting with a familiar tool like Postgres is often the right pragmatic choice, teams must be prepared to re-evaluate their architecture as they scale. This case study serves as a powerful reminder that pushing a general-purpose database beyond its intended design often leads to complex, expensive, and brittle systems. For OpenReplay, the migration wasn't just about performance; it was about operational simplicity and cost-effectiveness. Running on a single server with a simple Time-To-Live (TTL) setting for data retention is far more manageable than maintaining a complex, sharded Postgres cluster. As data volumes continue to grow across the industry, we can expect to see more engineering teams making similar strategic migrations to specialized analytical databases to stay competitive and deliver the real-time insights their users demand.

This case study demonstrates the practical limits of traditional relational databases like Postgres for high-ingestion observability workloads. It provides a real-world blueprint for architects on how a columnar database like ClickHouse can solve scaling challenges, offering sub-second analytics on terabytes of daily data on a single machine.

By migrating to ClickHouse, OpenReplay achieved massive performance gains on a single server, likely reducing infrastructure costs and operational complexity. This enables them to offer a powerful, real-time product without the expense of a large, sharded database cluster, improving their competitive position and margins.