Abstract digital art of efficient data pipelines, symbolizing cost optimization and performance gains in cloud infrastructure.

DatabaseHigh

LivePerson Slashes GCP Data Costs

TL;DR: LivePerson significantly cut its Logstash processing costs on Google Cloud by over 50%. The company achieved this by systematically benchmarking GCP machine types, ultimately switching to AMD Milan-based instances. They also found that Kafka compression codec selection independently boosted throughput.

By Taranpreet SinghElastic Blog1h ago1 min readupdated 1h ago

Source

Key facts

Category: Database
Impact: High
Published: 1h ago
Source: Elastic Blog

Full summary

LivePerson cut Logstash processing costs over 50% on GCP by benchmarking machine types and switching to AMD Milan-based instances.

LivePerson has successfully reduced its Logstash data processing costs on Google Cloud Platform (GCP) by more than 50%. The company achieved this significant saving through a methodical process of benchmarking different virtual machine types. Their analysis revealed that switching their Logstash workloads to instances powered by AMD's Milan processors provided the best price-to-performance ratio, which was the primary driver of the cost reduction. In a related but independent finding, the team also discovered that the choice of compression codec used with their Kafka data streams had a substantial impact on overall throughput, providing an additional avenue for optimization.

This case study offers valuable, practical insights for any organization running data-intensive workloads on public cloud infrastructure, particularly for DevOps, SRE, and engineering leadership. It demonstrates that default or legacy instance selections are not always the most cost-effective and that periodic benchmarking can unlock substantial savings. The specific findings regarding the efficiency of AMD Milan instances for Logstash and the performance impact of Kafka compression codecs are directly actionable for teams using this common data stack. The result underscores the importance of a data-driven approach to infrastructure management.

Why it matters

This case study provides a practical, data-driven roadmap for reducing cloud costs on common data infrastructure. It proves that significant savings are achievable by benchmarking core components like compute instances and software configurations, rather than accepting default settings.

Business impact

The findings directly translate to lower operational expenditures (OpEx) for companies running similar data pipelines on GCP. A 50%+ cost reduction on a major data processing workload can free up significant budget for other strategic initiatives and improve overall financial efficiency.