Grafana Tempo Makes Trace Data Safer and Smarter

Grafana Tempo 3.1 introduces powerful query-based data redaction, giving teams precise control over sensitive information in their observability pipelines.

Grafana Labs has released Grafana Tempo 3.1, a significant update to its distributed tracing system that gives teams more control over data privacy and analysis. According to reporting from InfoQ, the new version introduces two major features: the ability to redact, or remove, sensitive information from traces using queries, and an experimental function to make metrics more accurate when dealing with sampled data. Distributed tracing is a critical practice for understanding how requests travel through modern, complex applications. By tracking a single user action across dozens of microservices, teams can pinpoint bottlenecks and errors. However, these traces can sometimes capture sensitive user data, creating a security risk. This update directly addresses that challenge while also improving the quality of insights teams can derive from their systems.

At the core of the update is a more powerful and flexible way to handle data privacy. Previously, if a team needed to remove a trace containing sensitive information, they had to identify it by its specific ID and manually request its deletion. Grafana Tempo 3.1 replaces this cumbersome process with query-based redaction. Using TraceQL, Grafana's query language for traces, operators can now write rules to automatically find and remove data. For example, a team could create a query to redact all traces associated with a specific user ID or any trace that contains a particular piece of personally identifiable information (PII). This shifts data privacy from a reactive, manual task to a proactive, automated policy. The release also introduces an experimental feature to improve metric accuracy. Many organizations sample their traces—collecting only a fraction of the total—to manage costs. The new `@sampled` query hint adjusts calculations to account for this, providing a more statistically accurate view of system performance, like average latency, from an incomplete dataset.

These new features reflect two major trends in the software observability landscape: the growing importance of data privacy and the constant trade-off between cost and data fidelity. As regulations like GDPR and CCPA become more stringent, companies face increasing pressure to manage sensitive data across all their systems, including internal monitoring tools. A stray email address or credit card number in a trace log is not just a technical oversight but a significant compliance risk. Query-based redaction provides an essential tool for data governance, allowing security and compliance policies to be enforced directly within the observability pipeline. Similarly, the challenge of managing massive data volumes is universal. While collecting 100% of trace data offers perfect visibility, it is prohibitively expensive for most. The move toward making sampled data more intelligent and useful represents an industry-wide effort to deliver high-quality insights without overwhelming budgets.

For developers, security professionals, and IT operations teams, Grafana Tempo 3.1 offers immediate practical benefits. The ability to programmatically redact data reduces the manual workload and lowers the risk of a data breach through exposed PII in logs. Teams can now build a safer, more compliant observability platform by default. The sampling-aware metrics, though still experimental, promise to deliver more trustworthy performance data, leading to better-informed decisions about system optimization and resource allocation. As this is the first minor release built on the new 3.0 architecture, users can expect Grafana Labs to continue refining these capabilities. Teams should monitor the sampling-aware metrics feature as it moves toward general availability, as it has the potential to fundamentally improve the value they get from their existing tracing instrumentation without increasing data storage costs.

This update gives engineers powerful new tools for data governance within their observability stack. Query-based redaction offers a scalable way to enforce privacy policies, while sampling-aware metrics provide more accurate performance insights from incomplete data, addressing a core trade-off in modern distributed tracing.

The new redaction feature helps companies reduce the risk of exposing sensitive customer data, simplifying compliance with regulations like GDPR. More accurate metrics from sampled data allow businesses to make better operational decisions without incurring the high cost of full-fidelity data collection.