Your observability bill is a line item nobody scrutinises until it’s shocking. It grew quietly, quarter over quarter, faster than your traffic — and no one ever made a decision that it should. That’s not a pricing problem. It’s an ownership problem, and it’s fixable without going blind.
The instinct when the bill lands is to panic-cut: turn off logs, drop retention, sample aggressively. That trades a cost problem for a reliability problem — you save money right up until the incident where the data you’d have needed was the data you deleted. The goal isn’t cheaper observability. It’s observability that costs what it should, because someone decided what it should cost.
Why Observability Bills Spiral
Telemetry spend grows through a set of mechanisms that are individually reasonable and collectively ruinous.
- Data volume outpaces traffic. You don’t just log more because you serve more requests. You log more per request — every new service, every debug line someone added during an incident and never removed, every “let’s just capture everything to be safe.” Volume compounds independently of how many customers you have.
- High-cardinality metrics explode silently. Someone adds a metric tagged with user ID, or request ID, or full URL path. Each unique value is a new time series, and the count multiplies out to millions. Cardinality is the single most common cause of a monitoring bill that quintuples overnight, and it happens from one innocent-looking label.
- Retention defaults are set once and never revisited. Everything is kept for the same long window because that was the default, or because someone was cautious once. You’re paying to store months of debug logs you will never query, at the same tier as the data you actually need.
- Vendor pricing punishes exactly what grows. As covered in what monitoring vendors won’t tell you about pricing, the meters — ingest volume, cardinality, host count, seats — are calibrated to grow with your success. The bill is designed to scale up faster than your usage, and default settings walk you straight into it.
None of these involve anyone deciding to spend more. That’s the point. The bill grows in the absence of a decision.
The Real Waste Is What Nobody Looks At
Before cutting anything, look at what you’re actually paying for. On most teams, a large share of telemetry spend buys data that is never queried, never alerted on, and never looked at. It exists because it was easy to emit and nobody ever turned it off.
The audit that finds this is unglamorous but high-yield: which log streams have zero queries against them? Which metrics back no dashboard and no alert? Which high-cardinality dimensions is anyone actually slicing by? The answers usually reveal that a meaningful fraction of the bill is pure dead weight — data that provides no signal and generates no judgment, just cost.
Cutting dead weight is the free lunch of observability cost control. It reduces spend with zero reliability downside, because by definition nobody was using it. Most teams never do this audit, because it’s nobody’s job and it’s tedious, so they pay for the dead weight indefinitely.
Cut Cost Without Going Blind
Once the dead weight is gone, the remaining reductions require judgment — trading some data for savings without losing the signal that matters. The techniques are well understood; the skill is in applying them selectively.
Tier your retention. Not all data deserves the same lifespan. Keep high-resolution data for the short window where incidents actually get debugged, and roll older data down to cheaper aggregates or cold storage. You rarely need per-request logs from three months ago at the same fidelity as yesterday’s.
Sample the high-volume, low-value streams. Traces and verbose logs from healthy, successful requests are mostly redundant — you don’t need all ten thousand of them, you need a representative sample plus every error. Intelligent sampling keeps the signal and drops the repetition. The trap is uniform sampling that throws away errors along with successes; the errors are the whole point.
Kill the cardinality bombs. Find the labels driving your time-series count — usually a handful of high-cardinality dimensions — and remove or bucket them. This is often the single biggest lever, and it’s frequently a one-label fix that cuts spend dramatically with no loss of anything anyone was using.
Match resolution to how you actually query. Metrics collected every second but only ever viewed at minute granularity are paying for detail nobody consumes. Align collection resolution to real query patterns.
The discipline throughout: cut what isn’t watched, keep what drives detection, diagnosis, and decisions. Every cut should be tested against a simple question — if there were an incident tomorrow, would I regret not having this? If the honest answer is no, cut it.
The Reason This Doesn’t Get Done
Observability cost control isn’t hard because the techniques are hard. It’s hard because it’s continuous work that nobody owns. The audit isn’t a one-time cleanup — new services keep emitting, someone keeps adding a debug log, cardinality creeps back in. Spend that’s cut today drifts back up within two quarters unless something keeps watching it.
That’s the same structural gap that shows up across reliability: the fix is a function, not a project, and functions without owners rot. On a small team, no one is measured on the observability bill until it’s a crisis, so the routine pruning that would prevent the crisis never happens. The bill grows in the space where accountability should be.
And there’s a real tension here that makes it worse: the person best placed to cut telemetry safely is the person who understands what each stream is for. Cut by someone who doesn’t, and you delete the data your next incident needed. So the cutting requires both the discipline to do it regularly and the operational context to do it safely — a combination that’s rare on a lean team.
Own the Bill, Not Just the Data
Observability should cost what it’s worth, and knowing what it’s worth requires someone who understands both the bill and the reliability the data buys. Cut blindly and you go dark at the worst possible moment. Don’t cut at all and the bill scales with your vendor’s incentives, not yours. The answer is neither panic nor neglect — it’s ongoing, informed ownership of the tradeoff.
Vigil by IOanyT owns the observability cost-versus-signal tradeoff as a managed function — we audit out the dead weight, tier retention, kill cardinality bombs, and keep spend in line without ever cutting the data an incident would need.
Your engineers build. We keep observability sharp and the bill honest.