20 pods each report a p99. What is the fleet's p99?

transcript

You run 20 API pods. Each one reports its own p99 latency. So what's the p99 for the whole fleet? From those, you can't say.

A summary pre-calculates quantiles. On each instance. A histogram just counts observations into buckets. Say, 10 milliseconds up to 10 seconds.

Here's the catch. A summary's per-pod p99s can't be aggregated across instances. And total time over request count? That's only the mean. Not a percentile.

Histograms fix it. Each pod exposes its bucket counts. Counts add up. Sum the same bucket across every pod. Then compute p99 from the merged buckets, with histogram quantile. One real fleet number.

Your latency SLO is about the whole service, not one pod. So expose histograms, and compute the percentile after the merge. Use them for latencies, sizes and durations. Anything you'll want a percentile of.

Summaries are less flexible. You can't combine them. Prefer histograms. Counts merge. Percentiles don't.

Full breakdown in the lecture.