16 bytes a point, squeezed to 1.5. Nothing lost.

transcript

This trick makes monitoring data ten times smaller. And it doesn't throw away a single number.

50 million series. A point every 10 seconds. That's 5 million a second. Each point is a timestamp and a float. 16 bytes. Keep one week of that? About 48 terabytes.

A normal database makes each point a row. Plus index. Plus a B-tree insert. But metrics have three habits. They arrive in time order. They're never updated. And each value looks a lot like the one before.

Timestamps first. A point every 10 seconds. So every gap is 10. So store how the gap changed. Usually it didn't. Zero. And a zero takes one bit. Now the values. XOR each one with the value before it. A value that barely moved is mostly zeros. Keep only the changed bits.

Raw, a point is 16 bytes. With these two tricks? About 1 point 5. Facebook's Gorilla paper, in 2015, reported even less.

That 48 terabyte week? About 4 point 5. That's what makes a year of history affordable. Store series, not rows. Then store only what changed.

Full breakdown in the lecture.