Clicked at 10:00. Arrived at 10:04. Which minute counts it?

transcript

A phone in a tunnel taps an ad. 10:00 and 30 seconds. The click reaches us at 10:04. Which minute gets it?

Count by arrival time? It lands in 10:04. Wrong minute. And any lag shifts clicks into the wrong minute. Lag is worst at traffic peaks. So your busiest minutes are least accurate. So count by event time. The minute the click happened.

But here's the catch. When is a minute finished? A tunnel click could still be on its way.

Flink keeps a watermark, with 30 seconds of tolerance. Once it passes the end of 10:00, that count is published. Our click arrives at 10:04. Allowed lateness. The 10:00 row updates. Much later than that? It goes to a side output. And the hourly batch counts it from the raw archive anyway.

So the dashboard shows 412 at 10:00. Fresh within a minute. Up to 10:52, minutes are final. Reconciled. They won't change. Later minutes can still tick up. And invoices only use final rows.

Count by when it happened. Not when it arrived. Publish fast. Finalise later. Bill only what's final.

Full breakdown in the lecture.