The nightly rebuild that could let a phishing link through

transcript

A new phishing link arrives at exactly the wrong moment. It would be in neither filter. It would walk straight through.

GlobalForum checks every link against 1 point 2 billion known bad URLs. All squeezed into a Bloom filter of about 2 point 15 gigabytes. One rule above all. When the filter says no, it must be right. Zero false negatives.

But the list never stops growing. Feeds add millions of threats a day. New ones go straight into a small buffer. Its own Bloom filter, sized for 10 million. Every check asks the big filter first. Not found? Then the buffer. Every 24 hours, a fresh big filter is built from the database. Then swapped in.

But here's the trap. The rebuild starts with a database snapshot. Then a new phishing link arrives. Too late for the snapshot. The new filter swaps in. The buffer gets cleared. That link is in neither. The filter says no. A false negative.

The fix is all about order. Start a fresh buffer before the snapshot. Keep checking the previous buffer too, until the swap. After the swap, the old buffer's threats are in the snapshot. Now drop it.

Nothing falls in the gap. Every threat is always in some filter that's checked. Accept a few false positives. Never a false negative.

Full breakdown in the lecture.