Your search took 30 seconds. A search engine takes 50 ms.

transcript

A shopper types: red running shoes, nike, size 10. Your store takes 30 seconds to answer. Then the database falls over.

The quick answer is SQL. Name like red, like running, like shoes. On 50 million products. And type nikee by mistake? Zero results. Same query, on a search engine. 50 milliseconds. Ranked, best match first. It even finds crimson, not red. And 234 more behind it.

But here's the catch. Fast isn't enough. Two users type the same word: apple. One wants fresh apples, and apple pie. The other wants an iPhone. Same query, completely different intent. Who ranks first?

50 million products, 20 fields each. A billion field values to index. 10,000 queries a second, about 1,000 in flight at once. Each one hits every shard. And an index over 500 gigabytes. Kept in sync with the database, always.

Five days. How engines think. Keeping the index fresh. Ranking. Autocomplete. Staying up. Search is deceptively complex. It's not just put the data in Elasticsearch.

Full breakdown in the lecture.