Redis serves thousands of connections from one thread. How?

transcript

Redis serves thousands of connections. From a single thread. Give each connection its own thread instead? You stall around 10,000. Why?

The simple way. Every connection gets its own thread. The thread calls read, and sleeps until data arrives. It can't do anything else. So 10,000 connections means 10,000 threads. Mostly asleep. All costing memory and switches.

The trick: one thread watches every socket. The old way, select, tops out at 1,024. And it scans all of them, every call. 10,000 sockets, one ready, 10,000 scanned. epoll fixes it. Register each socket once. Then it hands back only the ready ones. Just the one.

Put it in a loop. Wait on epoll for every client at once. A socket is ready? Run its handler. Then straight back to waiting. Node, nginx and Redis all work this way.

But here's the catch. One handler runs 2 seconds of heavy maths. The loop only switches when a handler yields. CPU work never does. Everyone waits. So push CPU work to a worker pool. Keep the loop for I/O.

Now scale it. A million threads, about a megabyte of stack each. A terabyte, just for stacks. Instead, one event loop per core. 32 cores, about 30,000 connections each. And under a kilobyte of state per connection.

Lots of connections, mostly idle? Don't park a thread on each one. Let one loop wait on all of them.

Full breakdown in the lecture.