A messaging app lives on open connections: lines held open between devices and servers. So how many can one machine hold?
A few months earlier, WhatsApp had blogged that its servers were doing 1 million TCP connections on a single box. On 6 January 2012 it posted an update, "1 million is so 2011": it could now "easily push our systems to over 2 million tcp connections". Its proof: one server reporting 2,277,845 open sockets. And, the post said, "with plenty of CPU and memory to spare": the CPU was 41.9% idle and 27 GB of memory was free.
The server had 24 CPUs (Intel Xeon X5675) and about 100 GB of memory, running FreeBSD 8.2. The post listed its operating-system settings, such as a limit of 2.4 million sockets and 3 million open files. The last important piece, it said, was Erlang, a programming language. It ended with a P.S.: WhatsApp was hiring.
Two years later, on 19 February 2014, Facebook announced it would buy WhatsApp for about $16 billion. WhatsApp had over 450 million people using it each month, and was adding more than 1 million new registered users a day.
The team behind it was tiny. Jim Goetz of Sequoia Capital, WhatsApp's investor, wrote that day: "With only 32 engineers, one WhatsApp developer supports 14 million active users." He said the service processed 50 billion messages a day using Erlang, "an unusual but particularly well-suited choice".
- 2,277,845 open connections on one server, January 2012
- 41.9% of its CPU still idle
- 450 million monthly users at the 2014 acquisition
- 32 engineers, one for every 14 million active users, per Sequoia
Why it matters: How much one machine can do sets how many machines, and people, you need. Know your per-server limits, tune the operating system for them, and pick tools built for many connections at once.