Before Pokémon GO launched, Niantic and Google did what careful teams do: they guessed. They picked a target for player traffic, called it 1x, and planned for a worst case of about 5x.
The game opened first in Australia and New Zealand. According to Google's account, within 15 minutes traffic was far past what Niantic expected. It quickly surged to 50 times the original target, ten times the worst case. The US launch was due the next day.
The game ran on Google's cloud. Its servers were containers managed by Google Container Engine (a hosted version of Kubernetes), and its main database was Cloud Datastore. The launch was also the first job for a new Google team, Customer Reliability Engineering, whose engineers worked inside Niantic's team. Google's post admits "not everything was smooth sailing at launch": when the game became unstable, the two teams fixed one problem after another.
The boldest fix came before the launch in Japan. To fit more than a thousand extra machines into the cluster, they upgraded it to a newer version of Container Engine while millions of new players were joining. Google compared it to swapping a plane's engine in flight. They also replaced the network load balancer, the system that spreads incoming connections across servers, with a newer one built for HTTPS traffic.
It paid off. When Pokémon GO opened in Japan two weeks after the US, new sign-ups were three times the US launch, and Google reports it launched without incident.
- Planned: 1x target, about 5x worst case
- Actual: 50x the target, 10x the worst case
- Traffic passed expectations within 15 minutes of the first launch
- A live cluster upgrade to make room for 1,000+ more machines
- Japan sign-ups: 3x the US launch
Why it matters: An estimate is a guess, and a hit product can beat your worst case by ten times. What saved Pokémon GO was a platform that could keep growing and a team that could change it while it was live.