Dropbox always ran part of its system itself: the metadata (which files you have, in which folders) and its web servers lived in data centers it managed. But the files themselves, the actual bytes, were stored on Amazon S3. By 2016 Dropbox had more than 500 million sign-ups and 500 petabytes of user data, up from about 40 petabytes in 2012.
In the summer of 2013, building Dropbox's own storage system became a major project, run by a small team. They called it Magic Pocket.
They moved carefully. In August 2014 came the "dark launch": data was mirrored between two regional locations and the system was ready to store user data, yet Dropbox kept extra backups for another six months, just in case. On 27 February 2015, for the first time in its history, Dropbox began storing and serving user files exclusively in-house. Next it set itself an aggressive goal: grow the system to more than 500 petabytes in six months.
Then came the sprint, which the team named BASE Jump because the schedule left so little time to "open the parachute". From 30 April 2015 they installed servers in three regional locations fast enough to keep up with the data pouring in, over a network that moved data at more than half a terabit per second at peak. Their goal was 90% of data in-house by 30 October 2015. They reached it on 7 October, with no major service disruptions and no lost data.
Dropbox's 2018 IPO filing reports that this "Infrastructure Optimization" cut its infrastructure costs by $39.5 million in 2016 and a further $35.1 million in 2017, each compared with the year before.
- 500 petabytes of user data by 2016
- 90% of it moved in-house, reached 7 October 2015
- Peak transfer over half a terabit per second
- Infrastructure costs down $39.5 million (2016) and $35.1 million (2017), per the S-1
Why it matters: Migrate in stages: build, run it dark, keep backups after you switch, then scale. At large, steady scale, owning hardware can cost far less than renting it.