Code Spaces hosted Subversion (SVN) and Git code repositories for its customers, on Amazon's cloud. On Tuesday 17 June 2014, a "well orchestrated" DDoS, a flood of junk traffic, hit its servers. That happened quite often, the company wrote. This time it was just the start.
An unauthorised person had got into Code Spaces' Amazon EC2 control panel, the web console that manages everything in the account, and left messages asking the company to contact a Hotmail address. That contact turned into an attempt to extort a large fee to make the DDoS stop.
Checking what the intruder could reach, the company found no access to its machines so far, because the intruder didn't have its private keys. But the control panel was enough. When the team changed passwords to take the account back, the intruder had already created a number of backup logins, and on seeing the recovery attempt started randomly deleting things. By the time Code Spaces regained control, the intruder had removed all EBS snapshots (disk backups), S3 buckets, all AMIs (machine images), some EBS volumes and several machine instances.
"In summary, most of our data, backups, machine configurations and offsite backups were either partially or completely deleted," the statement said. It all took place over a 12 hour period. Some SVN and all Git repositories could still be exported, but every backup and snapshot was gone, along with all the EBS volumes holding database files.
The same statement ended the company. The cost of the incident, and of refunding customers left without the service they paid for, put Code Spaces in "a irreversible position both financially and in terms of on going credibility". It ceased trading.
- 17 June 2014: the DDoS, and then the takeover
- 12 hours: how long it all took, by the company's account
- 0 private keys: when the company checked, the intruder had no access to its machines
- All EBS snapshots, S3 buckets and machine images deleted
Why it matters: A backup that the same login can delete is not a separate copy. Keep backups in a different account with different credentials, and guard the control panel: it can destroy everything.