~learn.

A brand-new S3 test bucket ran up a $1,300 bill in a day.

A developer made a storage bucket for a quick test. Strangers' machines sent it nearly 100 million requests, and he got the bill.

Amazon Web Services · 2024 · 2 min read · a true story, sources below

Maciej Pocwierz, a software engineer at a company in Warsaw, created a single Amazon S3 storage bucket in the eu-west-1 region for a proof-of-concept project, and uploaded a few test files. Two days later he checked his bill, expecting to still be inside the free tier.

Instead he found charges of about $1,300, from nearly 100 million requests to write files (PUT requests) in a single day.

The cause was a coincidence in naming. Anyone who knows an S3 bucket's name can send it requests. A popular open-source tool, which he chose not to name, stored backups in S3 by default, and its placeholder bucket name happened to be exactly the name he had picked. Copies of that tool whose owners had never configured the backup setting were sending their backups to his bucket. These were unauthorized requests, and at the time S3 charged the bucket owner for unauthorized requests too. At $0.005 per 1,000 PUT requests, that adds up fast when machines send thousands per second.

He also showed the other side of the danger. When he briefly opened the bucket to public writes, it collected more than 10 GB of other people's data in under 30 seconds.

His write-up spread quickly. AWS's chief evangelist Jeff Barr replied: "We agree that customers should not have to pay for unauthorized requests that they did not initiate." On 13 May 2024, AWS announced that bucket owners would no longer be charged for requests refused with an "Access Denied" (HTTP 403) error when they come from outside their own AWS account or organization.

The numbers
  • 1 new test bucket
  • Nearly 100 million unwanted PUT requests in a day
  • About $1,300 in charges
  • More than 10 GB of strangers' data in under 30 seconds once opened to public writes

Why it matters: In a pay-per-request system, someone else's traffic can become your bill. Think about who can reach your endpoints, avoid guessable names, and set billing alerts so a surprise shows up in hours, not days.

Sources