The FAA paused every US departure after files were deleted while fixing a database sync.

A NOTAM (Notice to Air Missions) warns pilots about things like closed runways, flocks of birds or volcanic ash. Late on 10 January 2023, the FAA's NOTAM system became unreliable.

Technical experts tried, among other things, switching to a backup database. There were three: one in Oklahoma City and two in Atlantic City. Through the night, a telephone hotline gave users updates. By early morning the system seemed restored, but formatting problems persisted, so the FAA's air traffic leadership ordered the databases rebuilt.

With the morning rush coming, acting FAA Administrator Billy Nolen ordered a ground stop at about 7:15 a.m. EST, pausing every departure in the United States. He later told senators he lifted it at 9:07 a.m., after resiliency testing. Senate Commerce chair Maria Cantwell put the toll at more than 1,300 cancelled flights and more than 9,500 delayed.

The FAA first traced the outage to "a damaged database file". On 19 January it said: "contract personnel unintentionally deleted files while working to correct synchronization between the live primary database and a backup database." It found no evidence of a cyber-attack.

Why didn't the backups help? At a Senate hearing on 15 February, Nolen explained that as updates are made, such as deleting outdated NOTAMs, the system synchronises them across the primary and the backups. Cantwell summed it up: the backup "was affected by the same deletion".

The FAA's fixes were telling. It added a one-hour synchronisation delay between the primary and the backup database, "to ensure that bad data from a database cannot affect a backup database", and a rule that more than one person must be present when work is done on the database.

Why it matters: A replica that copies every change instantly also copies every mistake. A backup you can recover from needs a delay, a snapshot or independence from the live system.