~learn.

One flight plan with two places called DVL stopped UK air traffic control's flight data.

A route listed Devil's Lake, North Dakota. The system read it as Deauville, France, and both copies shut down in about 20 seconds.

NATS (UK air traffic control) · 2023 · 2 min read · a true story, sources below

On 28 August 2023 at 08:32, NATS's flight plan processing system (FPRSA-R) received the plan for a flight from Los Angeles to Paris Orly. Its job is to cut out the UK part of each route and pass it to controllers' screens.

Eurocontrol had added extra waypoints, as usual. FPRSA-R found the UK entry point and searched for the exit. Its first two candidates weren't in the original plan, so it skipped them. The third, DVL, was. But in the original plan DVL meant Devil's Lake, North Dakota, not Deauville, France. Same three-letter code; the report says over 3,000 waypoints worldwide share an abbreviation.

Now the flight appeared to leave UK airspace before entering it. The system judged that not credible, raised a critical exception and, as designed, put itself into maintenance mode so no bad data reached controllers. The backup took over, received the same plan, hit the same error and did the same. About 20 seconds after the plan arrived, all automatic processing had stopped. By hand, staff could process about 60 flight plans an hour, against 700 to 800 in a typical busy hour.

It got worse. The plan stayed at the front of the message queue, because it was only removed once processing finished. Each time engineers reconnected the system, the plan was sent again and knocked it straight back into maintenance mode. Nothing could drop a message after repeated failures. With the manufacturer, Frequentis Comsoft, the team finally moved it into a paused queue, four hours in.

Flight restrictions ran from 11:00 and were all lifted by 18:03. The CAA estimated that over 700,000 passengers were affected.

The numbers
  • About 20 seconds to take down both the primary and the backup
  • About 4 hours stuck in a loop on one message
  • 700–800 flight plans an hour automatically, about 60 by hand
  • Over 700,000 passengers affected
  • About £65m estimated cost to airlines

Why it matters: Set one bad message aside; don't let it stop everything. Give queues a dead-letter path after repeated failures. A backup running the same code rejects the same input.

Sources

the idea behind it · W03 · DAY 04Dead Letters and Poison Pillsread the lesson →