Knight Capital lost $460 million in 45 minutes because one server missed a deploy.

Knight Capital was a US broker whose trading made up about ten percent of all trading in listed US stocks. Its system SMARS took big "parent" orders and split them into smaller "child" orders sent to the market.

For a New York Stock Exchange program starting on 1 August 2012, Knight wrote new SMARS code. It replaced old code for a feature called "Power Peg", unused since 2003, and it reused the flag that used to switch Power Peg on. Power Peg had a weak spot: in 2005, the part that counted filled shares, and told it to stop, was moved elsewhere. Nobody retested Power Peg afterwards.

From 27 July, Knight put the new code on SMARS's servers in stages. A technician missed one of the eight servers. No second person checked the deploy, and no written procedure required it.

On 1 August, the seven updated servers worked. Orders with the reused flag that reached the eighth woke up Power Peg, which sent child orders again and again without counting the fills. From 8:01 a.m., before the 9:30 open, an internal system had sent 97 emails mentioning "Power Peg disabled", but they weren't designed as alerts and weren't acted on. While staff hunted for the cause, they removed the new code from the seven good servers. That made it worse: Power Peg now ran there too. Knight had no procedures telling staff when to disconnect a misbehaving system from the market.

In about 45 minutes, 212 customer orders became over 4 million executions, and Knight lost over $460 million on the positions it was left holding. The SEC fined Knight $12 million; Knight settled without admitting or denying the findings.

Why it matters: Dead code on a production server is still live code. Delete it, never reuse an old switch for a new job, check every deploy reached every machine, and decide in advance when to pull the plug.