All or nothing: a flight, a car, and no hotel

transcript

One click books your flight, hotel and car. All or nothing. This customer got a flight, a car, and no hotel.

A flight from New York to Tokyo. Three nights at a hotel. A car. One button. Because nobody wants a hotel in Tokyo and no flight there.

But flight, hotel and car are three services. On three continents. Between them, a tenth of a percent of packets get lost. And at peak, two or three crashes an hour. Failure here is the weather.

WonderTravel used two-phase commit. First, every service locks its part and votes prepared. All voted yes. So the coordinator sends commit to everyone. Flight commits. Car commits. But a network partition cuts off the hotel. Its commit is lost. The hotel never hears the decision. In doubt, it keeps the room locked.

The customer sees a flight. Booked, and charged. A car. Booked, and charged. And a hotel that's locked, but not confirmed. The exact partial booking the design existed to prevent.

But here's the catch. In theory, the coordinator resends commit when the network heals. This one had no durable decision log. And no retry. So it never did.

So WonderTravel moved to a saga. Commit locally. Undo in reverse on failure. Because recovery is as important as the happy path.

Full breakdown in the lecture.