~learn.
?
SEV-1 · Nidhi Pension Trust · 06:40

Both regions deleted at once.

Fictional company and people. Modelled on a real public postmortem, named at the end.

you're on call tonight

Nidhi Pension Trust runs a retirement fund's member systems for 6 lakh members, on a cloud provider's private cloud in two regions. It's 06:40. The channel fills up as the night goes on; at 5 points you make the call. A wrong move costs minutes and users, and someone tells you why.

in the channel
  • YouOn-call infrastructure engineer
  • Ramya SubramanianHead of technology operations, incident commander
  • Karthik RameshPrincipal cloud architect
  • Anjali DeshpandeEngineer, backup and recovery
  • Farida ContractorTechnical account manager at the cloud provider
  • Rohit MalhotraMember communications lead
  • Suresh IyerCEO
~13 min · 5 calls · +60 XP