~/himanshu
← Case filescheckpoint 1 / 14 · ~19 min left
CASE-013SYSTEM DESIGN19 MIN

25.3 Million People, One Cricket Match: Why Hotstar Scaled Before the Star Walked In

Himanshu Kukrejareads
Blueprint cover for the case file "25.3 Million People, One Cricket Match: Why Hotstar Scaled Before the Star Walked In"
FIG. 1 · Blueprint · CASE-013

Lesson · 16 min

Understanding the Problem: A Million New Viewers Every Minute#

Rain stopped play in Manchester on 9 July 2019, so India and New Zealand came back on the reserve day, 10 July, to finish their World Cup semi-final.

You're watching on your phone, on a bus or at a chai stall. India need 240 to win. They are 5 for 3 almost before you've settled in. Rohit Sharma, KL Rahul and Virat Kohli are all out for 1. You almost stop watching.

Then Ravindra Jadeja and MS Dhoni start fighting back. Your phone buzzes with a notification. So do millions of other phones. People open the app, and the number watching the same live stream at the same moment starts to climb nearly with every ball: from 18.2 million in the 44th over to 24.4 million by the 47th.

In the 49th over it reaches 25.3 million, at the time the most people who had ever watched one live stream at once, anywhere in the world. In that same over, Martin Guptill throws down the stumps from the deep and Dhoni is run out by a few centimetres. India lose by 18 runs.

The interesting part isn't the record. It's that the servers for those 25.3 million people had to be running before they showed up. Hotstar's engineers have said, in their blog and talks, that the usual cloud trick of "add servers when things get busy" was far too slow for cricket. This case file is about why, and what they did instead.

India vs New Zealand, World Cup semi-final, 10 July 2019
  • 25.3M — people watching on Hotstar at the same moment
  • ~1M/min — new viewers arriving during the chase
  • 1M+ — requests per second
  • 10 Tbps+ — video bandwidth at the peak
  • 100M+ — different people who used Hotstar that day
  • 50M — the concurrency Hotstar had rehearsed for
ExplainerThe whole story: Hotstar, 25 million viewers, one match

During the 10 July 2019 World Cup semi-final Hotstar reached 25.3 million concurrent viewers, arriving at about a million a minute, faster than 5-to-6-minute autoscaling could react. So it pre-warmed and climbed a capacity ladder on concurrency, read the scorecard as a forecast, used the CDN as a shock absorber and switched extras off in panic mode. The run-out that sent millions to the home page was tamed with jitter, backoff and rehearsals that prepared for 50 million.

First, What Does "Concurrent" Mean?#

Two numbers often get mixed up.

  • Reach counts every different person who watched at any point in the day.
  • Concurrency counts how many are watching at the same instant. It's how many streams the system is serving right now.

Think of a railway station. Reach is everyone who passed through today. Concurrency is the crowd on the platform at 6 PM. You build the platform for the 6 PM crush.

Reach vs concurrency

Reach

  • Every different person, at any time
  • Hotstar: more than 100 million users that day
  • Spread over hours, so gentle on servers

Concurrency

  • Everyone watching right now
  • Hotstar: 25.3 million at the peak
  • Decides servers, bandwidth and database size

The Shape of a Cricket Match#

Most websites get busy in a smooth curve. Cricket traffic looks like the score. At Rootconf 2019, Hotstar cloud architect Gaurav Kamboj walked through the real concurrency graph for this match.

What the traffic did, over two days
  • Day 1 · toss —
    3 to 4 million open the app just to see the toss
  • Day 1 · New Zealand bat —
    viewers climb to 13.9 million, dipping at drinks breaks
  • Day 1 · rain —
    a drop to about 5 million, who stay on watching rain and highlights for three to four hours
  • Day 2 · India start their chase —
    from about 1.5 to 2 million to nearly 15 million in minutes, roughly a million new viewers per minute
  • Day 2 · 5 for 3 —
    the growth stalls
  • Day 2 · the fightback —
    notifications go out; about 1.1 million per minute arrive, about 13 million to 25 million in around ten minutes
  • Day 2 · 49th over —
    the peak, 25.3 million
  • Day 2 · Dhoni run out —
    the sudden drop, the most dangerous moment of all

Hotstar's blog calls a wave like this a tsunami: a surge that arrives in minutes, not hours. And the scariest moment isn't the peak. It's the drop after it (Chapter 5).

Chapter 1: Why Autoscaling Shows Up Late#

Autoscaling is the standard way to run servers in the cloud. You set a rule like "if average CPU goes above 70%, add servers". The cloud notices your servers are busy and starts new ones. It works like a thermostat, and for most websites it's perfect.

But a thermostat reacts after the room is already hot. A new server isn't instant: the cloud must find a machine, boot it, start the app and register it with the load balancer (the traffic director that spreads requests across servers). Kamboj put the whole chain at five to six minutes.

DiagramThe reactive autoscaling chain (as described in the 2019 talk)
%% title: The reactive autoscaling chain (as described in the 2019 talk)
flowchart TD
  T(["Crowd arrives<br/>~1M new viewers per minute"]):::hot --> M["Servers get busy<br/>CPU rises"]:::service
  M --> A{"CPU above the<br/>alarm line?"}
  A -->|"yes · minutes later"| R["Ask the cloud<br/>for more servers"]:::service
  R --> B["Machine found and booted"]:::service
  B --> S["App starts<br/>about a minute"]:::service
  S --> L["Registered with the<br/>load balancer"]:::edge
  L --> H(["Finally serving viewers<br/>5 to 6 minutes after the ask"]):::bad
WorksheetHow far behind does reactive scaling fall?
  • New viewers during the chase: ~1 million per minute
  • Time for a new server to be ready: 5 to 6 minutes
  • Viewers who arrive while it boots: 1M × 5 to 6 = 5 to 6 million

Five or six million people arrive before the help does. And at Hotstar's scale, the talk listed more trouble with standard AWS Auto Scaling groups:

  • "Insufficient capacity." You ask for 200 servers and the cloud has 50 of that type free in that zone.
  • One machine type per group. If that type runs out, the group is stuck.
  • Small steps. Servers arrive 10 or 20 at a time. Going from 100 to 800 could take 10 to 15 minutes, and when one zone ran short and kept being retried, Hotstar had seen 25 minutes.

Akash Saxena, then Hotstar's CTO, had written the conclusion in 2018: "Given the time it takes to auto-scale, it's a useless strategy when you need deal with spikes." And the line that sums up this whole story: "Do you end up ordering food as guests start arriving for your party. I hope you don't. We don't either."

Chapter 2: The Capacity Ladder#

So you order the food before the guests arrive.

Pre-warm. Before the match, the team scales everything up to the level its traffic models predict. Normal autoscaling still runs to keep pools topped up, but the big jump has already happened. Even load balancers need it: Saxena noted in 2017 that cloud load balancers "need to be specially 'warmed up'", and by 2019 one wasn't enough per app, so Hotstar ran four or five load balancers per application with weighted routing between them.

Climb rungs, not a ramp. Instead of tiny reactive steps, Hotstar wrote ladders: a table of how many servers each application gets at each level of concurrency. Kamboj's example: at 3 million viewers, this many servers; at 10 million, that many. As the crowd nears the top of a rung, you step up and keep headroom, spare capacity above what you need right now.

DiagramThe capacity ladder (illustrative rungs, real decision rules)
%% title: The capacity ladder (illustrative rungs, real decision rules)
flowchart TD
  C(["Live signals<br/>platform concurrency · requests per app"]):::client --> D{"Headroom on the<br/>current rung running out?"}:::hot
  D -->|"no"| W["Hold this rung"]:::ok
  D -->|"yes · decide within ~90 seconds"| U["Step up to the next rung"]:::core
  U --> P["Pre-built images<br/>app ready in about a minute"]:::service
  U --> F["Spread across many<br/>machine types and zones"]:::service
  P --> N(["Next rung live<br/>before the crowd arrives"]):::ok
  F --> N
  E["Cricket context<br/>star batter in · notification planned · timeout due"]:::note -.-> D

The rules behind it, from the talk:

  • Scale on concurrency and request rate, not CPU. Hotstar benchmarked how many requests each server can handle (its rated capacity) and stepped up when an app neared it. A server at 60–70% CPU that isn't slowing anyone down is not a reason to scale.
  • About 90 seconds to decide, since the app itself takes about a minute to boot.
  • Pre-baked server images, so no setup scripts run after boot.
  • Many machine types. A backup group with a different type kicks in automatically if one runs out, and AWS Spot Fleet lets one request spread across many types and zones.

Read the scorecard like a forecast#

This is what makes it so particular to cricket. Kamboj said the team watches the match "from a different angle": if Dhoni is about to bat, that can mean an extra 2 or 3 million viewers, so they get ready to scale. When a star gets out, traffic drops by 2 or 3 million.

Then there's marketing. When something exciting happens, a push notification may go out.

WorksheetWhat one push notification can do
  • People who receive it: 150 to 200 million
  • Share who tap it: 2 to 4%
  • Extra viewers the talk said to expect, in a very short span: 4 to 6 million

So the ladder always keeps a notification's worth of headroom spare.

Then automate it. Ladders needed humans stepping them up, which Saxena called effective and safe but costly. In 2019 Hotstar moved key services to Kubernetes (a system that runs many small app containers) and built its own autoscaling engine, running it in shadow (deciding alongside the humans without acting) until they trusted it. The result, in Saxena's words: "We supported 2x more concurrency in 2019 with 10x less compute overall."

Chapter 3: The Shock Absorber in Front#

25.3 million people watching means a flood of data leaving every second: more than 10 terabits per second, per the talk (Kamboj estimated that was around 70–75% of India's available internet bandwidth at the time).

A CDN (content delivery network) carries that: servers placed inside cities and internet providers, each called an edge. Video is cut into chunks a few seconds long, and everyone watching at the same quality wants the same chunks. So the edge near you fetches a chunk once, keeps a copy (a cache) and hands it to thousands nearby. The original lives at the origin, and the CDN's job is to make sure the origin almost never hears from viewers directly.

DiagramLayers of caching in front of the origin (simplified, illustrative)
%% title: Layers of caching in front of the origin (simplified, illustrative)
flowchart TD
  V(["Millions of viewers"]):::client --> E1["Edge · Mumbai"]:::edge
  V --> E2["Edge · Delhi"]:::edge
  V --> E3["Edge · Chennai"]:::edge
  E1 -->|"cache miss only"| MT["Mid-tier cache<br/>collapses duplicate requests"]:::cache
  E2 -->|"cache miss only"| MT
  E3 -.->|"edge down · viewers spill over"| MT
  MT -->|"one request per chunk"| O["Origin<br/>video packaging · APIs"]:::core
  O --> DB[("Databases and caches<br/>hard to grow mid-match")]:::store

Many CDNs add a middle layer. AWS calls its version Origin Shield: requests for the same missing object "are consolidated with other requests for the same object, resulting in as few as one request going to your origin." Hotstar hasn't published its exact CDN layout, but the talk describes the same shape: the edge near you, a mid-tier layer, then the origin.

Saxena titled a 2019 blog section "It's Not The CDN (alone)". A CDN is one tool, and you have to "architect your CDN to sit in front of your origin and build an amazing shock absorber". Two failures show why:

  • An edge goes down. Its viewers fall back to the mid-tier or origin. At 10 million viewers, even 5% spilling over is half a million extra requests on servers not sized for them.
  • Caches expire together. A TTL (time to live) is how long a cached copy is kept. If many objects expire at once (Saxena's examples: customer-facing timers and authentication tokens), every cache asks the origin at once. He called it a TTL cloudburst.

Chapter 4: Panic Mode#

Hotstar decided in advance what to switch off. In the talk, Kamboj called the must-always-work services P0: video, advertising, subscriptions and payments. Others, like recommendations and personalised rows on the home screen, could be turned off for maybe half an hour at the key moments, freeing their capacity for P0. This is graceful degradation: giving up nice-to-have features one by one instead of falling over all at once. The last-resort version was panic mode.

Normal vs panic mode

Normal

  • Video and ads play
  • Login and subscription checks as usual
  • Personalised home page and recommendations
  • Payments in real time

Panic mode

  • Video and ads play (never switched off)
  • Login failing? Watch without logging in until it's fixed
  • A simple default page, no personalisation
  • Payments failing? Skip payment and watch with an ad until it's fixed

Three details make it work:

  1. Fail quietly. A switched-off service returns a normal 200 OK, so services that depend on it don't fail and the app shows no scary pop-up.
  2. Tell the clients. A special code says "I'm in panic mode", so the app knows the problem is on Hotstar's side, backs off and takes the simpler path.
  3. Watch the gauge. A tool showed each service's load against its rated capacity. At around 80 to 90%, the team put it into panic mode by hand before it could drag others down.

The idea goes back to 2017, when Saxena described "positive panics" for Tier 1 actions (login, subscription, watch) that keep working visibly while the backend defers work, and "negative panics" for Tier 2 actions (account, logout) that can show a temporary failure. By 2019, facing a possible 30 million-plus match, the team "converged on a sequencing of which service to jettison (turn off), as the concurrency marched upwards". His summary: "Nobody cares for your surround features if they cannot consume your primary feature. The video must play on."

DiagramDegrading one step at a time
%% title: Degrading one step at a time
stateDiagram-v2
  Normal: Normal · every feature on
  Degraded: Degraded · extras jettisoned in a planned order
  Panic: Panic · only P0 left
  Normal --> Degraded: a service nears its rated capacity
  Degraded --> Panic: still climbing · 80 to 90 percent of rating
  Panic --> Degraded: fixed or traffic falls
  Degraded --> Normal: back under the rating
  class Normal ok
  class Degraded queue
  class Panic bad

Chapter 5: The Thundering Herd When the Wicket Falls#

Here's the twist. The peak doesn't kill a streaming platform. The drop does.

While you watch, your app does very little: it fetches the next few seconds of video and sends an occasional heartbeat. Then Dhoni is run out. You're upset. You press back.

The app opens the home page: the big banner, "continue watching", suggestions from your history. Each is an API call to a backend service. If millions leave the app, fine. If millions press back in the same few seconds, millions of home-page requests hit services that were quiet a moment ago. This is a thundering herd: many clients, triggered by one event, hitting the same thing at the same time.

DiagramThe wicket falls
%% title: The wicket falls
sequenceDiagram
  autonumber
  participant F as Millions of phones
  participant V as Video CDN
  participant H as Home page APIs
  participant D as Databases
  F->>V: next video chunk (cheap, cached)
  Note over F: Dhoni run out · millions press back
  F->>H: banner · continue watching · recommendations
  H->>D: look up history and personalised rows
  D-->>H: slow under the sudden load
  H-->>F: slow pages · apps retry · even more load
  Note over F,H: With jitter, backoff and cached fallbacks,<br/>the herd is spread out instead

Hotstar learned this in 2017, at an India vs Pakistan Champions Trophy match: "Virat Kohli walks in to bat — a notification is sent to a large segment and boom!" The login API went from 10,000 to 125,000 requests per second in a matter of seconds. The front end was scaled for 7 million viewers, but the login database wasn't scaled to match, and logins are hard to cache. It "started to melt down".

The fixes, from Hotstar's 2018 section titled "No Dumb Clients":

  • Jitter. Add a small random delay to non-urgent requests so a million phones don't ask in the same second. (Marc Brooker's AWS article on backoff explains why: without randomness, retrying clients move in lockstep.)
  • Exponential backoff. After a failure, wait before retrying, and wait longer each time. In Saxena's words: "Retries will make the problems worse."
  • Cache on the phone. If the home page from ten minutes ago is good enough, show it.
  • Pre-scale the screen people flee to. The home page must be ready the instant the video stops, because nobody knows when the wicket will fall.

Chapter 6: Rehearsing the Match Before It Happens#

Hotstar ran game days: full rehearsals firing fake, match-shaped traffic at the real platform. In 2018 they used Flood.io on top of Gatling, an open-source load testing tool. By 2019 the load generator was an in-house project called Project HULK, described in Kamboj's Rootconf and AWS re:Invent talks.

WorksheetHow big is the fake crowd?
  • Load machines: 3,000+ AWS c5.9xlarge servers
  • Each: 36 virtual CPUs and 72 GB of memory
  • CPUs: 3,000 × 36 = 108,000 vCPUs
  • Memory: 3,000 × 72 GB = 216 TB
  • Network out: 200 Gbps

Running from one region overwhelmed CDN edges and affected other customers, so HULK spread across eight AWS regions. The team also used logs from real matches to learn how traffic splits across services at each rung (at 5 million viewers, video APIs are flat out while payments are quiet) and shaped the fake traffic to match.

On top of load tests came chaos engineering: breaking things on purpose, in a controlled way, to see if the system copes. The talk's scenarios were match-night fears: a notification storm, a tsunami and sudden drop, slow scale-up, an edge failure, bandwidth limits, and extra latency (the talk's example: just +50 ms on one content API can slow the home page and app start-up downstream).

The goal is to find each service's breaking point and stay well below it. In one striking example from Saxena's 2019 post, a harsh simulation exposed a system using more than 70% of its compute for a feature Hotstar wasn't even using. Tuning it freed massive headroom without a single extra server.

DiagramThe rehearsal loop
%% title: The rehearsal loop
flowchart TD
  L[("Logs from real matches<br/>requests per service at each rung")]:::store --> M["Traffic model<br/>India bat first · rivals play · notification storm"]:::service
  M --> H["Project HULK<br/>3,000+ machines in 8 regions"]:::hot
  H --> P["Real platform<br/>CDN · load balancers · apps"]:::core
  C["Chaos tests<br/>edge failure · slow scale-up · added latency"]:::queue --> P
  P --> B["Breaking points found<br/>rated capacity per service"]:::bad
  B --> F["Fix · cache · re-size ladders<br/>plan what to switch off"]:::ok
  F --> M

That's why the team was ready for 50 million that day, about twice the real peak. Kamboj said that even 35 or 40 million would have been fine.

What Good Looks Like: The Match-Night Playbook#

Hotstar's approach, in order
  1. Model the crowd from past events: the shape, not just the peak.
  2. Find every breaking point with realistic load tests and chaos drills.
  3. Write the ladder: servers per service at each level of concurrency, plus headroom.
  4. Pre-warm servers and load balancers, and size data stores for the peak.
  5. Scale ahead on concurrency and request rate, using the match (and marketing's plans) as a forecast.
  6. Shield the origin with layered caches, sensible TTLs and clients that jitter, back off and cache.
  7. Decide what to switch off before the match, and do it calmly when a service nears its rating.
  8. Keep the video playing. Everything else is optional.

The Numbers, Then and Now#

Concurrency records cited by Hotstar, JioHotstar and the ICC
  • 8M — YouTube, Red Bull space jump, 2012
  • 10.3M — Hotstar, IPL 2018
  • 18.6M — Hotstar, IPL 2019 final
  • 25.3M — Hotstar, India vs New Zealand, 10 July 2019
  • 5.9 crore (59M) — Disney+ Hotstar, India vs Australia World Cup final, 19 November 2023
  • 7.25 crore (72.5M) — JioHotstar, India vs New Zealand T20 World Cup final, 8 March 2026

The record kept moving. At the 2023 World Cup, Disney+ Hotstar broke its peak several times, ending at 5.9 crore for the final, which the platform's head called "almost twice as high as any peak concurrency record set before the tournament." Viewing dipped to 4.6 crore after key Indian wickets fell: the same pattern as 2019, a crowd that grows with the score and leaves with the wickets. And it kept climbing: on 8 March 2026, the T20 World Cup final on JioHotstar (Disney+ Hotstar's successor) peaked at 7.25 crore, according to the ICC, the record as of October 2026.

Lessons for Any Engineer#

Few of us serve 25 million at once, but most of us run something with a predictable spike: a noon sale, a 10 AM exam result.

  • Concurrency, not total users, sizes your system.
  • If traffic grows faster than servers start, reactive autoscaling is always late. Pre-warm and scale ahead.
  • Scale on the signal that predicts load (users, requests, a scheduled event), not one that reports it after the fact (CPU).
  • The drop can hurt more than the peak. Get the screen everyone runs to ready.
  • Clients are part of your capacity plan: jitter, backoff, local caching.
  • Decide what to give up before you're under pressure, and rehearse until you know your breaking points.

Summary: Order the Food Before the Guests Arrive#

On 10 July 2019, 25.3 million people watched India lose a semi-final, live, at the same moment. The video kept playing through a million-a-minute rush, a rain-soaked night and a run-out that sent millions of thumbs to the back button.

None of that was luck. It was a ladder climbed a rung ahead of the crowd, a CDN built as a shock absorber, a list of features to switch off, and a fake crowd that had already played the match.

The next time a notification tells you your favourite batter has just walked in, remember: someone saw it coming and added the servers a few minutes ago.

Sources#

End of file · CASE-013

Read it through to close the case.

0/14 sections read. Each one counts after a few minutes of actual reading in it; skipping ahead doesn't. The case closes itself once every section is read.

All case files →
Debrief · close the case+50 XP for the right call

It is the 44th over. 18 million people are watching, the number is climbing about a million a minute, and a new server takes 5 to 6 minutes to be ready. Dhoni is batting. What do you do?

> debrief --interactiveCASE-013 · +25 XP at 5/7
End-of-file activity · decide

Run match night

You're on the capacity desk for the India vs New Zealand semi-final. The score moves, the crowd moves with it. Make the call at each moment before the traffic does.

  • ~4 MIN
  • 7 STEPS · 7 PTS
  • SOLVE AT 5/7
  • +25 XP
System DesignHotstarLive StreamingScalabilityAutoscalingCDNLoad TestingGraceful DegradationIndiaReal-World Engineering

Discussion

Add to the post-mortem
loading the record…