NSE Stock Exchange.

How Do You Decide Who Was First When Five Million Messages Arrive Every Second?

(An illustrative scene: Kavya, Rohit and every number on their screens are invented.) It is Tuesday, 24 March 2026, and the weekly Nifty options expire at 3:30 pm. Kavya Menon, an engineer on the operations floor of an exchange in Mumbai's Bandra Kurla Complex, has three numbers on the wall in front of her: messages a second into the matching engines, the lag of the market-data streams, and the count of member sessions that have been throttled. At 9:15 am the first number read a few hundred thousand. By 2:45 pm, with every weekly option a few rupees from expiring worthless or doubling, it has crossed three million and is still climbing.

In Indore, Rohit Sharma, a part-time options trader, taps "Buy" on a Nifty call that costs ₹4.20. His order travels from his broker to an exchange gateway, past a risk check, into a queue for that one contract and into an order book in memory, behind every order at the same price that came before it. In the same second, algorithms in racks a few metres from the matching engine send tens of thousands of messages for the same contract. Every one, and Rohit's, must be put in one exact order, matched by one rule, and published to every screen in the same sequence.

At 3:30 pm the options expire, and by the next day every trade must be guaranteed, netted and settled. And if the building's storage, telecom links or risk system fail at 2:58 pm, the market has to restart from another site with nothing lost.

This is the system we'll design today.

The Interview Begins#

You're interviewing at a firm that builds market infrastructure. The interviewer draws a single horizontal line on the whiteboard and writes "the order book" on it. On the left: "1,325 members, 200,000 terminals, 12,000 servers in our building". On the right: "everyone who wants to know the price".

You have: that was Zerodha Kite (Bonus Problem 13), where a broker took Asha's order at 9:15:00, checked her money, and sent it over a leased line to "the exchange's door". This quest starts on the other side of that door.

Your Questions#

Functional Requirements#

Non-Functional Requirements#

The Expiry Afternoon#

Partitions of the Matching Engine#

Market Data and the Journal#

Key Metrics Summary#

System Architecture#

%% title: An exchange: order entry, one sequenced core, and everything that reads its output
flowchart TD
  subgraph L1["Layer 1 · Members"]
    direction LR
    BRK@{ icon: "sd:building", label: "Brokers' offices<br/>leased lines, 200,000+ terminals" }
    COLO@{ icon: "sd:servers", label: "Co-location racks<br/>1,680 racks, members' servers" }
  end
  subgraph L2["Layer 2 · Order entry"]
    direction LR
    GR@{ icon: "sd:router", label: "Gateway router<br/>least-loaded gateway" }
    GW@{ icon: "sd:gateway", label: "Gateways<br/>session, message rate" }
    CHK["Checks before the book<br/>price band, quantity, value"]:::service
  end
  subgraph L3["Layer 3 · The core, one per partition"]
    direction LR
    SEQ["Sequencer<br/>next number, journal"]:::hot
    ME["Matching engine<br/>in-memory book, price-time"]:::core
    JRN@{ icon: "sd:log", label: "Journal<br/>replicated to near site" }
  end
  subgraph L4["Layer 4 · Everything downstream"]
    direction LR
    MD@{ icon: "sd:stream", label: "Market data<br/>multicast tick-by-tick" }
    CLR["NSE Clearing<br/>novation, margins, settlement"]:::core
    SURV@{ icon: "sd:monitor", label: "Surveillance<br/>alerts, positions" }
  end
  BRK ==> GR
  COLO ==> GR
  GR -.->|"assigns"| GW
  GW ==> CHK ==> SEQ
  SEQ ==> JRN
  SEQ ==> ME
  ME ==>|"acks, trades"| GW
  ME ==> MD
  ME ==>|"trades"| CLR
  ME --> SURV
  CLR -.->|"disable member"| CHK
  MD -.-> COLO
  class BRK,COLO client
  class GR,GW edge
  class JRN store
  class MD queue
  class SURV service

KEY INSIGHTS:

• One line decides "first". Everything before the sequencer can be parallel and stateless; everything after it reads one ordered stream. The sequencer is the only place an order gets its place in line
• The core is deterministic. A matching engine that is a pure function of its input log can be replicated by replaying the log and recovered by replaying it again
• Outputs fan out, inputs don't. One trade becomes two confirmations, a market-data tick, a clearing record and a surveillance event; all derive from the same sequence, so they agree

Data Flow: Rohit's Order at 2:47 pm#

%% title: One order from a broker's session to a trade, a tick and a clearing record
sequenceDiagram
  autonumber
  participant B as Broker's server
  participant G as Gateway
  participant S as Sequencer, partition 7
  participant M as Matching engine
  participant D as Market data, stream 7
  participant C as NSE Clearing
  B->>G: New order · BUY 1 lot NIFTY 24MAR call @ 4.20
  G->>G: message rate OK · price band OK · value OK
  G->>S: forward
  S->>S: seq 812,330,114 · journal, replicated
  S->>M: apply in sequence
  M-->>G: ack · order number 07 + sequence
  G-->>B: order accepted
  M->>M: crosses a resting sell at 4.20
  M-->>G: trade confirmations to both members
  M->>D: tick · trade and book change
  M->>C: trade for novation
  Note over D: one multicast packet, every subscriber at once

MEANWHILE, IN PARALLEL:

• Thousands of other orders for the same contract get the next numbers in the same second; each number is a place in the queue
• NSE Clearing updates both members' margins as trades arrive; a member that breaches is disabled at the order-entry checks

Deep Dive 1: The Sequencer and the Matching Engine (Week 5 Concepts: Consistency Models, Leader Election; Week 3: Logs)#

This builds on Week 5, Day 1 · Consistency Models, Week 5, Day 5 · Leader Election and Week 3, Day 1 · Queue vs Stream.

What is published: NSE has matched orders by "automated real-time price-time priority" since NEAT launched in fiscal 1995 (draft prospectus, June 2026). Its tick-by-tick feed is split into streams, each contract assigned to one (NSE market data circular, April 2025). SEBI's 2019 order described the tick-by-tick path of that time as data flowing from the communication gateway to "the Matching Engine (which matches data based on price–time priority)", then to a post-trade system, and then to a data centre whose "Sequencer" arranged data from multiple receivers "in sequence". How NSE's engine is threaded, partitioned and replicated, and how its order numbers are formed, is not in its public documents; what follows is a design.

%% title: One partition: many gateways in, one sequence, one thread, many readers out
flowchart TD
  G1@{ icon: "sd:gateway", label: "Gateway 1" }
  G2@{ icon: "sd:gateway", label: "Gateway 2" }
  G3@{ icon: "sd:gateway", label: "Gateway N" }
  SEQ["Sequencer, partition 7<br/>assign seq, append to journal"]:::hot
  JRN@{ icon: "sd:log", label: "Journal<br/>input log, replicated" }
  ME["Matching thread<br/>books for its contracts"]:::core
  OUT@{ icon: "sd:stream", label: "Output log<br/>acks, trades, book changes" }
  G1 ==> SEQ
  G2 ==> SEQ
  G3 ==> SEQ
  SEQ ==> JRN
  SEQ ==> ME
  ME ==> OUT
  class G1,G2,G3 edge
  class JRN store
  class OUT queue

• Partition by instrument. Every contract belongs to exactly one partition; hot weekly options are spread across partitions using the previous expiry's message counts.
• One writer per partition. A single sequencer thread numbers messages from all gateways and journals them; the matching thread applies them in that order. No locks, because nothing is shared.
• The sequence number is the clock. "Time priority" means sequence priority; gateway receive times are for measurement and surveillance only.
• Time is an event. The close, the end of pre-open order entry, a halt and a resume go through the sequencer like orders, so a replay sees them at the same points.
• Order numbers come from the sequence. (partition, sequence) is unique without coordination and reproducible on replay.
• Outputs are a log too. Acks, trades and book changes go to an output log that gateways, market data, clearing and surveillance each read at their own pace.

# core/book.py

"""
A price-time priority book applied by one thread from a sequenced log
(illustrative). No clocks, no randomness: the same log, the same trades.
"""

from collections import deque
from sortedcontainers import SortedDict   # ordered by price, deterministic


class Book:
    def __init__(self, contract):
        self.contract = contract
        self.bids = SortedDict(lambda p: -p)       # best (highest) bid first
        self.asks = SortedDict()                   # best (lowest) ask first
        self.orders = {}                           # order_no -> order, lookups only

    def apply(self, seq, msg, emit):
        if msg.kind == "NEW":
            self._new(seq, msg, emit)
        elif msg.kind == "CANCEL":
            o = self.orders.pop(msg.order_no, None)
            if o:
                level = self._side(o.side)[o.price]
                level.remove(o)
                if not level:
                    del self._side(o.side)[o.price]
                emit("CANCELLED", seq, o.order_no)

    def _new(self, seq, msg, emit):
        o = msg.as_order(order_no=(msg.partition, seq))   # number = place in line
        emit("ACK", seq, o.order_no)
        book = self.asks if o.side == "BUY" else self.bids
        while o.qty and book:
            best_price, queue = book.peekitem(0)
            if (o.side == "BUY" and best_price > o.price) or (o.side == "SELL" and best_price < o.price):
                break
            resting = queue[0]                          # earliest at the best price
            q = min(o.qty, resting.qty)
            emit("TRADE", seq, buy=o if o.side == "BUY" else resting,
                 sell=resting if o.side == "BUY" else o, qty=q, price=best_price)
            o.qty -= q
            resting.qty -= q
            if resting.qty == 0:
                queue.popleft()
                self.orders.pop(resting.order_no)
                if not queue:
                    del book[best_price]
        if o.qty:                                       # rest the remainder at the back
            self._side(o.side).setdefault(o.price, deque()).append(o)
            self.orders[o.order_no] = o
            emit("BOOK", seq, o.side, o.price)

    def _side(self, side):
        return self.bids if side == "BUY" else self.asks

The Pre-Open Call Auction#

What is published: NSE runs a pre-open session from 9:00 to 9:15 am. Orders can be entered, modified and cancelled from 9:00 to 9:08, with a "system driven random closure between 7th and 8th minute"; orders are matched and the opening price determined from 9:08 to 9:12; and 9:12 to 9:15 is a buffer for the transition to continuous trading. The opening price is the price at which the maximum volume can trade; if several prices tie, the one with the minimum unmatched quantity. The random closures for equities and equity derivatives are independent. After an index-based market-wide circuit breaker or an outage, the market reopens with a pre-open session. In derivatives it applies to near-month stock and index futures, not options (NSE member FAQ, November 2025), and NSE introduced it there in December 2025 (draft prospectus).

• Random closure is a fairness tool. If order entry closed at exactly 9:08:00.000, the last microsecond would be the most valuable, and the fastest participants would wait for it. A close at an unknown moment within the eighth minute removes the prize for being last.
• The auction is events in the log too. "Order entry closed" is a sequenced message at a randomly chosen point; the matching that follows is deterministic, so a replay finds the same opening price.

%% title: A trading day for one segment, as sequenced session events
stateDiagram-v2
  [*] --> PRE_OPEN_ENTRY: 9:00 am
  PRE_OPEN_ENTRY --> PRE_OPEN_MATCH: random close, 7th to 8th minute
  PRE_OPEN_MATCH --> BUFFER: opening price set
  BUFFER --> CONTINUOUS: 9:15 am
  CONTINUOUS --> HALTED: circuit breaker or outage
  HALTED --> PRE_OPEN_ENTRY: reopen with a call auction
  CONTINUOUS --> CLOSED: 3:30 pm
  CLOSED --> [*]
  class CLOSED ok
  class HALTED bad
  class PRE_OPEN_MATCH queue

Deep Dive 2: The In-Memory Book, Its Replicas and Its Recovery (Week 1 Concepts: Replication Trade-offs; Week 3: Audit Logs)#

This builds on Week 1, Day 2 · Replication Trade-offs and Week 3, Day 5 · Audit Log System.

What is published: NSE's primary data centre is in Mumbai and its DR data centre in Chennai, and "a supplemental NDR located near the PDC replicates data in real time, targeting near-zero data loss". The DR site mirrors the primary "for all critical applications, enabling switchover in under 45 minutes", and NSE runs "full live operations from the DR site twice a year". After February 2021 it moved key systems "from SAN-based to host-based replication", added replication link providers, limited link utilisation "to safeguard failover capacity", and keeps a 4:1 ratio for critical telecom links (draft prospectus, June 2026). SEBI's September 2024 circular requires every stock exchange to have a near site "to ensure near zero data loss", with "synchronous replication or appropriate replication" to it and a recovery point objective of "near zero"; clearing corporations must replicate synchronously for zero data loss (SEBI circular, 12 September 2024).

%% title: Where each copy of the journal lives, and when an output may leave
flowchart TD
  SEQ["Sequencer<br/>primary, Mumbai"]:::hot
  HS["Hot standby<br/>same site, applies the log"]:::core
  NS@{ icon: "sd:database", label: "Near site<br/>synchronous journal copy" }
  DR@{ icon: "sd:region", label: "DR site, Chennai<br/>asynchronous, full mirror" }
  GATE{"Input durable on<br/>standby and near site?"}
  OUT@{ icon: "sd:stream", label: "Acks, trades, ticks<br/>released in order" }
  SNAP@{ icon: "sd:file", label: "Snapshots<br/>every few minutes" }
  SEQ ==> HS
  SEQ ==> NS
  SEQ -.-> DR
  HS --> GATE
  NS --> GATE
  GATE ==>|"yes"| OUT
  HS -.-> SNAP
  class NS,SNAP store
  class DR external
  class OUT queue

• State machine replication. Primary and standby are two copies of one deterministic function fed the same journal: Week 1's leader-follower replication, replicating inputs rather than results.
• Output gating. An output leaves only after its input is on the standby and the near site; the near-site round trip is the price of near-zero data loss.
• Promotion needs a fence. A coordinator bumps an epoch on promotion; gateways reject output from an old epoch, so a returning primary is fenced off.
• Determinism is tested continuously. Replicas hash their books every N messages and compare; a mismatch marks the standby unfit for promotion before anyone needs it.
• Snapshots bound recovery time. A partition's live orders are small, so a snapshot every few minutes means a restart replays minutes, not a day.
• The journal is also the audit trail for surveillance and for any dispute about who was first.

# core/replica.py

"""
Applying a replicated journal with output gating and determinism checks
(illustrative). Outputs leave only when the input is durable on the standby and
the near site; replicas compare a hash of their books every CHECK_EVERY messages.
"""

import hashlib

CHECK_EVERY = 100_000


class Partition:
    def __init__(self, books, epoch):
        self.books, self.epoch, self.applied, self.pending = books, epoch, 0, []

    def on_sequenced(self, seq, msg, durable_upto, release):
        outputs = []
        self.books[msg.contract].apply(seq, msg, lambda *o: outputs.append(o))
        self.applied = seq
        self.pending.append((seq, outputs))
        while self.pending and self.pending[0][0] <= durable_upto():
            s, outs = self.pending.pop(0)
            release(self.epoch, s, outs)                 # in sequence order, never ahead of durability
        if seq % CHECK_EVERY == 0:
            return seq, self.state_hash()                # sent to the peer for comparison

    def state_hash(self):
        h = hashlib.sha256()
        for c in sorted(self.books):                     # deterministic order
            for side in (self.books[c].bids, self.books[c].asks):
                for price, queue in side.items():
                    h.update(f"{c}|{price}|".encode())
                    for o in queue:
                        h.update(f"{o.order_no}:{o.qty},".encode())
        return h.hexdigest()

Deep Dive 3: Order Entry: Gateways, Throttles and Checks Before the Book (Week 1 Concepts: Rate Limiting; Week 3: Backpressure)#

This builds on Week 1, Day 3 · Rate Limiting at Scale and Week 3, Day 3 · Backpressure and Flow Control.

What is published: Members pay for connectivity by "Messages Per Second (MPS)" category per asset class (draft prospectus, June 2026), and each co-location IP has a message rate that the member can upgrade or downgrade (NSE circular, 31 July 2026). NCL's risk management includes "online monitoring of member positions and automatic disablement from trading when limits are breached" (draft prospectus), and NSE offers kill switches at trading-member and user level (NSE member FAQ, November 2025). Since SEBI's April 2018 circular, every algorithm approved by an exchange gets a unique identifier, tagged on its orders. SEBI's February 2025 retail-algo circular requires algo orders sent through a broker's API to carry an exchange-provided identifier, and a retail client's own algo to be registered once it crosses an orders-per-second threshold, which the brokers' industry standards forum sets under the exchanges (SEBI circular, 4 February 2025). How NSE's gateways allocate sessions and enforce rates is in its confidential member protocols; the gateway below is a design. Exchanges typically reject messages above a connection's rate and disconnect a connection that goes far above it.

%% title: What an order meets before the book
flowchart TD
  IN@{ icon: "sd:api", label: "Message from a session<br/>connection, user ID, algo ID" }
  RATE{"Within subscribed rate?<br/>sliding 1 s window"}
  FORM{"Valid contract, tick,<br/>price band, quantity?"}
  LIM{"Under user and order<br/>value limits?"}
  STAT{"Member and user<br/>enabled?"}
  SEQ["To the partition's sequencer"]:::hot
  REJ["Reject with an error code"]:::bad
  CUT["Disconnect the connection<br/>above 2× its rate"]:::bad
  NCL["NSE Clearing<br/>real-time margins"]:::core
  IN ==> RATE
  RATE ==>|"yes"| FORM
  RATE -->|"over rate"| REJ
  RATE -->|"far over, 2×"| CUT
  FORM ==>|"yes"| LIM
  FORM -->|"no"| REJ
  LIM ==>|"yes"| STAT
  LIM -->|"no"| REJ
  STAT ==>|"yes"| SEQ
  STAT -->|"disabled"| REJ
  NCL -.->|"disable on breach"| STAT
  class IN edge

• Allocate capacity, not connections. The router places each connection by the message capacity already allotted on each gateway and breaks ties randomly, not by who asked first.
• A sliding window, not the clock's second. A per-clock-second counter lets a member send its whole budget at the end of one second and again at the start of the next. A window over the last 1,000 ms closes that gap.
• Reject first, disconnect second. Just over the rate, the member gets errors and can slow down; far over it, something is wrong on its side, so the connection goes.
• Backpressure stops at the member's door. The core never slows to absorb a burst; Week 3's rule is to push back at the edge, never queue unboundedly in the middle.
• Static checks are local. Price bands, tick size, freeze quantity and value limits need only reference data cached on every gateway.
• A kill switch per member, decided in advance. Disabling a member or a user ID at the gateways is one write that takes effect on the next message. Knight Capital had no procedures telling its staff when to disconnect a misbehaving system (war story: Knight Capital lost $460 million in 45 minutes because one server missed a deploy); an exchange should decide its own triggers before the afternoon it needs them.
• Margin is a status, not a query. The clearing corporation computes it; the gateway reads one bit per member, pushed on change. Without that monitoring the market can't safely continue, as February 2021 showed (Deep Dive 6).

# gateway/throttle.py

"""
Per-connection message rate with a sliding one-second window, and the cheap checks
before the book (illustrative). Over the subscribed rate: reject; over 2×:
disconnect. Static limits are cached reference data; margin is a pushed status bit.
"""

from collections import deque

WINDOW_NS = 1_000_000_000


class ConnectionThrottle:
    def __init__(self, subscribed_rate):
        self.rate = subscribed_rate
        self.sent = deque()                        # receive times of accepted + rejected msgs

    def admit(self, now_ns):
        while self.sent and self.sent[0] <= now_ns - WINDOW_NS:
            self.sent.popleft()
        self.sent.append(now_ns)
        n = len(self.sent)
        if n <= self.rate:
            return "ADMIT"
        if n <= self.rate * 2:
            return "REJECT_RATE"                   # error code back to the member
        return "DISCONNECT"                        # log off the connection


def check(order, ref, user, member_status):
    c = ref.contracts.get(order.token)
    if c is None:
        return "INVALID_CONTRACT"
    if order.price % c.tick or not (c.band_low <= order.price <= c.band_high):
        return "PRICE_OUTSIDE_BAND"
    if order.qty % c.lot or order.qty > c.freeze_qty:
        return "QUANTITY_FREEZE"
    if order.qty * order.price > user.order_value_limit:
        return "ORDER_VALUE_LIMIT"
    if member_status.get(order.member) != "ENABLED":
        return "MEMBER_DISABLED"                   # flipped by NSE Clearing on a breach
    return None

Deep Dive 4: Exactly-Once Orders Across a Disconnect (Week 2 Concepts: Idempotency, Timeout Hell)#

This builds on Week 2, Day 2 · Idempotency in Practice and Week 2, Day 1 · Timeout Hell. The broker's half of this problem, one order per client tap, is in Zerodha Kite's Deep Dive 1.

What is published: NSE's public documents don't describe how a member recovers a session or changes a live order; those rules are in its member protocols, which are marked confidential. What follows is our design, using two patterns common in exchange order-entry protocols: a reconnecting member replays its own messages by sequence, stream by stream, from the last one it saw, and a modification names the version of the order it was based on, so it fails if the order traded in between.

%% title: Reconnect, replay, then act on what really happened
sequenceDiagram
  autonumber
  participant B as Broker's server
  participant R as Gateway router
  participant G2 as New gateway
  participant P as Partition 7 output log
  B->>R: logon · session dropped at 3:01:07
  R-->>B: gateway 2 · new session key
  B->>G2: download stream 7 from last seq seen
  G2->>P: read this trader's messages after that seq
  P-->>B: ACK new order 07-0000812330990
  P-->>B: TRADE older order · 4 lots at 5.90
  Note over B: the new order exists · the older one changed
  B->>G2: modify older order · last activity = the trade
  alt reference matches
    G2-->>B: modified
  else stale reference
    G2-->>B: rejected · stale version, re-read
  end

• The sequence is the cursor. "What did I miss?" is a range query on a per-stream log, which is why the replay goes stream by stream.
• Ask, don't resend. A timeout says nothing about whether the other side acted (Week 2). Read the authoritative log; resend only what the receiver can recognise.
• Conditional writes on a version. The last-activity reference is optimistic concurrency: "change the order as I last saw it". A fill in between makes it fail, so a broker can't re-buy quantity that already filled.
• Member references as idempotency keys. Our addition: a reference per new order, unique per session per day, remembered at the gateway; a duplicate returns the original order number.
• Downstream, exactly-once is at-least-once plus a high-water mark. Clearing and surveillance record the last sequence applied per partition, so redelivery is harmless.
• The exchange is the source of truth. The broker reconciles to it, as the Zerodha quest put it; the exchange's job is to make that reconciliation a simple replay.

Deep Dive 5: Market Data: Multicast, Fairness and the Co-location Lesson (Week 4 Concepts: Fan-Out; Week 8: Sequenced Event Streams)#

This builds on Week 4, Day 4 · Feed Caching and Week 8, Day 1 · Event Ingestion.

What SEBI found (2019): In an order dated 30 April 2019 in the matter of OPG Securities, SEBI's whole-time member described NSE's tick-by-tick feed as it ran from 2010 to 2014. Tick data "was earlier disseminated over TCP/IP wherein the information is delivered one–by–one", to dissemination (POP) servers and then to members in order of login: "the earliest login TM is ranked first", and the rank "does not change even if a TM gets disconnected on that trading day and reconnects". NSE had developed a "Randomizer" in 2011 that "would randomly pick a connection to begin dissemination of data" but implemented it only for servers carrying customised data, not for the primary and secondary POP servers. The secondary POP server, "a fail–safe/fall back POP Server", "would invariably be lesser loaded", so a member on it could get data ahead of members on the primaries. The order found that OPG Securities had made unauthorised connections to the secondary POP server on 134 days and made unlawful gains of ₹15.57 crore, and barred the firm from trading for its own account, and its directors from the securities market, for five years. NSE introduced multicast tick-by-tick data in derivatives on 7 April 2014 and in the cash market on 10 November 2014.

What SEBI decided about NSE, and what happened on appeal: In a separate order the same day, SEBI's whole-time member directed NSE to disgorge ₹624.89 crore with 12% interest from April 2014 and barred it from the securities market for six months. The order found that NSE had not violated SEBI's regulations against fraudulent and unfair trade practices, but held that it "had not exercised the requisite due diligence while putting in place the TBT architecture" (as summarised by SAT, January 2023; NSE draft prospectus, June 2026). On 23 January 2023 the Securities Appellate Tribunal took a different view of the architecture. It found randomness in how data reached the ports, so that a member who logged in first "was not assured to receive the tick first", and held that NSE had not breached the fair-access regulation. It found lapses instead: members' IPs were spread unevenly across ports with no load balancer, and frequent connections to the secondary server weren't monitored. It set aside the disgorgement, upheld the six-month bar and the system-audit direction, and directed NSE to deposit ₹100 crore in SEBI's Investor Protection and Education Fund because NSE "is required to pay a price for the lack of due diligence" (SAT order, 23 January 2023). SAT also set aside, on 12 July 2023, a ₹1 crore penalty that a SEBI adjudicating officer had imposed on NSE in the matter. In a related "dark fibre" case, SEBI had alleged preferential point-to-point connectivity for certain members through an unauthorised service provider, "conferring unfair latency advantage"; SAT set aside SEBI's ₹62.58 crore disgorgement direction against NSE on 9 August 2023 and a ₹7 crore penalty on 14 December 2023 (draft prospectus).

For OPG Securities, SAT upheld the violations in January and July 2023 but sent the amounts back to SEBI; on rehearing, SEBI's adjudicating officer imposed a ₹5 crore penalty on the firm and its directors, plus ₹10 lakh each on the firm and one director (SEBI order, 2 April 2025). On SAT's remand SEBI also re-examined whether NSE and seven former officials had colluded with OPG. On 13 September 2024 it held that the evidence did not establish collusion or connivance and disposed of those proceedings without any direction, while recording that NSE "did not have a detailed defined policy for the use of Colo facility" (SEBI order, 13 September 2024). SEBI appealed the SAT orders on NSE to the Supreme Court. NSE applied to settle the co-location and dark fibre cases in June 2025 and revised its terms in March 2026; SEBI accepted the revised terms in principle on 30 July 2026, for ₹1,491.21 crore in all (NSE disclosure, 30 July 2026), and NSE paid the balance on 31 July. The Supreme Court disposed of SEBI's dark fibre appeals on 3 September 2026 and its co-location appeals by an order dated 18 September 2026, in light of the settlement (NSE disclosures, 4 and 23 September 2026).

What changed: SEBI's August 2016 discussion paper proposed minimum resting times, random speed bumps and separate queues for co-location orders (Millennium Post, 6 August 2016). Its April 2018 circular instead required exchanges to give the tick-by-tick feed "to all the trading members, free of cost", to offer managed co-location for smaller members, to publish "minimum, maximum and mean latencies and latencies at 50th and 99th percentile", and to tag every approved algorithm's orders with a unique identifier. NSE's tick-by-tick feed today goes out on UDP multicast in streams, each stream on "two separate Multicast Channels (Source 1 and Source 2) ... on active-active basis", with the feed on one channel "delayed and lagging behind the other". A client that misses ticks on both can ask a recovery server over TCP, which serves them from a file and so lags the live feed, or rebuild the book from a snapshot (NSE market data circular, 25 April 2025).

%% title: Tick-by-tick multicast: two feeds per stream, and recovery for gaps
flowchart TD
  ME["Matching engine<br/>output log, partition 7"]:::core
  PUB["Publisher<br/>stream 7, seq per stream"]:::hot
  A@{ icon: "sd:network", label: "Multicast feed A<br/>UDP, every subscriber" }
  B@{ icon: "sd:network", label: "Multicast feed B<br/>same packets, lagging" }
  SUB@{ icon: "sd:servers", label: "Member's feed handler<br/>arbitrate A and B" }
  LB@{ icon: "sd:load-balancer", label: "Recovery limits<br/>per member, rate-limited" }
  REC@{ icon: "sd:server", label: "Recovery servers<br/>missed ticks over TCP" }
  BC@{ icon: "sd:building", label: "Broadcast and vendors<br/>limited depth, everyone else" }
  ME ==> PUB
  PUB ==> A ==> SUB
  PUB ==> B ==> SUB
  SUB -.->|"gap on both"| LB --> REC
  PUB --> BC
  class SUB client
  class LB edge
  class REC store
  class BC external

• Fairness is a property of the transport. In a sequential sender the n-th write is always later than the first. Multicast moves the copying into switch hardware; what remains is the physical path, so our design would also equalise and measure the path to every co-location rack.
• Sequence numbers make UDP usable. A gap shows the moment the next packet arrives, and the second feed fills most gaps without asking anyone.
• Recovery must not become a back door. It is point-to-point, so it is rate-limited and slower than the feed; nobody gains by "recovering".
• A backup is for failures only. The 2019 order's lesson for designers: a fallback less loaded than the main path is a faster path. Allow it only when the primary is down, and log every use.
• Publish what fairness looks like. Latency percentiles from reference racks (SEBI, 2018) let members check the claim of equal access.

Deep Dive 6: Disaster Recovery and Failover: When the System Next Door Fails (Week 2 Concepts: Circuit Breakers; Week 10: Incident Management)#

This builds on Week 2, Day 3 · Circuit Breakers and Week 10, Day 5 · Incident Management. The full story is the war story India's biggest stock exchange stopped trading for four hours.

What is published: On 24 February 2021, issues in the links with telecom service providers "impacted our Storage Area Network ('SAN') system deployed in production and near disaster recovery site, resulting in the primary SAN becoming inaccessible to host servers. While the trading system was not impacted, outage of risk, clearing and settlement systems at NCL, index and surveillance systems at our Company led to a complete halt in all trading segments" (NSE draft prospectus, June 2026). NSE halted trading in all segments from 11:40 am; trading resumed at 3:45 pm after a 15-minute pre-open session from 3:30 pm, and the day was extended to 5 pm (SEBI settlement order, 28 June 2023). SEBI's investigation found that trades from 10:06 am until the halt had not been subject to online real-time risk management (same order). NSE's prospectus describes the halt as lasting "five hours and 24 minutes", which matches the span from 10:06 am to the 3:30 pm reopening session; trading itself was stopped from 11:40 am to 3:45 pm. SEBI imposed a financial disincentive of ₹25 lakh by a letter of 2 July 2021 because recovery "exceeded the stipulated recovery time objective"; its show-cause proceedings of August 2021 were settled in June 2023, with NSE paying ₹49.77 crore and NCL ₹22.88 crore (draft prospectus; SEBI settlement order, 28 June 2023). On 22 March 2021 SEBI required market infrastructure institutions to declare a disaster within 30 minutes of a critical system failing and restore operations "including from DRS within 45 minutes of the declaration", set a recovery point objective of 15 minutes, and required unannounced live trading from the DR site at 45 minutes' notice (SEBI circular, 22 March 2021). NSE's remediations since are listed in Deep Dive 2; it has also built an air-gapped "Isolated Recovery Environment" with immutable backups for cyber incidents (draft prospectus).

%% title: Failover as one unit: detect, halt, declare, switch, replay, reopen
sequenceDiagram
  autonumber
  participant OPS as Operations
  participant PDC as Primary site
  participant DR as DR site
  participant GR as Gateway router
  participant MEM as Members
  PDC-->>OPS: risk, index, surveillance unhealthy · 2:58 pm
  OPS->>PDC: halt all segments
  OPS->>MEM: notice within minutes · trading halted
  Note over OPS: decide within 30 minutes · disaster declared
  OPS->>DR: activate trading, risk, clearing, index, surveillance
  DR->>DR: replay journals to the last replicated sequence
  DR-->>OPS: books and positions reconciled
  OPS->>GR: point routers at DR gateways
  MEM->>GR: logon · new gateway
  MEM->>DR: message download · learn order states
  Note over DR,MEM: multicast streams restart at seq 1
  DR->>MEM: pre-open call auction, then continuous trading

• Find the shared fate. Two providers that fail together, and a near-DR copy on production's storage network, are one failure domain with two names. Map every critical system down to links, storage and power, and test them failing together.
• A dependency the market can't run without is part of the core. The risk system needs the matching engine's redundancy and drills; in 2021 the trading system was the healthy part.
• Prefer replication the application controls. Host-based replication of journals and risk state over independent links: Deep Dive 2's journal design, applied to every critical system.
• Halting is a circuit breaker for the whole market. Like Week 2's breaker, it fails fast instead of letting risk pile up; the reopening auction is the half-open probe.
• Decide with a clock, not a meeting. Pre-agreed triggers turn the 2:58 pm judgement call into a procedure.
• Drill it live. Twice-yearly live sessions and short-notice drills are what make a 45-minute switch credible.

An Expiry Tuesday#

After the Match: Clearing and Settlement Through NSE Clearing#

What is published: NSE Clearing Limited (NCL) acts as central counterparty: "we interpose ourselves between buyers and sellers, thereby assuming the counterparty credit risk of each transaction", NSE says. Its risk system "operates on a real-time, intra-day basis to establish and update margin requirements"; for derivatives it "constructs scenarios of probable changes in underlying prices and volatilities in order to identify the largest loss a portfolio might reasonably be expected to suffer from one day to the next", and cash market margins use value-at-risk, plus extreme loss margins. At least 50% of a clearing member's collateral must be cash or cash equivalents. NCL runs daily stress tests and keeps a Core Settlement Guarantee Fund of ₹13,079 crore (₹130,791.51 million, 31 March 2026); in a default, losses fall in order on the defaulter's money, insurance, clearing corporation resources, the Core SGF, further clearing corporation resources and, last, capped contributions from other members (NSE draft prospectus, June 2026).

India moved equities to T+1 in tranches from February 2022 to January 2023, smallest companies first (Thomas Murray, February 2024). SEBI's March 2024 circular introduced an optional T+0 cycle as a beta for 25 stocks and a limited set of brokers from 28 March 2024, trading from 9:15 am to 1:30 pm within a 100-basis-point band around the T+1 price, with no netting against T+1 obligations (SEBI circular, 21 March 2024). SEBI's December 2024 circular extended T+0 to the top 500 stocks by market capitalisation, 100 at a time each month from 31 January 2025, opened it to all brokers, and added a T+0 block deal window from May 2025 (SEBI circular, 10 December 2024). SEBI's December 2023 consultation paper had also proposed a second phase of "optional immediate trade-by-trade settlement for funds and securities", with trading until 3:30 pm (Business Today, 22 December 2023); NSE's June 2026 prospectus describes NCL as providing T+1 and T+0 cycles.

%% title: From a matched trade to settled money and securities
flowchart TD
  TR@{ icon: "sd:stream", label: "Trades<br/>output logs by partition" }
  NOV["Novation<br/>NCL becomes counterparty"]:::core
  MRG["Real-time margins<br/>VaR, scenarios, ELM"]:::hot
  STAT@{ icon: "sd:shield", label: "Member status<br/>enabled or disabled" }
  NET["Netting<br/>per member, per security"]:::service
  T0@{ icon: "sd:clock", label: "T+0 cycle<br/>trades to 1:30 pm, same day" }
  T1@{ icon: "sd:clock", label: "T+1 cycle<br/>next day" }
  BANK@{ icon: "sd:bank", label: "Clearing banks<br/>funds pay-in, pay-out" }
  DEP@{ icon: "sd:database", label: "Depositories<br/>securities pay-in, pay-out" }
  SGF@{ icon: "sd:wallet", label: "Core SGF<br/>if a member defaults" }
  TR ==> NOV ==> NET
  NOV ==> MRG ==> STAT
  NET ==> T0
  NET ==> T1
  T0 --> BANK
  T1 --> BANK
  T1 --> DEP
  MRG -.-> SGF
  class TR queue
  class STAT edge
  class BANK,DEP store
  class SGF store

• Novation lets the core be anonymous. A buyer's counterparty is the clearing corporation, so the matching engine can match purely on price and time.
• Margins run on the trade stream. The risk engine consumes trades from the output logs, exactly once by sequence, revalues positions and updates each member's status in seconds: the one bit the gateways read.
• Netting shrinks the work: 201 million trades become one net amount of money and one net quantity per security per clearing member per cycle.
• T+0 is a separate book and netting set, with its own session and price band, settled the same day.
• Instant settlement would change the shape: checking funds and securities trade by trade, at the match, instead of netting afterwards. The "margin is a status bit" design would have to become a pre-funded check per trade.

Edge Cases#

Surveillance#

What is published: NSE's surveillance system "provides real-time monitoring, automated alerts of abnormal trading activity and analysis of surveillance data"; NSE monitors social media and verifies rumours, runs pre-emptive measures such as the additional, graded and enhanced surveillance measures and periodic call auction sessions for selected stocks, and supports SEBI's own integrated surveillance department (draft prospectus, June 2026). SEBI's October 2024 circular noted that position limits for index derivatives were monitored at the end of the day and that "amidst the large volumes of trading on expiry day, there is a possibility of undetected intraday positions beyond permissible limits"; from April 2025 exchanges must also check them intraday, with at least four position snapshots a day "randomly taken during pre-defined time windows". From 20 November 2024 the same circular let each exchange offer weekly contracts on only one benchmark index, set a minimum value of ₹15 lakh for new index derivative contracts, and added a 2% extreme loss margin on short options on expiry day. SEBI's May 2025 framework then limited each exchange to one weekly expiry day, Tuesday or Thursday; NSE's contracts moved to Tuesday from 1 September 2025 (NSE draft prospectus). SEBI's August 2026 study found that 87.7% of individual traders in equity derivatives made net losses in FY2026, against 90.9% in FY2025, with aggregate net losses of ₹91,685 crore in FY2026 (SEBI study, August 2026).

• Surveillance reads the same log. Every message with its sequence, receive time, client code and algo ID is on it, so any book at any moment can be rebuilt by replay.
• Random snapshots are fairness again. A check at a known time can be dodged by trimming a position just before it, so the times are random, like the pre-open's close.
• Expiry-day patterns get their own alerts: large trades in the underlying near the close paired with large options positions, quote stuffing on one contract, modifications far outnumbering trades.

Monitoring Dashboard#

SLOs for a Trading Day#

The interviewer leans back. "One more."

Summary: Concepts Applied from 10-Week Course#

Why NSE Matters#

Self-Assessment Checklist#

After studying this case study, you should be able to:

The core:

Replication, entry and idempotency:

Market data, clearing and recovery:

Sources#

NSE (primary):

• Docs

  NSE — Draft Red Herring Prospectus, 17 June 2026, filed with SEBI (message and trade peaks; ~5 million messages a second; NEAT and price-time matching; data centres, co-location racks, terminals; DR sites and switchover; the February 2021 failure, disincentive, settlement and remediation; co-location and dark fibre proceedings, SAT orders and settlement applications; NCL's risk management and Core SGF; surveillance; investors, members and derivatives volumes): https://www.sebi.gov.in/filings/public-issues/jun-2026/national-stock-exchange-of-india-limited-drhp_102189.html

• Book

  NSE — Market Data consolidated circular NSE/MSD/67677, 25 April 2025 (tick-by-tick multicast streams, two active-active channels, TCP recovery and snapshot recovery): https://nsearchives.nseindia.com/content/circulars/MSD67677.pdf

• Paper

  NSE — Colocation update: bulk upload for IP message rate change requests, NSE/MSD/75498, 31 July 2026 (a message rate per co-location IP): https://nsearchives.nseindia.com/content/circulars/MSD75498.pdf

• Paper

  NSE — Member FAQs on Pre-open Session in Equity Derivatives, November 2025 (timings, random closure, equilibrium price, futures only, reopening after a halt, kill switches): https://nsearchives.nseindia.com/web/mediaattachment/2025-12/FAQs_for_PreOpen_Session_in_Equity_Derivatives_FO_Segment_20251202123640.pdf

• Paper

  NSE — Disclosure of the SAT order of 23 January 2023 in the co-location matter, with the order attached, 24 January 2023: https://nsearchives.nseindia.com/web/sites/default/files/inline-files/SAT%20Order-23012023%20Announcement_1.pdf

• Paper

  NSE — Regulation 30 disclosure: SEBI's in-principle acceptance of the revised settlement terms, 30 July 2026: https://nsearchives.nseindia.com//web/mediaattachment/2026-07/Reg_30-Settlement_Order-30.07.2026_20260730224443.pdf

• Paper

  NSE — Regulation 30 disclosure: Supreme Court disposal of the dark fibre appeals on 3 September 2026, 4 September 2026: https://nsearchives.nseindia.com//web/mediaattachment/2026-09/Reg_30_-_Settlement_of_Dark_Fibre_Case_20260904160928.pdf

• Paper

  NSE — Regulation 30 disclosure: Supreme Court order of 18 September 2026 disposing of the co-location appeals, 23 September 2026: https://nsearchives.nseindia.com//web/mediaattachment/2026-09/Reg_30_-_Settlement_of_Colo_Case_20260923165056.pdf

SEBI (primary):

• Book

  SEBI — Final Order in the matter of OPG Securities, WTM/GM/EFD/02/2019–20, 30 April 2019 (the TCP/IP tick-by-tick architecture and its findings): https://www.sebi.gov.in/sebi_data/attachdocs/apr-2019/1556633197432.pdf

• Read

  SEBI — Order in the matter of NSE and Others (Co-location), WTM/KV/MRD/MRD-PoD-3/30754/2024-25, 13 September 2024: https://www.sebi.gov.in/enforcement/orders/sep-2024/order-in-the-matter-of-nse-and-others-co-location-_86674.html

• Read

  SEBI — Adjudication order against OPG Securities and others in the matter of NSE co-location, 2 April 2025: https://www.sebi.gov.in/enforcement/orders/apr-2025/adjudication-order-against-opg-securities-pvt-ltd-and-others-in-the-matter-of-nse-co-location_93240.html

• Read

  SEBI — Settlement order in the matter of the trading halt on NSE, SO/VV/AS/2023-24/6558, 28 June 2023 (timeline of 24 February 2021; settlement amounts): https://www.sebi.gov.in/enforcement/orders/jun-2023/settlement-order-in-the-matter-of-trading-halt-on-the-national-stock-exchange-of-india-ltd_73222.html

• Read

  SEBI — Circular SEBI/HO/MRD1/DTCS/CIR/P/2021/33, 22 March 2021 (BCP and DR of MIIs: 30-minute declaration, 45-minute RTO, 15-minute RPO, unannounced live trading from DR), text: https://taxguru.in/?p=763305

• Paper

  SEBI — Circular SEBI/HO/MRD/TPD/P/CIR/2024/119, 12 September 2024 (near sites, near-zero RPO), copy: https://ftp.caalley.com/sebi24/1726134414026.pdf

• Read

  SEBI — Circular SEBI/HO/MRD/TPD-1/P/CIR/2024/132, 1 October 2024 (expiry-day measures; intraday position snapshots): https://www.sebi.gov.in/legal/circulars/oct-2024/measures-to-strengthen-equity-index-derivatives-framework-for-increased-investor-protection-and-market-stability_87208.html

• Paper

  SEBI — Circular SEBI/HO/MRD/MRD-PoD-3/P/CIR/2024/20, 21 March 2024 (beta T+0), copy: https://ftp.caalley.com/sebi24/1711018384850.pdf

• Paper

  SEBI — Circular SEBI/HO/MRD/MRD-PoD-3/P/CIR/2024/172, 10 December 2024 (optional T+0 for the top 500), copy: https://ftp.caalley.com/sebi24/1733829354031.pdf

• Book

  SEBI — Circular SEBI/HO/MRD/DP/CIR/P/2018/62, 9 April 2018 (free tick-by-tick feed, managed co-location, latency percentiles, algo IDs), summary: https://corporatelawreporter.com/measures-to-strengthen-algorithmic-trading-and-co-location-proximity-hosting-framework/

• Paper

  SEBI — Circular on safer participation of retail investors in algorithmic trading, 4 February 2025, copy: https://ftp.caalley.com/sebi25/1738665456458.pdf

• Read

  SEBI — Profitability of Individual Traders in the Equity Derivatives Segment (FY25–FY26), August 2026: https://www.sebi.gov.in/reports-and-statistics/research/aug-2026/study-profitability-of-individual-traders-in-the-equity-derivatives-segment-fy25-fy26-_103835.html

Reporting:

• Blog

  Millennium Post — SEBI proposes new framework for algo trading, co-location, 6 August 2016: https://www.millenniumpost.in/sebi-proposes-new-framework-for-algo-trading-co-location-156859

• Read

  News On AIR — NSE, BSE swap derivatives expiry days, 1 September 2025: https://www.newsonair.gov.in/nse-bse-swap-derivatives-expiry-days

• Paper

  Business Today — SEBI's consultation paper on instant settlement, 22 December 2023: https://www.businesstoday.in/amp/markets/top-story/story/sebi-releases-consultation-paper-on-implementation-of-instant-trade-settlement-what-it-says-410575-2023-12-22

• Read

  Thomas Murray — T+1 settlement cycles: lessons from India, 29 February 2024: https://thomasmurray.com/insights/t1-settlement-cycles-lessons-india-and-asia-pacific

• Read

  Angel One — NSE lists at ₹1,800 on BSE, September 2026: https://www.angelone.in/news/ipos/nse-share-price-stock-lists-at-1800-00-with-a-0-84-premium-on-bse

Design note: the architecture beyond what NSE's prospectus, circulars and FAQs and the SEBI and SAT orders describe, the code, thresholds, dashboards and every number labelled as an assumption or estimate are an illustrative design, not NSE's internal systems. The expiry-afternoon shape, burst factor, partition capacity and count, message and journal sizes, hottest-contract share, the results-day forecast, the 2× disconnect threshold, the session replay and last-activity checks, and the member-reference de-duplication are assumptions or design choices. Kavya Menon, Rohit Sharma and everything they see are illustrative. Regulatory facts are summarised for engineering context, are not legal advice, and should be checked against current SEBI circulars.

Further Reading#

Related systems to study:

• Read

  Zerodha Kite (Bonus Problem 13 in this series): the broker's side of the door: the 9:15 spike, client margin, one order per tap, and market data to a million screens

• Read

  Visa and Mastercard (Bonus Problem 4): authorisation now, clearing and settlement later, through a network in the middle

• Read

  Razorpay (Bonus Problem 18): settlement and reconciliation from a payment provider's side

• Read

  Dream11 (Bonus Problem 25): a scheduled spike and a deadline that must mean one thing, decided in one place

From the war stories:

• Read

  India's biggest stock exchange stopped trading for four hours

• Docs

  Knight Capital lost $460 million in 45 minutes because one server missed a deploy

• Read

  In the 2010 Flash Crash, an algorithm sold faster the more trading it helped cause

Foundational concepts (this course):

• Read

  Week 1, Day 2 — Replication Trade-offs

• Read

  Week 1, Day 3 — Rate Limiting at Scale

• Read

  Week 2, Day 1 — Timeout Hell

• Read

  Week 2, Day 2 — Idempotency in Practice

• Read

  Week 2, Day 3 — Circuit Breakers

• Read

  Week 3, Day 1 — Queue vs Stream

• Read

  Week 3, Day 3 — Backpressure and Flow Control

• Read

  Week 3, Day 5 — Audit Log System

• Read

  Week 4, Day 4 — Feed Caching

• Read

  Week 5, Day 1 — Consistency Models

• Read

  Week 5, Day 5 — Leader Election

• Read

  Week 8, Day 1 — Event Ingestion

• Read

  Week 10, Day 5 — Incident Management

"One sequence per book, one packet for everyone, and nothing acknowledged that a surviving copy can't replay."