Segment takes in hundreds of thousands of events per second from its customers' apps and forwards them to tools such as Google Analytics: over a hundred types of destinations.
At first, every event went through one queue. If one destination slowed down or went down, its retries flooded that queue and delivery got slower for every destination. So the team gave each destination its own service and its own queue. Now one bad destination only backed up itself. Later, each destination's code got its own repo too.
It worked, and then it grew. The team added about three destinations a month. Shared libraries ended up at different versions in different repos, because updating one meant testing and deploying dozens of services. Each service had its own load pattern, from a handful of events a day to thousands a second, and the on-call engineer was often paged for load spikes. Eventually 3 full-time engineers spent most of their time just keeping the system alive.
So they went back. The now over 140 services became one. A new system, Centrifuge, replaced the per-destination queues. All the code moved into one repo, with one version of each of its 120 unique dependencies. Tests that called real partner APIs were slow and flaky; a tool called Traffic Recorder replayed saved responses instead, so tests for all 140+ destinations ran in milliseconds.
Afterwards, one engineer could deploy the service in minutes. The team made 32 improvements to the shared libraries in the microservices era, and 46 in the year after. The trade-offs, as Segment lists them: one destination's bug can now crash the service for all of them, and in-memory caches work less well, spread across 3,000+ processes.
- Over 140 services merged into 1
- 3 full-time engineers mostly keeping the old system alive
- 120 unique dependencies brought to one version each
- Shared-library improvements: 32 before, 46 in the year after
Why it matters: Microservices are a trade, not a default. Isolation has a price in deploys, versions and on-call pages; when that price outgrows the benefit, merging back can be the right call.