The Sticky Routing Amplification Trap
TRIGGER
Infrastructure bugs affecting less than 1% of requests were causing disproportionately severe user complaints—the same users kept reporting degradation while others saw normal performance, making bug reports appear contradictory and unreliable.
APPROACH
Anthropic's investigation revealed that 'sticky routing' (directing follow-up requests to the same server that handled previous requests) meant that once a user hit a buggy server, subsequent requests continued going to that server. Input: user requests routed via session affinity. Output: concentrated degradation on specific users while aggregate metrics show minimal impact. A context window routing error initially affected 0.8% of requests, but sticky routing meant 30% of Claude Code users who made requests during the affected period had at least one degraded message—and those users experienced repeated degradation. Resolution required fixing the routing logic to properly separate short-context and long-context server pools.
PATTERN
“Sticky routing concentrates intermittent bugs onto unlucky users—they experience 100% degradation while your dashboard shows 99% success. When user reports seem exaggerated versus metrics, check if they're stuck on buggy infrastructure.”
✓ WORKS WHEN
- Session affinity or sticky routing is used for cache efficiency or conversation continuity
- Infrastructure has heterogeneous configurations (different context windows, hardware platforms, feature flags)
- Bug affects specific server subset rather than all traffic uniformly
- User reports describe persistent degradation while aggregate metrics show minimal impact (<1-2%)
✗ FAILS WHEN
- Routing is fully random with no session affinity—bugs distribute evenly across users
- Infrastructure is homogeneous with identical configuration across all servers
- Bug affects all traffic equally (e.g., model weights issue)
- Single-request workloads where users don't have follow-up interactions