Design for a System That Is Refusing Work

Systems & Architecture · errors, status, scale

Updated 2026-08-04

Understand that a healthy system under extreme load turns requests away on purpose, a technique called load shedding. Refusing some work quickly is how the rest of the work survives, so this state is a success for the system and needs to read as one for the user.
Distinguish "we are busy" from "this is broken". They look identical to a user and lead to completely different behavior: one is worth waiting for, the other is worth abandoning.
Give the wait a shape. A position in line, an estimated time, or a plain Unusually busy, try again in a minute all beat a generic error.
Protect work already in progress ahead of work not yet started. Someone halfway through checkout losing their session hurts far more than someone being asked to wait at the door.
Say which capability is limited rather than blocking the whole product. Exports being paused is tolerable, and a product that appears entirely down is not.
Do not encourage the behavior that makes it worse. Under load, remove the aggressive auto-retry, extend the polling interval, and make the manual retry deliberate rather than inviting.
Keep the queue honest. If a request is going to be dropped, drop it visibly rather than accepting it and quietly discarding it, because the user will wait for a result that will never arrive.
Prepare the copy in advance. Load shedding messages get written during an incident by whoever is closest, and that is how a product ends up apologizing for a bug it does not have.

Turning work away is a deliberate act of engineering, and the interface should describe it as a busy service rather than a failing one.