An Operation That Takes Too Long to Do Synchronously Should Not Be Synchronous
Key takeaway: Synchronous processing is the right default for fast operations, and it becomes the wrong choice once an operation is slow enough that holding the request open for its duration is a problem — at that point the operation should be accepted, acknowledged immediately, and processed asynchronously, with the result made available later.
When Synchronous Processing Becomes a Problem
A request that triggers a slow operation — generating a large report, processing a video, sending a batch of emails, running a long computation — holds the client’s connection open for the entire duration if done synchronously. The client waits, the server holds a thread or connection for the whole operation, and if the operation takes longer than the client’s timeout, the client gives up and the work may be lost or duplicated.
This is a problem at any scale where slow operations are common, because holding connections and threads open for long operations consumes server resources that could be serving other requests, and it makes the API’s latency unpredictable — a single slow operation can tie up resources and degrade the experience for unrelated requests.
The Asynchronous Pattern
The asynchronous pattern accepts the request, returns an immediate acknowledgement with an identifier for the operation, and processes the work in the background — a queue or background worker picks up the job and executes it, and the client later checks the status or receives a notification when it is complete.
| Step | What happens |
|---|---|
| Client submits request | Server accepts it, returns an operation ID immediately |
| Background processing | A worker executes the operation asynchronously |
| Client checks status | Client polls or receives a notification when done |
| Client retrieves result | Result is available once the operation completes |
This decouples the client’s request from the duration of the work — the client gets an immediate response, the server does not hold a connection open for the whole operation, and the work proceeds in the background regardless of whether the client is still waiting.
The Trade-Offs
The cost of the asynchronous pattern is added complexity — the client now has to handle a two-step flow of submitting and then retrieving, the server needs a queue and workers, and the system needs to handle the state of in-flight operations, including failures and retries of the background work.
The benefit is that slow operations no longer tie up request resources, the API’s latency stays predictable and fast, and the background work can be retried and monitored independently of the request that triggered it. The decision of whether to go asynchronous is essentially a decision about whether the operation’s duration makes the synchronous approach’s costs unacceptable.
Deciding Where the Threshold Is
There is no universal threshold, but the guiding question is whether the operation’s duration is short enough that holding the request open is acceptable — if an operation routinely takes longer than a client is willing to wait, or longer than the server can afford to hold resources, it should be asynchronous. Operations that are fast and infrequent can stay synchronous, where the simplicity of a direct request-response is a genuine advantage.
The Bottom Line
Make operations asynchronous once they are slow enough that holding the request open is a problem — accept the request, return an immediate acknowledgement with an operation ID, process the work in the background, and let the client check status or receive a notification when it is done. Accept the added complexity of the two-step flow and background processing as the price of keeping the API’s latency predictable and not tying up request resources on slow work.
