Backend & APIs

A Webhook With No Retry Policy Is a Notification You Will Never Receive

Key takeaway: A webhook is only as reliable as the delivery mechanism behind it, and without a retry policy with backoff, a receiver that is temporarily unavailable or slow will permanently miss events, silently, with no way for either side to know what was lost.

Why a Single Delivery Attempt Is Not Enough

A webhook sends an HTTP request to a receiver’s endpoint when an event occurs. If the receiver is briefly down, restarting, or slow at that exact moment, the request fails or times out, and if the sender makes only a single attempt, the event is lost permanently — the receiver never learns it happened, and neither side has any record that it was missed.

This is not an unusual scenario; it is the normal reality of operating any service, where brief unavailability and latency spikes are routine. A webhook system that does not account for this is not occasionally unreliable — it is reliably unreliable, losing a steady background rate of events that nobody notices until a specific important event turns out to have been missed.

What a Retry Policy Actually Provides

A retry policy re-attempts delivery of a failed webhook after a delay, typically with exponential backoff — retrying after a short interval, then a longer one, then longer still — which gives a briefly unavailable receiver time to recover while not hammering it with immediate retries that would fail again and add load while it is still struggling.

Attempt Delay before retry
First failure Short delay (e.g. seconds)
Second failure Longer delay (e.g. minutes)
Subsequent failures Increasingly long delays
After max attempts Event marked failed, needs manual or out-of-band handling

The backoff is important because immediate retries against a receiver that is still down or slow will simply fail again, wasting resources and potentially worsening the receiver’s problem by adding load during its recovery — a well-designed backoff gives the receiver time to recover before the next attempt.

The Retry Limit and What Happens After It

Retries cannot continue forever, so a webhook system needs a defined maximum number of attempts, after which the event is marked as failed and handled out of band — typically by making it visible in a dashboard or log where a human or an automated process can investigate and re-deliver it manually, rather than silently giving up.

This out-of-band handling is what turns a webhook system from one that silently loses events into one that surfaces them for resolution, and it is a critical part of the design that is easy to overlook — a retry policy that eventually gives up but does nothing visible with the failed events is only slightly better than no retry policy at all.

The Receiver’s Side of the Contract

The receiver also has responsibilities — it should respond quickly to acknowledge receipt, and it should handle the fact that a retry may deliver the same event more than once, since a delivery that succeeded but whose acknowledgement was lost will be retried, meaning the receiver must be able to process the same event idempotently rather than applying it twice.

The Bottom Line

Implement a retry policy with exponential backoff for webhook delivery, since a single attempt will silently lose events whenever the receiver is briefly unavailable or slow. Define a maximum number of attempts and surface events that exhaust their retries for out-of-band handling rather than silently dropping them, and require receivers to process deliveries idempotently, since retries can deliver the same event more than once.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button