Outgoing Webhooks: Hard Because the Failure Happens on Someone Else's Server

Signing, retries, ordering, and the slow receiver — why sending an HTTP request looks easy until you send it to a thousand customers.

Receiving a webhook is easy. Sending one is hard — because you depend on a server you do not own, which may be slow, down, or return 200 and drop the message. From your customer's point of view, all of that is your problem.

Sign every payload

The receiver needs to verify the request really came from you. Send an HMAC signature over the raw body along with a timestamp header, and document how it is computed. Include the timestamp in the signed content, or an old message can be replayed with its valid signature. Tell receivers to reject anything outside a short window.

Assume duplicates and say so

Exactly-once delivery is not practically achievable across a network. Send a unique id per event and document explicitly that receivers must ignore what they have already processed. That is not a concession, it is a clear contract — the alternative is a customer who reads duplicates as a bug in your system.

Retries and final failure

1. Retry with increasing gaps across hours, not minutes — most outages are short but not seconds long.
2. Treat only 2xx as success, 4xx as permanent failure not worth retrying, and 5xx as transient.
3. Put a short timeout on the response; one slow receiver must not stall the whole queue.
4. Automatically disable an endpoint after sustained failure and notify the customer — delivering to a dead URL for weeks is pure waste.

A customer endpoint that has been broken for a month should not cost you a delivery attempt every minute forever.

Ordering is not guaranteed — say that too

An "updated" event can arrive before "created" because of a retry. Attach a timestamp and a sequence number, and document that receivers should rely on those rather than arrival order. Strict ordering guarantees are expensive and most cases do not need them.

Give customers the diagnostics

Log every delivery attempt with response code and duration, and show it to the customer. Add a manual resend button and a test tool that sends a sample event. Those three eliminate most integration support tickets, because they turn "it doesn't work" into information.

Do not invent your own signing scheme: Standard Webhooks is an open specification covering signatures, timestamps and replay protection, and RFC 9421 HTTP Message Signatures is the formal standard for anyone who needs something broader.

If you are building a webhook system into your own product, signing and replay protection are what get forgotten. Talk to us.

FAQ

Full payload or just an id?

Send enough to act on without a follow-up call, and keep sensitive fields out. An id alone doubles the load on your own API.

How do I handle secret rotation?

Allow two active secrets at once for a window, so customers can switch without downtime.

What if a customer wants multiple endpoints?

Support them with per-endpoint event selection; sending everything everywhere wastes both sides' resources.

النسخة العربية