MCPbundler
< All Posts
Engineering

Webhooks that fail loud, not quiet

MCP Team-August 9, 2026-3 min read

Webhook docs across the industry tend to nail one thing - "here's how to verify the signature" - and go quiet on the part that actually costs you debugging time later: what happens when your endpoint is down for ten minutes, or returns a 400 because you shipped a bug, or times out under load. We wanted ours to be boring in exactly that way - fully specified, nothing left as "it'll probably retry a few times."

So, precisely:

  • The retry schedule is five attempts: 1s, 5s, 30s, 2 minutes, 10 minutes. Not "a few retries with backoff" - that schedule, every time, for every failed delivery.
  • We only retry what's actually retryable. 429 and 5xx responses, plus timeouts and connection failures, get retried on that schedule. Any other 4xx doesn't retry at all - it's your endpoint telling us it's misconfigured, and hammering it five more times on a fixed schedule wouldn't fix that, it'd just add noise to your logs.
  • Signing follows GitHub's convention on purpose. X-Hub-Signature-256: sha256=<hmac> computed over timestamp + "." + body, with the timestamp shipped alongside in its own header so a signature can't be replayed against a different payload later. If you've verified a GitHub webhook before, you already know how to verify ours - we didn't invent a new scheme just to be different.
  • The signing secret is shown to you exactly once. Plaintext, at creation, never again. That's not us being precious about security theater - it's the only way "the secret only ever existed in your environment and our encrypted storage" stays true.
  • Test pings are real requests, not a lie. Hitting "send test ping" dispatches an actual signed HTTP call to your endpoint synchronously and hands you back the real response - not a simulated 200 that tells you nothing about whether your receiver actually parses the payload correctly.

The unglamorous parts mattered as much as the API surface: dispatch is pinned to a pre-validated IP so a target can't be pointed at an internal service after it passes the initial check, and delivery is deduplicated against Kafka redelivery so a rebalance can't double-fire the same event at your endpoint.

None of this is exotic. It's just the difference between a webhook system you can build reliable automation on top of, and one where "did that actually get retried?" is a support ticket instead of something you can answer yourself by reading the delivery history.