Contribution Date
Contribution Project
Contribution Details
A dispatcher that does not hold a connection through somebody else's timeout
The loop between the outbox and the sender, plus the administration surface for
registering an endpoint.
The dispatcher's shape is decided by one constraint: a delivery waits up to ten
seconds, and holding a pooled connection across that would let a handful of slow
endpoints exhaust the pool and take the API down with them. So each attempt is
three short transactions with the HTTP in between, and the `delivering` state in
the table stands in for the lock deliberately not being held. Reclaim runs first,
because a row stuck in `delivering` blocks every later event for its asset — a
worker that died mid-delivery silently halts one asset's stream, and the halt
would outlast the process without it.
Concurrency needs no semaphore. claim() already refuses to hand out two
deliveries for one asset, so a batch sent concurrently is concurrent across assets
and endpoints, which is exactly where it is safe. The ordering rule is doing the
accounting.
The chain drains before it sleeps: a pass that filled its batch queues the next
immediately, because a bulk publication of ten thousand assets should not take ten
thousand poll intervals to go out. And it starts when a subscription is created
rather than at boot, so a deployment where nobody has registered a webhook runs no
dispatch at all — which is most deployments.
The subscription URL is an SSRF vector, and that is the control this API is mostly
about. It is a server-side POST to an address the tenant chose, so without
validation a tenant registers http://169.254.169.254/ and reads cloud instance
credentials out of their own delivery log. Loopback, private, link-local,
unspecified and credential-bearing URLs are refused; https is required outside
development, where a receiver on localhost is how anybody develops against this.
The guard's real limit is in its own doc comment rather than left to be
discovered: it is host-based, so a hostname that resolves to a private address
still passes, and closing that means checking the address actually connected to at
send time.
The secret is returned by POST and never again — a receiver cannot verify anything
without it, and returning it on every read would put it in the response of an
endpoint an integration polls. The response that does carry it explains the scheme,
including why the timestamp matters, so a customer does not have to go looking.
Two more: the delivery log carries no payloads, because it is the largest column
on the query a screen runs most often and returning them would make this the
cheapest way to read a tenant's whole change history. And retry takes only a dead
letter, and only one belonging to the subscription in the path — reviving
something in flight would break the per-asset ordering, and an unchecked path
segment would let a guessed id confirm a delivery exists.
One thing I caught in review rather than in a test: the worker's client builder had
unwrap_or_default(), which would have silently fallen back to a client that
follows redirects — undoing the one security property the builder exists to set. It
is a startup failure now.
8 API cases, 15 of them refused URLs.
Contribution Issue Link
Files count
0
Patches count
1