Half a library invisible to search, because derive never emptied

Contribution Date
Contribution Project
Contribution Details
Half a library invisible to search, because derive never emptied Strict priority is a queue until the urgent band stops draining. A load run ingested 2,637 assets across five tenants; every upload fans out into `derive` jobs at priority 40, so that band was never empty, and behind it 1,280 `index` and 1,280 `similarity` jobs sat at `attempts = 0` with `run_after` half an hour in the past. Never claimed once. `load-0078.jpg` was in the library and a text search for it returned nothing. `claim` already round-robins across tenants so one tenant cannot starve another. It had no equivalent for the other axis that starves, so a quarter of every batch is now reserved for the background band — pooled and oldest-first, not a slot per band. The worker claims four at a time, so there is no batch size at which every band gets its own slot: reserving per band would leave interactive work one slot in four and still starve the lowest bands. Pooling bounds the wait instead of guaranteeing a slot, because a band's head advances every time it is picked. Aging the background band into the interactive one would also have worked, by contradicting the boundary `JobSpec::priority` documents. This keeps that contract: interactive work takes every slot the reserve does not, and the reserve is empty whenever the background bands are. Never the last slot either — at `limit = 1` there is no reserve, so a single-job claim is still decided by priority alone. Verified on the running system: zero background jobs completed in thirty-five minutes before, eight in the eight minutes after.
Contribution Author
Bassam Ismail
Files count
0
Patches count
1