Contribution Date
Contribution Project
Contribution Details
An upload that never becomes an asset, and the outage that does not wave things through
Go-live item 5. M1 listed a virus scan; nothing existed.
## Talking to clamd rather than shelling out
`dam_media::antivirus` speaks `clamd`'s `INSTREAM` over a socket. The protocol is four
lines of framing — `zINSTREAM\0`, length-prefixed chunks, a zero terminator, one reply —
which is why there is no dependency for it.
`clamscan` was the alternative and it is worse twice over: it reloads the entire signature
database on every invocation, seconds of CPU and hundreds of megabytes per file, and it
takes a path — meaning the bytes would have to be written to a shared filesystem before we
have decided to accept them.
## Scanned before promotion
Infected bytes never reach a content-addressed key and never become an asset. Quarantining
the asset row afterwards was the alternative, and it leaves the object sitting in the
library's own namespace while turning "is this asset safe" into a question about a column.
The already-promoted retry path skips the scan deliberately: those bytes were scanned by the
attempt that promoted them, and re-reading a 200 GB master to re-scan it on every transient
failure would make an outage expensive.
## Three outcomes, three kinds of failure
**Infected** is permanent, with the signature in the log — a signature does not change on a
retry.
**Unreachable** is transient. The upload waits in staging and finalises when `clamd`
returns. This is fail-closed, and the recovery is what makes that acceptable rather than
merely strict: verified by pointing the worker at a dead port, watching the job re-queue
with no asset created, then pointing it back and watching that same upload finalise itself.
A configurable fail-open was the alternative and it is a footgun — the setting that lets
ingest continue during an outage is the setting still switched on a year later.
**Unintelligible** is permanent and deliberately not read as clean. A parser that fell
through to `Clean` would turn any future protocol change into a silent bypass of the only
thing standing between an upload and the library, so `parse` errors on anything it does not
recognise and there is a case asserting that for four different replies.
## Large files are not scanned, and this says so
Past `security.max_scan_bytes` — 100 MB by default, matching `clamd`'s own `StreamMaxLength`
— an upload is **accepted unscanned** with a warning naming the size. A DAM whose purpose is
video masters and layered PSDs cannot refuse every large file.
That is a real hole and `docker/DEPLOY.md` states it in those words rather than implying
coverage that does not exist. What actually protects those files is elsewhere: nothing is
executed, derivatives render in a sandbox with the environment cleared, and delivery is a
signed redirect rather than a served file.
## Verified against a real ClamAV, not only a fake
Eight unit cases, including a fake `clamd` that asserts the framing from the server's side —
the zero-length terminator is the mistake that presents as "the scanner is down", so it is
checked rather than assumed, with a payload larger than one chunk to exercise the loop.
Then the real thing (arm64 needed `--platform linux/amd64`; there is no native image): EICAR
refused permanently as `Eicar-Test-Signature` with no asset created, a clean PNG through to
three derivatives, and the outage-and-recovery sequence above.
Off by default, because requiring `clamd` on every developer machine to accept an upload
would make the dev stack a three-container affair and a security control that blocks the
inner loop is one somebody disables permanently. The cost is that a deployment which never
sets it never scans, so it is listed as required in `docker/DEPLOY.md` and the worker logs
`virus scanning enabled` at startup — the line to check after a deploy.
Not built: re-scanning existing assets when signatures update. A scan happens once, at
ingest.
Contribution Issue Link
Files count
0
Patches count
1