Stop the sniffer panicking on binary content

Contribution Date
Contribution Project
Contribution Details
Stop the sniffer panicking on binary content The SVG search truncated the lowercased head at byte 1024. That head comes through from_utf8_lossy, which replaces every invalid sequence with a three-byte replacement character, so on binary input byte 1024 is very often inside one — and slicing a &str there panics. Reachable from any upload of a file that is not text, which is most of them. Whether it fired depended on where the replacement characters happened to fall, so it was a panic that showed up on some binary files and not others. In the worker it meant a dead process with a panic message where a sniff verdict belonged, which is the failure dam-pipeline denies expect_used to avoid. Truncate to the nearest char boundary at or below 1024 instead. Found by transferring a nine-megabyte binary through the migration path. The regression test sweeps the starting offset rather than picking one blob, because which byte lands on the boundary depends on the content and a single input would pass over a fix that was still wrong by one.
Contribution Author
Bassam Ismail
Files count
0
Patches count
1