#1559

Slack media backfill: 2,748 failed PDF fetches record no reason and are never retried

OpenJoiopened 2d ago

msgvault v0.22.0, macOS, SQLite archive with Slack sources from several workspaces.

After the first scheduled backfill-slack-media run I counted the Slack attachment rows. 10,483 are stored. 9,153 are skipped (9,111 participant_threshold, 42 size_cap). 2,817 are attachment_state=failed with attachment_skip_reason=fetch_failure, and 2,748 of those are application/pdf; the rest are 40 text/plain, 14 png, 10 jpeg and 5 svg. The failed PDFs are 186 KB to 1.8 MB, under the size cap, and images from the same workspaces download fine.

Two things make this hard to work with:

  1. No reason is recorded. The row says only fetch_failure, sync_runs.error_message is empty, and the sync summary counts each one as an error (one workspace reports 222 errors for 222 pending media) without saying what failed.
  2. Failed rows are never retried. backfill-slack-media prints "0 downloaded, N still pending" per workspace within seconds, and a verbose run on a workspace with 3 pending made no HTTP request, so fetch_failure rows are not in the eligible set.

Two asks:

  1. Record why a fetch failed (HTTP status, content-detector verdict, or the exception) on the attachment row and in the sync summary.
  2. Let backfill-slack-media retry fetch_failure rows at least once, or add a flag that does.

My guess at the cause, since the failures are almost exactly the PDF set: the downloaded bytes are rejected by the content detector rather than by transport, the same shape as kenn-io/docbank#426 (a PDF without a final %%EOF). I can run a build against this archive and report what the new reason field says.

Activity

0

No activity yet.