#1557

IMAP: persistent per-account folder exclusion, and optional deletion reconcile under a folder filter

Openqtld88opened 2d ago

Use case

I archive an IMAP mailbox with msgvault serve and want two things at once:

  1. Default syncs skip Trash and Junk.
  2. A message I move to Trash after it was synced eventually drops out of the active archive (marked source-deleted, then removed by gc).

Today I can get the first or the second, not both.

What I found

Checked against v0.21.0, v0.22.0 and main at 89354fab. The code paths below are the same in all three.

  1. Folder flags last for one command. docs/usage/imap.md says "Folder flags apply to one command invocation". A [[accounts]] entry only has email, schedule and enabled (AccountSchedule), and runScheduledIMAPSync adds no folder filter. The only persistent filter is the folders allow list in the source's stored IMAP config (Config.Folders). Scheduled syncs do read it, but no command sets it (add-imap writes the config without it), and there is no exclude counterpart. My workaround is a host cron that runs msgvault sync <id> --skip-folder Trash --skip-folder Junk.
  2. A filtered run never tombstones. labelsSnapshotFilteredLocked() in internal/imap/client.go is true when a date filter, Config.Folders, --folder or --skip-folder is set. The client then builds no mailbox deltas, and saveIMAPFolderStates returns before ApplyIMAPMailboxDeltasForSyncContext. Memberships, folder state and tombstones are never written. A message moved from INBOX to the skipped Trash folder keeps deleted_from_source_at NULL, so gc has nothing to purge. On my archive of about 1,000 messages, no row has the marker set.
  3. An unfiltered scan reconciles, but does not solve this either. A message moved to Trash gets a Trash membership, so it is tombstoned only once the server expunges it from Trash. Trash and Junk content is archived in the meantime.
  4. Publishing deltas for the scanned mailboxes only is not safe with the current store function. ApplyIMAPMailboxDeltas treats the delta set as the complete topology: retireAbsentIMAPMailboxes drops the memberships of every mailbox without a delta, and captureUntrackedIMAPMessageIDs tombstones live messages that have no membership left. A partial delta set would tombstone everything in the skipped folders.

Proposal

Two independent pieces. I can send them as separate PRs.

A. Persistent exclusion. Add skip_folders next to the existing folders allow list in the source config, give it a supported way to be set (an add-imap flag or a config key), and have scheduled syncs honor it. CLI --folder and --skip-folder keep overriding it for one run.

B. Opt-in deletion reconcile for filtered scans, off by default. A separate store path that applies deltas only to the mailboxes the run scanned, leaves stored memberships of other mailboxes alone, and tombstones a message only when no membership remains anywhere.

One consequence of B: msgvault never sees a skipped folder, so it holds no membership there. A message moved into any skipped folder would be tombstoned, and gc would then purge it. That is the behavior I want for Trash and Junk, but not for a skipped folder such as Archive/Newsletters. B may need to be limited to folders the user marks as "deleted" (or to \Trash and \Junk roles), rather than to every skipped folder.

Questions

  • Is A the right shape? Should the exclude list live in the source's stored config, like folders, or in config.toml?
  • Is B acceptable in principle, and if so, should it apply to all skipped folders or only to folders marked as deleted? I want to test the failure cases (UIDVALIDITY change, interrupted run, a message present in both a scanned and a skipped folder) before writing code.
  • #1513 (open) widens Gmail full sync to include Spam and Trash, to match complete presence reconciliation. This request goes the other way for IMAP. If B conflicts with that direction or with the archive-first approach, is A alone still welcome?
  • Related: #1341 (open) refuses IMAP archive moves into a folder outside the account's effective folder filter. A persistent exclude list would likely interact with that check.

Once you tell me the direction, I will open a draft PR for A linked to this issue.

Activity

0

No activity yet.