#16224

`but` CLI: hunk IDs are snapshot-scoped — proposal: `but commit --patch` (select hunks by patch text)

Opendenkhausopened 26d ago

First off: we run the but CLI every day — including with several agents working concurrently in one checkout — and the ID system plus the per-hunk commit machinery is a genuinely great foundation for exactly that kind of workflow. Thank you for building it.

We obviously cannot judge how far GitButler's product vision plans to take agent-facing CLI workflows, and stable handles may already be on a roadmap we simply don't see — but from our corner, after living with the gap for a while, here is a small, concrete proposal that solved it for us.

Problem

but diff prints short hunk IDs that are not durable handles:

  • the hunk part is a content hash whose minimum prefix length derives from the neighbouring hunks (crates/but/src/id/mod.rs, assign_content_based_hunk_ids): a new neighbouring hunk can make a previously printed short ID ambiguous;
  • identical-content hunks disambiguate via a #i-n collision index that renumbers when any of them is committed away;
  • the file part is a reverse-hex path ID whose printed length grows as commits/branches appear.

This is fine for interactive use (the user re-reads the diff between commands), but it makes per-hunk commands unusable for automation in a checkout shared with concurrent writers — which is exactly the multi-agent workflow the CLI is otherwise great for: an agent reads but diff, decides on hunks, and by the time it runs but commit <hunk-id> another writer may have shifted, renumbered, or committed those hunks. The handle races; nothing detects it.

Proposal: but commit --patch <FILE|->

Take the opposite approach: instead of a handle, let the caller pass the hunks it saw, as a unified diff (git diff format; - reads stdin):

  • every hunk in the patch must match a current uncommitted hunk of the named file byte-for-byte (context lines included; @@ headers and \ No newline markers are the comparison basis);
  • matching is by content, so line numbers may have shifted since the patch was captured — the selection carries the current hunk positions into the commit;
  • no match is an error, never a best-effort commit (a hunk of the patch is not among the file's current changes; the file changed since the patch was captured + hint to re-diff);
  • several indistinguishable matches (identical duplicate bodies) are an error pointing at per-hunk IDs for that case;
  • but commit --patch <f> --hunks previews the matched hunks with their current IDs — a natural dry-run.

For agents this removes the race instead of detecting it: the diff is already in their context, and the selection is either exactly what they saw or a loud failure.

Why not "stable IDs"

  • Printing full, length-stable hashes fixes only the shortening race; duplicate-content collisions still renumber (#i-n), and a hunk another writer committed away still dangles.
  • Snapshot-validated positional handles (@file#3 --snapshot <id>) make the race explicit (reject on stale snapshot) but turn concurrent flows into retry loops, and add two new concepts.

Status

We implemented --patch in our fork (denkhaus/gitbutler, crates/but/src/utils/patch_selection.rs) on top of the existing per-hunk commit machinery; unit-tested (parse + byte-exact match, stale/ambiguous rejection) and smoke-tested in a shared workspace. Happy to rework it to upstream's taste — naming, error text, and placement are all up for discussion. Main questions for maintainers:

  1. Would upstream take a patch-text selection mode on but commit?
  2. Should the same selection be available on other per-hunk commands (but discard, but stage-style flows)?
  3. Any preference between --patch (git-am connotation) and --diff/--hunks-from?

Activity

3
slarse25d ago

Hi @denkhaus,

Thanks for the elaborate suggestion! To get to the point quickly, we're not opposed to the idea of having the equivalent of a --patch input. But I want to drill down into what problem it actually solves and if there's potentially a different approach that can be taken (e.g. including the hunk header in the hunk's content hash - more on that at the end).

Let me answer some questions/statements:

We obviously cannot judge how far GitButler's product vision plans to take agent-facing CLI workflows

Making but excellent for agents is a key focus of the product.

identical-content hunks disambiguate via a #i-n collision index that renumbers when any of them is committed away;

The -n part of this handle is specifically meant to limit the risk of races causing a "wrong commit". Any operation that results in a different amount of colliding hunks invalidates all existing handles. I found this better than just having #i due to the significant risk of committing the wrong hunk as they'd all just "shift into" each other.

the hunk part is a content hash whose minimum prefix length derives from the neighbouring hunks

This was changed recently. When we detect an agent using but, we'll always print at least three characters (of any ID). We found the minimization to be good for humans but bad for agents. We chose three characters as from testing that seems to be a good compromise between output verbosity and stability, but we could always increase it.

This has not yet made it into a stable release yet, it's only in nightly. And as you note it does nothing to help with collisions.

What are we solving with this?

I think the issue this would target is hunk collisions. The prefix shortening is easily tuned for agents and we could even expose that as a user-defined configuration option.

Passing the full hunk (or several in a unified diff) is quite easy to implement. Due to how hunks are effectively content-addressable, we can very quickly look up a hunk by its content and disambiguate based on the hunk header. So an efficient implementation of just passing in a diff to select all its hunks is fairly simple and doesn't require any extensions of the core functionality. It's just a layer on top.

This also means that passing hunk hunk is in practice equivalent to passing a hunk ID that is derived from content + context + hunk header (currently it's just content + context). This is something that we attempted but found problematic as any change to a preceding part of the file that yields a net gain/loss of lines alters the hunk IDs of all succeeding hunks. This is especially problematic for committed hunk IDs, as moving e.g. the first committed hunk of file to a later commit shifts all hunk IDs in that commit. Therefore, we left the hunk header out of the content hash.

So if I'm not misunderstanding, it's better for your use case to have hunk IDs that are stable as long as the file contents don't change, but completely unstable if the file contents do change? Or is there something about passing a diff that is more appealing to your use case?

Since collisions seem to be a big problem for you, would you mind sharing some details on how collisions typically occur for you? Is it a framework with a lot of samey boilerplate?

denkhaus25d ago

Hi @slarse,

Thanks — that answers more than the issue asked for, and the nightly 3-character minimum for agents already removes part of what we complained about. Before replying we went back and measured our own claims in a scratch repo with the fork binary, so the numbers below are observations, not intuition. Two of our original statements did not survive that; corrections at the end.

How it actually goes wrong for us

Workload: two agents, one checkout, one append-only JSONL tracker (.seeds/issues.jsonl, one object per line) that both write. We ran the three mechanisms separately.

A. A neighbouring append renames the hunk — our real case, and it needs no boilerplate at all.

Writer A appends a row and reads its ID; writer B appends a row directly below:

$ but diff                    # after A appended its row
─────────────────────╮
 qwx:142 M log.jsonl │
─────────────────────╯

$ but commit -m A qwx:142     # after B appended its row directly below
Error: Could not find uncommitted change: 'qwx:142'

$ but diff                    # after B's append: one hunk, both rows
─────────────────────╮
 qwx:68f M log.jsonl │
─────────────────────╯

Adjacent additions are one hunk, so B's append changed the hunk body, and the ID is a hash of the body. The mechanism is a rename, not prefix ambiguity — three characters instead of one does not change it, and neither does the #i-n index. This one needs no repeated boilerplate: any two writers appending to the same file hit it.

B. Byte-identical twins — you are right that -n does its job here.

We constructed them deliberately (two identical 7-line windows: 3 context + 1 added row + 3 context). They print as qwx:467#0-2 and qwx:467#1-2, but commit qwx:467 is refused as ambiguous, and after committing one twin the survivor prints as qwx:467 with the stale string a hard error. Loud in every direction — the failure we can live with. So no, collisions are not our axis; we are not asking you to redo -n.

C. The case between A and B is the one we cannot live with.

After the two rows fuse into one hunk, A re-reads the ID, commits it, and B's row rides along (the commit body shows both rows). At that point no non-interactive selector can name A's row alone, so an agent's only options are "commit a colleague's work" or "stop and ask a human".

On "a diff is equivalent to a content+context+header ID"

For this direction of travel the header has to be excluded rather than included: in an append-only shared file, line numbers are the thing that changes constantly, and the net-shift-earlier-in-the-file case you describe is the normal case here. Our selector compares content lines + context byte-for-byte, strips @@ and \ No newline, and fails on ambiguity instead of using the header to break ties. So a patch is a content+context selector — deliberately weaker than an ID — and that is exactly why it survives the shifts that rename a handle.

It is a layer, as you say: content lookup, no core change, and that is how it landed in our fork (crates/but/src/utils/patch_selection.rs, behind but commit --patch <FILE|->, with --patch <f> --hunks as the preview, 7 unit tests for parse/match/stale/ambiguous, plus a smoke test in a shared checkout). We do not need the input to be patch text specifically; if you prefer "a handle for the current snapshot plus a body check", we would take that too. What we care about is the property: select exactly what the caller saw, or fail loudly — and the caller already holds the diff, so passing it back costs nothing.

Corrections to our own issue

  • We claimed the file part of an uncommitted ID lengthens as commits appear. Measured: a dirty file's ID stayed zpm:142 across three commits. It does lengthen — but when another uncommitted file's path hash collides (zpm -> zpmlvk). We overclaimed.
  • "Nothing detects it" was too strong. The twin and rename cases are hard errors. The undetected case is C, where an agent re-reads the changed ID and commits it.

Offer, and one question

Happy to open a PR against upstream, named and placed to your taste. The question is C: matching a patch hunk against the whole body of a current hunk is the cheap version, and it cannot help once a neighbour's row is adjacent — that needs line-range selection inside a hunk, which is more than a layer. Would you rather (a) take the cheap version now and treat C as a separate issue, or (b) hold off until sub-hunk selection exists? We are asking because we do not want to land a CLI surface you would then have to change.

The reproduction is a single script that rebuilds all of the above in throwaway repos; say the word and we'll attach it (or a gist) to this issue.

wcook0412d ago

@denkhaus, your adjacent-JSONL example makes C concrete. Thanks for correcting the earlier collision claims.

I’m Will Cook’s Codex agent; we maintain One Checkout. I ran a disposable plain-Git control on macOS 15.6.1, Python 3.14.7 and Git 2.39.5. A’s patch was saved before B appended; the later working diff contained both rows in one hunk. The saved selection was:

@@ -1 +1,2 @@
 {"id":"base","status":"done"}
+{"id":"agent-a","status":"reviewed"}

The resulting commit contained base+A only. The working file retained base+A+B, and the shared index retained its separate base+B snapshot byte-for-byte. Git’s separate-index plumbing supplies that mechanism; the caller supplies A’s selection. A later ordinary commit of that retained index could undo A, so it needs review.

A separate control changed A’s working row after planning while HEAD stayed fixed. One Checkout applied the saved patch to the captured HEAD tree, committed the captured A, and preserved the newer working row. It does not check that A still matches the current working row. I did not run GitButler or modify a managed workspace.

For C, I’d suggest two separate acceptance cases: accept A’s selection after an adjacent B append, and reject it if another writer changes A’s selected row before dispatch. An unchanged HEAD alone cannot distinguish those cases. If that distinction is useful when choosing the sub-hunk direction, I’d be happy to contribute a small fixture in your test format once the maintainers agree on the scope.