#29496
Issues/vlang/v/#29496

-d ownership cannot compile cmd/v: ~28 GB arena, trips the 9984 MiB guard

Openmetif12opened 25d ago

-d ownership cannot compile cmd/v at master (1f69b62954)

Ownership checking drives the prealloc arena to ~28 GB live and trips the 9984 MiB memory guard. The control build succeeds.

Environment: Windows 11 x64, 16 logical CPUs, 15.4 GB RAM, bundled tcc. Quiet machine, 7.7-9.7 GB free, no other V processes running. Private TMPDIR.

# builds fine
v -cc tcc -o v_control cmd/v          # exit 0, 30s

# ownership-enabled compiler, then compile cmd/v with the ownership define
v -d ownership -o v_own cmd/v         # fails every time

Three failure shapes, all after the ck forks stage:

  1. error: v3 compiler memory usage reached 9984 MiB RSS during compilation (limit: 9984 MiB) -- then the process dies with an access violation, exit 0xC0000005.
  2. could not allocate 16777216 bytes, exit 1.
  3. With -no-memory-limit, so the guard is out of the picture, it still fails the same way, after peaking at ~8.5 GB RSS.

Instrumenting the arena counters (vmemory_block_new_sized / vmemory_block_free in vlib/builtin/prealloc.c.v) shows where it goes:

alloc-fail want_MB=16 blocks=2152 cumulative_MB=29236 live_MB=28667

~28 GB live across 2152 blocks. So this is not host memory pressure, not a shared build dir, and not the C toolchain -- it dies before any C compilation.

For contrast, -d ownership on the bare frontend (vlib/v/v.v) succeeds in ~35s at ~1.1 GB, so it is size-dependent rather than a blanket failure.

Two things look worth separating:

  • The memory itself. The prescans converge quickly (9 and 7 rounds over ~15,000 functions), so the fixpoint is not spinning. OwnershipState holds ~50 string-keyed maps, and OwnershipFrame clones 8 of them on every function and fn-literal entry (checker_ownership_d_ownership.v:4982-4990 and :5048-5056), with per-branch snapshots retained in branch_groups. Plausible source of size-dependent growth, though I have not isolated it.

  • The guard does not exit cleanly. In case 1 it prints the diagnostic and the process access-violates instead of returning. That looks like a separate defect worth fixing regardless of the memory question.

This is also why other_ci.yml has the -autofree cmd/v step commented out (#27116) and why the ownership/autofree CI steps are all if: ${{ false }}.

Activity

2
metif1225d ago

Correction to my own report above: the "success" contrast I gave was wrong.

I wrote that -d ownership on the bare frontend (vlib/v/v.v) "succeeds in ~35s at ~1.1 GB", implying the problem is size-dependent rather than a blanket failure. That measurement came from a compiler built without the ownership checker compiled in — it answers ownership support is not compiled into this v3 executable when asked to check anything, and the driver gate lets -d ownership through for compiler-entry inputs so that a plain compiler can bootstrap-build an ownership one.

So the target got the ownership define and the _d_ownership.v file selection, but no ownership analysis ran. The 35s / 1.1 GB figure is a no-op, not a result. From a compiler that actually has the checker, the same frontend target also needs ~28 GB.

What that changes:

  • The failure is not size-dependent between vlib/v/v.v and cmd/v; both exhaust the arena when checked by a real ownership compiler (~14.6k and ~15.1k functions respectively).
  • The useful contrast is not frontend-vs-driver but checker-present-vs-absent.

The rest of the report stands: the control build succeeds in 30s, the three failure shapes reproduce on a quiet box, the arena counters show ~28 GB live, and the guard prints its diagnostic and then the process access-violates instead of returning.

metif1225d ago

Two corrections to this report.

Scope. Everything here is Windows-only evidence: Windows 11 x64, 16 logical CPUs, 15.4 GB RAM, bundled tcc. I have not reproduced any of it on Linux. The memory guard is RSS-based (bench.v:13 samples current_rss_kb()), and the guard constant differs by build kind (10176 MiB user, 9984 MiB compiler-tree and self-host, bench.v:10-12), so how much headroom a build needs is likely platform-dependent. If the failure turns out not to reproduce on Linux, the useful part is the mechanism, not the bug.

Overlap with #27116. That issue already reports that -autofree "produces a compiler that cannot reliably self-compile or compile non-trivial programs". My report is a narrower, sharper slice of the same area: the build fails outright at ~28 GB of arena rather than producing a compiler that then misbehaves. That difference may itself be the news, or it may be that #27116 predates the 45 ownership commits since it was opened.

I'd also have been better off reading #27116 before filing. The three autofree bugs listed there — shared sub-buffers, non-owning pointer locals, sumtype payload double-frees — describe the codegen consequences of the same missing analysis, so "ownership genuinely needs escape analysis" was already on the table.