#318

Bug audit: 18 reproduced findings (darwin-arm64 wrong-code, borrow soundness, parser, tokenizer, stdlib)

OpenPeterXMRopened 5mo ago

Summary

This report bundles 18 reproducible bugs found across the tokenizer, parser, type checker, borrow checker, direct backends, and source-backed stdlib. Each finding is reproduced on the current main (commit f1a5cc9), each file:line citation has been verified, and no finding duplicates an existing issue, PR, or commit. Severity ranges from wrong-code emission on darwin-arm64 (the default macOS host backend) through borrow-checker soundness gaps to documentation drift.

Methodology: six parallel investigations across disjoint domains, then four independent verification passes that re-reproduced each finding from scratch and cross-referenced against open issues, PRs, and git log. 18 of 18 verified CONFIRMED + UNFILED.

Bundling note

I'm filing this as a single audit rather than 18 separate issues to avoid flooding the tracker. Happy to split into per-domain or per-finding issues if that's easier to triage — let me know. I'm planning to send focused PRs for the highest-severity items (S1, S2) unless you'd prefer to fix them directly.


Tier S — soundness / wrong-code

S1. i64 comparison emits 32-bit CMP w on darwin-arm64

  • File: native/zero-c/src/emit_macho64.c:584
  • Reference (correct) impl: native/zero-c/src/aarch64_direct.c:416-419 chooses cmp_x vs cmp_w via a64_type_is_scalar64(value->left->type). The Mach-O direct emitter calls z_aarch64_emit_cmp_w unconditionally.
  • Repro (/tmp/s1.0):
fn shift32 i64 a i64
  ret + a 4294967296

pub fn main Void world World !
  let big i64 shift32(0)
  if == big 0
    check world.out.write "BUG: 2^32 == 0\n"
  if > big 0
    check world.out.write "ok positive\n"
  else
    check world.out.write "BUG: not > 0\n"
  • Expected: ok positive. Actual on darwin-arm64: BUG: 2^32 == 0 then BUG: not > 0 — every i64/u64 compare on darwin-arm64 compares only the low 32 bits.
  • Disasm proof: every comparison site in _main is cmp w8, w9 (encoding 6b09011f) where it should be cmp x8, x9.
  • Fix sketch: mirror the aarch64_direct.c:416-419 pattern at the macho call site — check a64_type_is_scalar64 and emit cmp_x when wide.
  • Distinct from #230 (which catalogs missing IR ops, not wrong-code in existing ops).

S2. macho-arm64 oversized stack frame silently truncated → segfault

  • File: native/zero-c/src/buildability_value_targets.c:6 — z_build_backend_is_aarch64_direct returns true for Z_DIRECT_BACKEND_ELF_AARCH64 and Z_DIRECT_BACKEND_COFF_AARCH64 but omits Z_DIRECT_BACKEND_MACHO64.
  • Consequence: buildability.c:181 routes ELF/COFF aarch64 functions to z_build_check_aarch64_function_shape (which gates frame_bytes > 4095 at buildability_value_targets.c:304). darwin-arm64 falls through to the x86-style shape check with no frame-size limit. The emitter at aarch64_emit.c:129-135 then masks sub sp, sp, #imm to 12 bits (imm & 0xfffu) with no overflow check.
  • Repro (/tmp/s2.0):
pub fn main Void world World !
  mut arr [1024]i32 [0_i32;1024]
  set arr[0] 1
  set arr[1023] 42
  if == arr[1023] 42
    check world.out.write "ok\n"
  • Expected: BLD004 at build time (matching ELF/COFF) or a correctly encoded prologue (LSL #12 or temp reg).
  • Actual: builds, prints ok, exits 139 (SIGSEGV). Boundary observed: n=956 (frame 4080) works (sub sp, #0xff0); n=957+ (frame ≥4096) silently truncates and segfaults.
  • Fix sketch: add || backend == Z_DIRECT_BACKEND_MACHO64 to z_build_backend_is_aarch64_direct.
  • Suggested test: mirror the existing conformance/native/fail/macho-large-byte-slice-blocked.0 shape.

S3. Dynamic-index array set arr[i] clears ALL borrow tracking (soundness)

  • File: native/zero-c/src/checker.c:189-205 — origin_path_array_segment_definitely_within. Line 197's wildcard-prefix short-circuit incorrectly returns true when the prefix is wildcard.
  • Repro (/tmp/s3.0):
pub fn main Void
  mut x i32 1
  let i usize 0
  mut arr [2]ref<i32> [(&x), (&x)]
  let y i32 2
  set arr[i] &y
  set x 99_i32
  • Expected: BOR001 on set x 99_i32 — with non-constant i, set arr[i] &y overwrites at most one of the two &x borrows; the other still borrows x.
  • Actual: zero check reports ok. The dynamic-index target gets path [*], then scope_replace_value_provenance_path_at (line 743) clears every concrete entry ([0], [1]) because the wildcard prefix is treated as definitely-within concrete indices.
  • Control: the same program with a constant index set arr[0] &y correctly raises BOR001 — only the dynamic-index path is unsound.
  • Asymmetry: origin_path_array_segment_match (line 172) correctly accepts wildcards for intersection checks. Only _definitely_within (containment) is unsound.
  • Fix direction: in _definitely_within, a wildcard prefix should NOT be treated as definitely covering concrete indices — dynamic-index writes downgrade to "maybe overwrites" (keep concrete borrows live).
  • Distinct from #31 (closed) which used the old curly-brace syntax and a different code path.

Tier A — high-value correctness

A1. std.math.modPowU32 u32 overflow for modulus > 65536

  • File: std/math.0:45-56 (__zero_std_math_mod_pow_u32)
  • * result current and * current current happen as u32 before % modulus. For modulus > sqrt(2^32) ≈ 65536, intermediates overflow.
  • Repro: mod_pow_u32(65535_u32, 4_u32, 4294967291_u32) returns wrong value (Python reference: 4293394483).
  • Fix sketch: widen intermediates to u64, narrow after %.
  • Related recent fix: commit 7976f0b "Fix std.math unsigned edge cases" touched math but didn't include mod_pow.

A2. Silent decode of unknown string escapes

  • File: native/zero-c/src/row_syntax.c:261 — catch-all else row_append_char(...)
  • Asymmetric with: row_scan_char:318-321 correctly emits PAR100 "unsupported character escape".
  • Repro: let s "\q" tokenizes silently to string "q".
  • Fix sketch: mirror char-scanner — reject unknown escapes with PAR100.

A3. "\0" silently → "0" (string scanner has no \0 case)

  • File: row_syntax.c:239-261 (string) vs :302 (char has escaped == '0' → NUL).
  • Note: \x00 is rejected at :253 ("zero byte"), but bare \0 falls through to the catch-all and silently decodes to literal '0'.
  • Fix: falls out of A2 (catch-all errors), plus decide policy on \0 (likely reject like \x00).

A4. Reserved-word table missing set, alias, interface

  • File: row_syntax.c:727-732 (row_is_reserved_word)
  • Repro: let set i32 0, let alias i32 0, let interface i32 0 all accepted as identifiers, while set is consumed as STMT_ASSIGN at :1296 and alias/interface at :1839/:1843. A binding can be declared with these names and then never reused at statement position.
  • Fix: add the three strings to the keywords array.

A5. Array literal [, 1, 2], [1,, 2], [,] silently accepted

  • File: row_syntax.c:987-999 (array loop); same shape at :1050-1056 (call args).
  • The comma loop continues without tracking whether a value was seen since the previous comma.
  • Fix: track expression-seen state across the comma loop.

A6. Array repeat [;0] accepted without count

  • File: row_syntax.c:992-996 — when ; is seen, value is pushed without verifying exactly one count already in args.
  • let x [;0] parses with args.len == 1 and array_repeat = true (malformed shape for the checker). [1 2; 3] parses with args.len == 3 and array_repeat = true.
  • Note: checker catches some forms via TYP002, but the parser constructs a malformed AST regardless.

A7. aarch64 SDIV with zero divisor silently returns 0

  • File: native/zero-c/src/aarch64_emit.c:229 (z_aarch64_emit_div_reg) — no zero guard. No checker/buildability diagnostic for constant / x 0 or % x 0 either.
  • Repro: let x i32 / 10 0 runs cleanly, x==0.
  • Portability claim violated: x86_64 IDIV traps with #DE; aarch64 SDIV returns 0. Same source, different runtime behavior, no documented contract.

Tier B — design / drift / UX

B1. i64.MIN / i32.MIN unrepresentable as a literal

  • File: checker.c:4664-4680 (validate_integer_literal_for_type).
  • The validator compares unsigned magnitude against info.max, so 2147483648 is rejected before - 0 2147483648 can fold into i32.MIN. There is no syntactic form that produces signed MIN.
  • Design call: add i32.MIN/i64.MIN constant, support unary minus, or special-case - 0 ... in the validator.

B2. std.path.normalize(".") returns empty string

  • File: std/path.0:130-131 — single-dot segment unconditionally consumed; no tail to emit . when no segments emitted.
  • POSIX realpath, Go filepath.Clean, Python os.path.normpath all return ..
  • Related recent fix: commit 8303bcf "Fix std.path normalization edge cases" touched path but bare . wasn't covered.

B3. std.path.is_sep accepts \ but docs say /-only

  • Code: std/path.0:2 returns true for both 47_u8 (/) and 92_u8 (\).
  • Docs: docs/articles/modules/path.md:16, 36 — "target-neutral lexical operations over /-separated paths" and "uses / as the portable package/example separator".
  • Effect: path.basename "foo\bar" returns "bar".
  • Pick a side: drop the byte-92 branch or update docs.

B4. Parser EOF "expected expression" diagnostic at (1,1) instead of last token

  • File: row_syntax.c:959 (row_parse_primary hardcodes row_diag(parser->diag, 1, 1, 1, ...)); same pattern at :806 (row_append_type_atom).
  • Repro: source pub fn main Void then let myvar reports at (1,1) instead of the location of myvar.
  • Fix: fall back to the previous token's line/column (see row_expect_word:721-722 for the existing pattern).

B5. Underscores rejected in float literals

  • File: checker.c:4684 — parse_float_literal returns false unconditionally if _ appears.
  • Inconsistency: parse_integer_literal accepts 1_000; parse_float_literal rejects 1_000.0.
  • Fix: one-line — strip _ before parse, matching the integer path.

B6. \xFF in char accepted but raw 0xFF byte rejected

  • File: row_syntax.c:306-317 (char \x no range check) vs :326-328 (raw bytes ≥ 128 rejected).
  • Pick a side: accept full byte range in both, or restrict \x to ≤ 0x7F.

B7. Raw NUL byte in source treated as EOF

  • File: row_syntax.c:500 — while (lexer.source[lexer.offset]) uses a C-string idiom with no length tracking. Same pattern in row_scan_string:217 and row_scan_char:286.
  • Repro: a file with embedded 0x00 mid-content reports "unterminated string literal" at the wrong column.
  • Fix: carry source length separately, or emit a precise diagnostic when a NUL is encountered.

B8. No \u Unicode escape — silently mis-decoded

  • File: same catch-all as A2 (row_syntax.c:261).
  • Repro: let s "A" decodes to "u0041" (the \ is consumed, u then literal 0041 follow).
  • Fix: falls out of A2 — once unknown escapes error, this becomes a precise diagnostic. Optionally implement \u{...} for UTF-8.

Related (non-duplicate) prior work

  • #230 — darwin-arm64 Mach-O missing IR ops (CGEN004 errors at build). Distinct category from S1/S2, which generate wrong code for IR ops that do compile.
  • #96 — superseded by #230, same domain.
  • #31 (closed) — type-checker fuzzing in the old curly-brace syntax. S3 is in the row-syntax-era checker (different code path).
  • commit 7976f0b "Fix std.math unsigned edge cases" — touched math but didn't include mod_pow.
  • commit 8303bcf "Fix std.path normalization edge cases" — touched path but missed bare ..

Methodology

  • Six parallel investigations across disjoint domains (tokenizer, parser, numeric, codegen, stdlib, type/borrow).
  • Four independent verification passes that re-reproduced each finding from scratch on f1a5cc9 and searched issues / PRs / git log for duplicates.
  • 18 of 18 findings verified CONFIRMED + UNFILED.

What's next

Happy to send focused PRs for any subset — S1 and S2 are the smallest diffs and have the highest user impact (silent miscompile / segfault on the default macOS host backend). Let me know if you'd prefer this split into multiple issues, want any items dropped, or would rather fix some directly.

Activity

1
PeterXMR5mo ago

Re-audit against current main (51041a5, v0.2.0)

The original report was filed against f1a5cc9 (674 commits ago) using the .row surface syntax, which has since been removed in the .row → canonical .0 migration. I re-tested every finding from scratch in canonical .0 on current main.

Result: 6 of 18 resolved, 12 still reproduce.

The split falls cleanly along the migration line: findings in code that was rewritten for .0 are fixed; findings in the canonical lexer's escape/byte handling, the checker, the aarch64/Mach-O backends, and std.path carried over and still reproduce.

✅ Resolved (6)

#FindingWhy it's resolved
S1i64 compare emits 32-bit CMP w on darwin-arm64Mach-O emitter is now width-aware — both sites emit cmp x (eb09011f); runtime correct.
A1std.math.modPowU32 overflow for modulus > 65536std/math.0 now widens intermediates to u64 and narrows on return; pow(65535,4,4294967291) is correct.
A4reserved-word table missing set/alias/interfaceAll three are now reserved in the canonical keyword table; used as identifiers → PAR100.
A5malformed array literals [, 1, 2] / [1,, 2] / [,] acceptedNow rejected at parse time (PAR100 "expected expression before comma").
A6array repeat [;0] accepted without countNow rejected (PAR100); valid [0_i32; 3] still accepted.
B4EOF "expected expression" diagnostic pinned at (1,1)Diagnostics now carry the correct line/column from the offending token.

🔴 Still reproduces on main (12)

Re-verified with canonical repros; file:line updated to the current tree.

#FindingCurrent location / note
S2macho-arm64 oversized stack frame silently truncated → segfaultFrame ≥ 4096 B emits sub sp, sp, #0x0 (12-bit immediate masked); ELF aarch64 correctly rejects the same program with BLD004 — Mach-O bypasses the frame-size gate.
S3dynamic-index arr[i] = &y clears all borrow tracking (soundness)checker.c origin_path_array_segment_definitely_within still treats a wildcard prefix as definitely-within; constant-index path correctly raises BOR001, dynamic-index path does not.
A2silent decode of unknown string escapescanon_scan_string has a catch-all *offset += 2; char scanner still rejects via PAR100 — asymmetry intact.
A3"\0" silently → "0"Same catch-all; \x00 is rejected but bare \0 is not.
A7aarch64 SDIV/UDIV by zero silently returns 0z_aarch64_emit_div_reg emits a bare divide with no guard; no checker/buildability diagnostic for / 0 or % 0 (constant or runtime). Diverges from x86_64 (#DE).
B1i64.MIN / i32.MIN unrepresentable as a literalMagnitude validated before unary minus folds → TYP016.
B2std.path.normalize(".") returns empty stringAPI is now buffer-based (Maybe<String>); still returns [] instead of ..
B3path separator accepts \ but docs say /-onlyNow the private __zero_std_path_is_sep (/ or \); observable via std.path.basename("foo\\bar") → bar. Docs still say /-only.
B5underscores rejected in float literals1_000.5 → TYP019, while integer 1_000 is accepted.
B6\xFF in char accepted but raw 0xFF byte rejected\x char escape has no range check; raw non-ASCII byte rejected — asymmetry intact.
B7raw NUL byte in source treated as EOFCanonical string/char scanners loop on a C-string with no length tracking; a mid-content NUL ends scanning early.
B8no \u Unicode escape — silently mis-decodedSame string catch-all swallows \u... (and \u{...}) silently; the char scanner whitelists escapes.

Notes

  • All resolved findings stay listed above for traceability; only the 12 marked 🔴 are still actionable.
  • The two soundness/wrong-code survivors — S2 (silent miscompile → segfault) and S3 (borrow-checker unsoundness) — are the highest priority and would each make sense as a focused standalone issue with a clean canonical repro if that's preferable to this bundle.