#1203

valu_fuzz f16: f16 multiply gives +0 instead of -0 on Apple A18 Pro (MoltenVK)

Openundeadhuman37opened 6h ago

Found while running TEST-58 for #166 (entry).

Setup: MacBook Neo (Apple A18 Pro, subgroup 4..32), macOS 26.6.2, MoltenVK 1.4.2, Homebrew llvm-mc 23.1.6, spirv-tools 1.4.363.0, commit d3841c0.

What happens: tools/valu_fuzz.py --family f16 (default seed 1, 50 programs) has 1 mismatch natively and 2 with --emulate. In both cases an f16 multiply of zero by a negative number gives +0 (0x0000) where the model expects -0 (0x8000). Every other bit matches. It shows up the same way every run. --family mem and lds_fuzz.py match fully on the same machine.

ProgramRunInstructionInputs (thread 0)GotWant
--show 1:23:32native and --emulatev_mul_f16_e32 v27, v32, v23v32 lo = 0x0000 (+0), v23 lo = 0xcbb2 (negative)0x400000000x40008000
--show 1:5:64--emulate only (native skips wave64)v_pk_mul_f16 v20, v35, v24 op_sel:[1,0] op_sel_hi:[0,1], high halfv35 lo = 0xe7a0 (negative), v24 hi = 0x0000 (+0)0x0000ba790x8000ba79

The same result shows on all 8 threads that are printed.

Reproduce:

python3 -I tools/valu_fuzz.py --family f16 --build runtime/build/linux
python3 -I tools/valu_fuzz.py --family f16 --emulate --build runtime/build/linux
python3 -I tools/valu_fuzz.py --family f16 --show 1:23:32 --build runtime/build/linux

Possible cause (not checked): the GPU or MoltenVK may not keep the sign of zero in f16 arithmetic unless the SPIR-V asks for it (for example with SignedZeroInfNanPreserve for 16-bit floats). I haven't checked whether other GPUs pass these two programs.

Note: my #166 entry calls the first case "high half lost". That's wrong. Both cases are the sign of a zero result: the low half in the first, the high half in the second.

Only repository-generated fuzz programs were used. AI-assisted. No game files.

Activity

9
undeadhuman376h ago

More on this. It isn't only f16, and it isn't the translator:

  • PR #664's cs_signed_zero_nan shows even an f32 v_mul_f32 -0.0 * 1.0 comes back +0 on this Mac (out[0] = 0, want 0x80000000). The translator emits a plain OpFMul.
  • It stays the same with SignedZeroInfNanPreserve 32 declared (#664 applied on main), and with MVK_CONFIG_FAST_MATH_ENABLED=0. MoltenVK reports shaderSignedZeroInfNanPreserveFloat16/32 = true anyway.
  • -0 + -0 keeps its sign. NaN checks pass with #664 and fail without it.

So the -0 product looks like a MoltenVK/Metal limit on Apple GPUs, not something the translator can fix by declaring the mode. Details are in my comment on #664.

AI-assisted. No game files.

Urban2255h ago

Same symptom on AMD, so this is not MoltenVK-only.

RX 7900 XTX (RDNA3), AMD proprietary 26.10.1 (LLPC), Windows 11, commit c63773d, llvm-mc 21.1.8.

python -I tools/valu_fuzz.py --family f16 --programs 200 --seed 3: 399/400 match native, 399/400 with --emulate. The one mismatch is the same program in both modes, --show 3:104:64:

thread 0..7 v22: got 0xc2df0000, want 0xc2df8000 (last written by op 11: v_mul_f16_e64 v22, v29, v20)

Only the sign of the low-half zero differs. Here the operands are the other way round from your cases: the low half of v29 is -0 (low 16 bits of the f32 that v_cvt_f32_f16_e32 v29, v30 produces from a constant) and v20 is positive (0x3c00 | (thread & 7) << 7, so 1.0 to 1.875). -0 times a positive value should stay -0.

Your two programs pass here: seed 1 is 400/400 native and emulated on this card, so 1:23:32 and 1:5:64 keep their -0. Seeds 7900 and 7901 are also 400/400.

Hypothesis, not verified: the -0 operand is a compile-time constant in this program, so the driver may fold 0 * x to +0. I have not tested with #664 applied; I can if that helps.

AI-assisted (Claude read the logs and wrote this); the runs are from my machine.

Urban2255h ago

Update for the AMD case I reported above: it is fixed by #664's SignedZeroInfNanPreserve 32. Seed 3 goes from 399/400 on main (426023a5) to 400/400 with the declaration added, same card and driver. Numbers are in #664.

So there are two different things under this symptom: on AMD (LLPC) the driver honours the execution mode and only drops -0 without it; on MoltenVK, per your follow-up, it is dropped even with the mode declared.

DomasUrbonas4h ago

Independently reproduced both signed-zero mismatches on an Apple M4 Pro.

Environment

  • GPU: Apple M4 Pro (subgroup 4..32)
  • OS: macOS 27.2.1
  • Driver: MoltenVK 1.4.2
  • LLVM: Homebrew llvm-mc 23.1.6
  • Commit: e7584c52

1. Native wave32 — v_mul_f16_e32

Command: python3 -I tools/valu_fuzz.py --family f16 --programs 30 --seed 1 --wave 32 --build runtime/build/linux

Result: 29/30 matched, one mismatch in cs_f16fuzz_s1_23_w32.

  • Got: 0x40000000
  • Expected: 0x40008000

2. Emulated wave64 — v_pk_mul_f16

Command: python3 -I tools/valu_fuzz.py --family f16 --programs 10 --seed 1 --wave 64 --emulate --build runtime/build/linux

Result: 9/10 matched, one mismatch in cs_f16fuzz_s1_5_w64.

  • Got: 0x0000ba79
  • Expected: 0x8000ba79

Both mismatches are identical to the original A18 Pro report: the sign of an f16 zero result is lost (+0 instead of -0).

No skipped or untranslated programs, and no harness errors in either run.

Additional TEST-71 memory and LDS fuzzing: 300/300 matched across the configurations tested. Full results posted in #166.

This confirms the same behavior on another Apple GPU, but does not independently establish whether MoltenVK or Metal is responsible.

AI-assisted analysis; all tests executed locally using synthetic programs. No game data.

DomasUrbonas4h ago

Follow-up: both mismatches disappear with KosmicKrisp on the same M4 Pro.

I repeated the exact tests from my previous comment, using the Vulkan loader instead of Leonida's default direct MoltenVK library:

LEONIDA_VULKAN_LOADER=/opt/homebrew/opt/vulkan-loader/lib/libvulkan.1.dylib

Verified driver selection with VK_LOADER_DEBUG=driver:

Using "Apple M4 Pro" with driver: ".../libvulkan_kosmickrisp.dylib"

KosmicKrisp 26.2.99, Vulkan 1.4.363, macOS 27.2.1, commit e7584c52.

Results:

TestMoltenVK 1.4.2KosmicKrisp
f16 native wave32, seed 1, 30 programs29/3030/30
f16 emulated wave64, seed 1, 10 programs9/1010/10

Both previously failing cases (1:23:32 and 1:5:64) now match the CPU reference model. No mismatches, skipped programs, untranslated programs, or harness errors.

The same synthetic programs therefore produce different results depending on the Vulkan driver, suggesting a MoltenVK-specific signed-zero handling issue rather than an unavoidable Apple GPU hardware limitation.

KosmicKrisp reports subgroup range 32..32; MoltenVK reports 4..32.

Edit: Additional validation: Extended the KosmicKrisp f16 fuzzing run to 50 programs per wave size:

python3 -I tools/valu_fuzz.py --family f16 --programs 50 --seed 1 --wave both --emulate --build runtime/build/linux

Result: 100/100 matched, with 0 mismatches, 0 skipped, 0 untranslated, and 0 harness errors.

This includes both previously failing seed-1 cases.

bobcatchris3h ago

Apple M2 (Mac mini, Mac14,3) data point

Environment: Apple M2, macOS 26.4 (25E246), MoltenVK 1.4.2 (device Vulkan 1.4.357), subgroup 4..32, main f85582e23d4ab9e19acbf694358f137d6b8531b1.

  • Seed 1 f16: native 196/200 (4 mismatches, wave64 skipped); --emulate 393/400 (7 mismatches).
  • Seed 3 f16: native 195/200 (5 mismatches); --emulate 391/400 (9 mismatches).

Every mismatch is the signed-zero class: +0 where -0 is expected. Seed 3 reproduces the exact AMD case 3:104:64 (got 0xc2df0000, want 0xc2df8000) and the original Apple 1:23:32 / 1:5:64 bit patterns on M2. mem and LDS fuzzing are clean.

This is a third Apple GPU matching the A18 Pro and M4 Pro reports, on direct MoltenVK (not KosmicKrisp). No new issue.

AI-assisted.

Jolymmiles3h ago

More data from an Apple M3 Max (MoltenVK 1.4.2 and KosmicKrisp from Homebrew mesa 26.3.1, commit f85582e2); full entry in #166: https://github.com/ItsNotRexed/Leonida/issues/166#issuecomment-7078295128.

valu_fuzz.py --family f16 --programs 200 --seed 1 --wave both: MoltenVK 196/200 native and 393/400 --emulate, KosmicKrisp 200/200 and 400/400. The 7 programs that fail on MoltenVK:

ProgramRunsOp that makes the -0Thread 0 gotThread 0 want
1:23:32native, --emulatev_mul_f16_e32 v27, v32, v230x400000000x40008000
1:174:32native, --emulatev_mul_f16_e64 v30, v22, v290x4f3a00000x4f3a8000
1:180:32native, --emulatev_pk_mul_f16 v21, v33, v20 op_sel:[1,0] op_sel_hi:[0,1]0x0000c95c0x8000c95c
1:191:32native, --emulatev_pk_mul_f16 v20, v20, v33 op_sel:[1,0] op_sel_hi:[1,0]0x000000000x80008000
1:5:64--emulatev_pk_mul_f16 v20, v35, v24 op_sel:[1,0] op_sel_hi:[0,1]0x0000ba790x8000ba79
1:152:64--emulatev_pk_mul_f16 v20, v30, v20 op_sel:[0,0] op_sel_hi:[0,1]0x000014610x80001461
1:171:64--emulatev_pk_mul_f16 v26, v35, v20 op_sel:[1,1] op_sel_hi:[1,1] (high half; a later v_lshrrev_b16 writes only the low half)0x0000b8d50x8000b8d5
  • Also -0 × positive in f16. The other six are +0 × negative. In 1:174:32 v22 low half is 0x8000 (-0) and v29 low half is 0x285d (positive); it also comes back +0, as #1376's fma(l, r, +0) predicts.

  • Only the sign differs. A --emulate rerun with every mismatched word printed gave the same 7 programs, 64 words each (one register per thread). All 448 differ only in the sign of zero halves (both halves in 1:191:32).

  • Same SPIR-V, two drivers. The 1:23:32 .spv run through vk_compute_test once per driver: thread 0 v27 is 0x40000000 on MoltenVK, 0x40008000 on KosmicKrisp. All 800 generated f16 .spv files were byte-identical between the MoltenVK and KosmicKrisp runs.

  • #1376's fix clears all 7. I reran the same f16 command (200 programs, native and --emulate) on the two MoltenVK main (5189f8de) builds made on this Mac for #1376, which differ only in the SPIRV-Cross revision:

    RunSPIRV-Cross aa217aeb (control)SPIRV-Cross 300681bb (fix)
    native wave32196/200, the same 4 programs200/200
    --emulate393/400, the same 7 programs400/400

    On the fixed build, 1:23:32 thread 0 v27 is 0x40008000.

Reproduce, MoltenVK then KosmicKrisp:

LLVM_MC=/opt/homebrew/opt/llvm/bin/llvm-mc python3 -I tools/valu_fuzz.py --family f16 --programs 200 --seed 1 --wave both --emulate --build runtime/build/mac
LEONIDA_VULKAN_LOADER=/opt/homebrew/opt/vulkan-loader/lib/libvulkan.1.dylib \
VK_DRIVER_FILES=/opt/homebrew/share/vulkan/icd.d/kosmickrisp_mesa_icd.aarch64.json \
LLVM_MC=/opt/homebrew/opt/llvm/bin/llvm-mc python3 -I tools/valu_fuzz.py --family f16 --programs 200 --seed 1 --wave both --emulate --build runtime/build/mac

AI-assisted (Claude Code). Synthetic programs only, no game files.

bobcatchris2h ago

Fresh Apple M2 confirmation at upstream/main c2967a5e, additive to the earlier M2 data point at f85582e2 (https://github.com/ItsNotRexed/Leonida/issues/1203#issuecomment-7078281958). Same machine, same result; plus new seed-5 data.

Apple M2 (Mac mini, Mac14,3), macOS 26.4 (25E246), MoltenVK 1.4.2 (device Vulkan 1.4.357), subgroup 4..32, Homebrew llvm-mc 23.1.6, Python 3.13.18.

valu_fuzz.py --family f16 --programs 200 --build runtime/build/linux:

  • seed 3 native: 195/200 match, 5 mismatched, 200 wave64 skipped.
  • seed 3 --emulate: 391/400 match, 9 mismatched, 0 skipped.
  • seed 5 native: 195/200 match, 5 mismatched, 200 wave64 skipped.
  • seed 5 --emulate: 393/400 match, 7 mismatched, 0 skipped.
  • 0 not translated, 0 harness errors in all four.

Got/want bits (thread 0; every difference is the sign of a zero f16 result, +0 where -0 is expected):

  • 3:194:32 (native and --emulate): got 0x40000000, want 0x40008000
  • 3:50:32 (native and --emulate): got 0x00000000, want 0x80008000
  • 3:62:32 (native and --emulate): got 0x00000000, want 0x80008000 (v24) and 0x00000000, want 0x80008000 (downstream v34)
  • 3:75:32 (native and --emulate): got 0x00000000, want 0x80000000
  • 3:195:32 (native and --emulate): got 0x0000d81b, want 0x8000d81b
  • 3:104:64 (--emulate only; wave64 skipped natively): got 0xc2df0000, want 0xc2df8000
  • 3:94:64, 3:110:64, 3:130:64 (--emulate only): got 0x40000000/0x40f60000/0x0000bc00, want 0x40008000/0x40f68000/0x8000bc00
  • 5:8:32, 5:10:32, 5:153:32 (native and --emulate): got 0x00000000/0x0000c0e2/0x00000000, want 0x80008000/0x8000c0e2/0x80008000
  • 5:44:32, 5:94:32 (native and --emulate): got 0x00090000/0xc08e0000, want 0x00098000/0xc08e8000 (plus downstream words)
  • 5:42:64, 5:145:64 (--emulate only): got 0x00005b34/0x41900000, want 0x80005b34/0x41908000

This matches the existing Apple reports bit for bit: the A18 Pro and M4 Pro cases 1:23:32 and 1:5:64, and the AMD case 3:104:64 reported by Urban225. M2 does reproduce it. No new issue. Synthetic fuzz programs only; no game files. AI-assisted.

Jolymmiles2h ago

Upstream PR for the MoltenVK side: https://github.com/KhronosGroup/MoltenVK/pull/2917. It moves MoltenVK's pinned SPIRV-Cross from aa217aeb to the current SPIRV-Cross main (57e4efcc). That revision contains the spvFMul fix (300681bb) and the MSL 3.2 inlined precise helpers (98cbb7c0).

Apple M3 Max, macOS 27.3. MoltenVK main built with each SPIRV-Cross revision; only the dylib changes between runs:

RunHomebrew 1.4.2aa217aeb57e4efcc (PR)
valu_fuzz.py --family f16 --seed 1 --wave 3249/5049/5050/50
--seed 1 --emulate98/10098/100100/100
--seed 4 --emulate98/10098/100100/100
scripts/run_gpu_tests.py85/139, 52 skipped85/139, 52 skipped85/139, 52 skipped, same status per test

Until a MoltenVK release includes it, LEONIDA_VULKAN_LOADER can point at a MoltenVK built from that branch. Note that Khronos marked MoltenVK deprecated on 2026-10-13 and now recommends KosmicKrisp on Apple Silicon, so the merge may take a while.

AI-assisted (Claude Code). Synthetic programs only, no game files.