#1636

Fuzz the JavaScript engine

Draftfazxeswants to merge intofrom

Summary

  • Add three fuzz targets, js_program, js_regexp, and js_json, seeded with programs models wrote, and list them for the nightly Fuzz workflow.
  • Fix a crash in String.prototype.normalize and localeCompare. The engine's C called its realloc callback through the wrong function type, which the function-type check traps on in release builds. The fork now wraps the callback, and FORK.md records the patch.
  • Leave the engine's C uninstrumented in fuzz builds, since Zig's fuzzer can't read C coverage. The targets follow the Zig code's coverage.

Stacked on #1635.

Architecture

Before, tests/fuzz/targets.json could name targets in fx or the session manager, and the engine was only tested by test-js:

flowchart LR
  T["tests/fuzz/targets.json"] --> B["build.zig reads targets and saved inputs"]
  B -->|"fx targets"| F["fuzz"]
  B -->|"session manager targets, in its build options"| S["fuzz-session-manager"]
  E["src/js engine"] -.->|"not fuzzed"| X["test-js"]

After, the engine is a third fuzz module with its own binary:

flowchart LR
  T["tests/fuzz/targets.json"] --> B["build.zig reads targets and saved inputs"]
  B -->|"fx targets"| F["fuzz"]
  B -->|"session manager targets, in its build options"| S["fuzz-session-manager"]
  B -->|"js targets, in the engine's build options"| J["fuzz-js"]
  J --> P["js_program, js_regexp, js_json in engine.zig"]
  P --> C["QuickJS C: no coverage instrumentation, function-type check traps"]

The engine imports nothing from fx, like the session manager, so its saved inputs travel in its own build options as fuzz_corpus_<target>.

Why wrap the callbacks

The function-type check found a real bug, so the fork fixes the calls instead of turning the check off for the engine's C, which would also hide the next mismatch. The fix is two wrappers, fx_dbuf_realloc_ctx and fx_dbuf_realloc_rt, with the exact DynBufReallocFunc type. FORK.md records them for the next upstream update.

Zig's sanitizer runtime has no handler for this check, so Debug builds used to leave it out. Trapping on it keeps the check in Debug, where the tests and the fuzzer run.

Testing

On #1635, a ReleaseSafe build that calls either method crashes:

error: 'probe-normalize.test.normalize and localeCompare return' terminated with signal TRAP

A Debug build of #1635 runs the same program without a trap, because it doesn't check the call. Built with this branch's flags, #1635's code traps in Debug too.

On this branch, the same program returns:

"\u00e9".normalize("NFD").length                            2
"e\u0301".normalize("NFC").length                           1
"\ufb01".normalize("NFKD")                                  "fi"
"\u1e9b\u0323".normalize("NFKC").length                     1
["b", "a", "\u00e9", "e"].sort((x, y) => x.localeCompare(y))  ["a","b","e","é"]
  • zig build test-js passes in Debug and ReleaseSafe, 26 of 26.
  • zig build fuzz -Dfuzz-target=<target> --fuzz=500 is clean for all three targets, and 2,000 more inputs each on js_regexp and js_json.
  • In a Debug build with these flags, a program that calls every function reachable from the globals, their constructors, and their prototypes (391 functions, 17,220 calls) finishes without a trap.

Activity

3

This pull request is a draft

Merging is unavailable until it is marked ready for review.