#164

ci-speedup: catalog gap for serial Lighthouse CI passes, detector drafted (OPT95)

Openbensynapseopened 2d ago

I ran ci-speedup against a site repo today. The slowest check a push waited on was a www job. Its time was two serial Lighthouse CI passes, mobile then desktop, 5 URLs x 5 runs each. No catalog detector matched the log. The pole dropped to the LLM fallback and the run captured a gap.

I drafted the deterministic detector the gap-to-catalog loop asks for. It sits on a branch of my fork so you can look before a PR: https://github.com/bensynapse/skills/tree/ci-speedup-lighthouse-audits

What it does:

  • A _parse_log leaf keyed on lhci's own lines. Running Lighthouse N time(s) on <url> lines are grouped into passes closed by Done running Lighthouse!. The pass is named from --config= or --preset= on the command line. Checking assertions against U URL(s), M total run(s) is quoted as evidence.
  • It fires at 20 audits or more and at least two audits per distinct URL. So lhci's default of 3 runs over two URLs stays silent. An unterminated pass is not counted.
  • The magnitude is the share of audits that re-audit a URL already audited once. On the capture: 50 audits, 45 repeats, 90 percent.
  • A new pattern, OPT95, with the hand-off. Read the config history first. Keep N at what the assertions' aggregation needs. One representative page per template. Run the presets or URL shards as matrix legs on separate runners, not one after another on one runner. Never two passes on one runner, because of port and CPU contention. The required-checks caveat for new job names is carried.
  • collect_runs.py now bins lighthouse and lhci step names as scan. The captured step was other, and the off-category demotion would otherwise keep the leaf from crowning the pole it was drafted for. Worth a look: OPT83 may now consider adjacent lhci steps parallelizable, and its YAML rules do not know about port contention.

Gate: draft_detector.py verify passes. test_blocking_path.py goes from 408 to 412 tests. The full skill and maintainer suites: 5421 passed, 14 skipped. One pre-existing failure in test_opt82_resolver_snapshot needs a pinned engine commit a shallow clone does not have. Everything in the test is synthesized. No line from the captured log is in the diff.

The fix on my side was a matrix split. Four legs, each device over half the pages, same runs and assertions. The critical path went from one full pass to about half a pass.

If this fits the catalog, I will open the PR from that branch.

Activity

0

No activity yet.