#3224
Pull Requests/catchorg/Catch2/#3224

Stabilize randomized Bazel shards

Openmattyclarksonwants to merge intofrom
1 commit1 file changed+6-0

Description

Catch2 defaults to randomized test order, but Bazel does not set TEST_RANDOM_SEED for ordinary sharding. Independent shard processes can therefore select overlapping cases and omit others.

When Bazel enables multiple shards, use declaration order only if the requested order is randomized. Preserve --order=decl and --order=lex, which already have stable membership, and leave non-sharded runs unchanged. Explicitly seeded random order is also replaced by this fallback.

GitHub Issues

None.

Activity

3
M
Stabilize randomized Bazel shards
58a415e
codecov[bot]7d ago

Codecov Report

:x: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 91.16%. Comparing base (0809213) to head (5f768cf).

Additional details and impacted files 
@@            Coverage Diff             @@
##            devel    #3224      +/-   ##
==========================================
- Coverage   91.18%   91.16%   -0.02%     
==========================================
  Files         205      205              
  Lines        9101     9103       +2     
==========================================
  Hits         8298     8298              
- Misses        803      805       +2
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
horenmarrequested changes7d ago
horenmar7d ago

No. Move the check downwards and account for cases where the RNG seed is properly set.