Description

Add an opt-in USE_RELATIVE_PATHS mode to catch_discover_tests for DISCOVERY_MODE PRE_TEST.

When enabled, the generated CTest include resolves the test executable, discovery script, working directory, and discovery cache relative to its own relocated build-tree location. This keeps the existing behavior unchanged by default, supports single- and multi-config generators, and requires CMake 3.24 for the path generator expressions. The option and version requirement are documented.

The regression relocates the build tree before first discovery, uses a generator-expression WORKING_DIRECTORY, verifies the discovered names, runs the actual tests, and cleans the generated discovery cache.

GitHub Issues

Closes #3051

Testing

Single-config Ninja relocation before first discovery: 5/5 tests passed
Multi-config Ninja/Zig in a path containing spaces: default and relocated relative modes both discovered 5 tests; relocated CTest passed
CMake 3.23.3: default mode configured successfully; opt-in mode failed with the expected CMake 3.24 requirement
Python syntax and git diff --check passed

Activity

Fix relative PRE_TEST discovery paths

899f969Jul 11

codecov[bot]3mo ago

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 91.16%. Comparing base (499442d) to head (899f969).

Additional details and impacted files

@@           Coverage Diff           @@
##            devel    #3175   +/-   ##
=======================================
  Coverage   91.16%   91.16%           
=======================================
  Files         206      206           
  Lines        9016     9016           
=======================================
  Hits         8219     8219           
  Misses        797      797

:rocket: New features to boost your workflow:

:snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

NgoQuocViet20012mo ago

Rebased onto current devel and resolved the upstream discovery-test changes while preserving the USE_RELATIVE_PATHS regression.

Focused Windows/MSVC validation passed for both the default and relocated relative modes: 22 test names matched the CTest listing, 22 matched TEST_LIST, and the relocated tests ran successfully. AppVeyor is also green across all 5 jobs.

NgoQuocViet20012mo ago

Rebased onto current devel — the branch had gone conflicting again and now applies cleanly.

The conflict, and why it was not a mechanical resolve. devel has since added TEST_PREFIX " prefix " / TEST_SUFFIX " suffix " to the discovery test, with VerifyRegistration.py asserting TEST_LIST against CTEST_NAME_PREFIX + name + CTEST_NAME_SUFFIX. This PR adds a second registration — the relative-registration/ subdirectory used for the relocation case — which had no prefix or suffix. Taking either side alone breaks it:

keep only mine → the main case loses the prefix/suffix coverage devel just added;
keep only devel's → the relative case registers plain names while expected_names demands decorated ones, so TEST_LIST mismatches.

So the resolution puts the same TEST_PREFIX / TEST_SUFFIX on the relative registration, which also makes the relocation case exercise the same discovery options as the main one rather than a reduced variant. expected_names then stays unconditional.

Verified, not assumed. Built and ran the real script under CMake 3.28.3 / g++ 13.3 (the relative branch needs ≥ 3.24):

$ python3 tests/TestScripts/DiscoverTests/VerifyRegistration.py <src> <out>
22 tests matched in CTest listing
22 tests matched in TEST_LIST variable      <- normal build
22 tests matched in CTest listing
22 tests matched in TEST_LIST variable      <- relocated, USE_RELATIVE_PATHS

I also checked the resolution was necessary rather than incidental — dropping the two lines back off the relative registration reproduces the failure:

  Catch2 name ' prefix Some test suffix ' not in TEST_LIST
  Catch2 name ' prefix [ suffix ' not in TEST_LIST
  ... (10 more)

The USE_RELATIVE_PATHS regression this PR exists for is intact, and the final diff is 5 files / +188 / −42 against devel.

No maintainer has looked at this yet and I know the queue is long — no rush from me. Flagging only that it is current and green again, so it is not blocked on being stale. Happy to close it if the approach is not one you want.