feat(benchmark): add clock resolution estimation iteration limits and CLI config

Open

Tyagiquamarwants to merge intofrom

Problem Description

Resolves #3180.

When running benchmarks in constrained environments (such as embedded targets with limited RAM, e.g. 16MB), estimate_clock_resolution could double iterations repeatedly in run_for_at_least until reaching excessive counts (e.g. 1,280,000+ points), causing std::bad_alloc due to concurrent allocations of times and deltas vectors. Additionally, clock resolution estimation time was hardcoded at 500ms without command-line configurability.

Solution

Added max_iterations bound support to run_for_at_least.
Added clock_resolution_estimation_max_iterations cap in estimate_clock_resolution and bounded sample count allocation in estimate_clock_cost.
Exposed --benchmark-estimate-clock-time on the command-line interface, ConfigData, Config, and IConfig (default: 500ms) matching --benchmark-warmup-time.
Added unit tests for clock estimation iteration limits in InternalBenchmark.tests.cpp.
Added CLI parsing test cases in CmdLine.tests.cpp.

Tests

Unit tests in InternalBenchmark.tests.cpp
CLI configuration tests in CmdLine.tests.cpp

Activity

feat(benchmark): add clock resolution estimation iteration limits and CLI config

2fb2016Oct 1

fix(benchmark): compare matching clock duration types

285dbceOct 8

Tyagiquamar22d ago

Fixed the mixed-duration std::max call by converting elapsed nanoseconds to FDuration before comparison. This preserves the one-nanosecond lower bound and fractional arithmetic. Docker Ubuntu 22.04/GCC 11.4/C++14 reproduced the original compiler error; the library and SelfTest build now pass. Focused warmup, resolution, estimate_clock_resolution and Benchmark Fibonacci checks passed (19 assertions, four cases, five samples per benchmark). clang-format 14 check on the changed block and whitespace checks passed. MSVC validation is left to CI.

fix(benchmark): update run-benchmark tick expectation for capped clock estimation

b450270Oct 20

test(benchmark): refresh ApprovalTests baselines for clock-limit CLI

037beffOct 20

fix(test): restore ApprovalTests high-byte chars and junit order

6fb6461Oct 20

codecov[bot]10d ago

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 91.19%. Comparing base (222e233) to head (6fb6461). :warning: Report is 11 commits behind head on devel.

Additional details and impacted files

@@            Coverage Diff             @@
##            devel    #3209      +/-   ##
==========================================
+ Coverage   91.17%   91.19%   +0.02%     
==========================================
  Files         206      205       -1     
  Lines        9080     9104      +24     
==========================================
+ Hits         8278     8302      +24     
  Misses        802      802

:rocket: New features to boost your workflow:

:snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.