I noticed the test suite success rate seems to depend on the display scaling factor, after trying various combinations in my own project. I get similar results from building using the supplied Makefile or .sln, though I haven't methodically gone through every combination - is this a known issue? If it isn't, I'll try a bit harder to put together a full set of repro steps and get some more results.
(All tests done with ffmpeg available, and https://github.com/ocornut/imgui_test_engine/pull/92 applied. Win11 tests were done with Win32+DX11, which seems to be the only option when building with VC++; macOS and Linux tests were done with SDL2+GL3.)
| OS | Scale | Failure count |
|---|---|---|
| Win11 | 150% | 4 |
| Win11 | 100% | 0 |
| macOS | 150% | 0 |
| Linux | 125% | 45 |
| Linux | 200% | 9 |
Activity
3
ocornut9mo agoTo elaborate on this statement from another thread:
Though, this could actually explain a couple of oddities I noticed in my initial tests
As per design the test engine is designed toward "being able to succeed in all cases" (spoiler alert: it's not there yet). For this I found it useful to design the imgui_test_suite around the idea that tests inheriting a state and having side effects was ultimately actually desirable. It's a bit counterintuitive because traditional test suites for other software would likely be designed around total control of initial state, but it has higher dev/maintainance cost. And our tests are closer to being system/integration tests than unit tests.
I deferred changing the font size because I knew it broke some tests, many have been fixed/improved over time and I committed that change last week. Varying scale is similar to running tests in a random order: it is likely to unearth many issues. Some issues are bug of the test itself, which we can aim to sturdy, but some are the manifestation of test engine not being able to do a thing in certain situation and I am committed to improve this. As of today, docking fails a few tests and I'm working on that, and I will increasingly try things with varying font sizes.
ocornut2mo agoLowered the fail count for docking with scaling to 2 tests. Along the way a few things were made more robust in the test engine as well (e.g. handling of moving items).
Yes. I only last week made the test suite display scale dependent and started discovering and fixing issues related to that (some in test suite, some in test engine). I actually spent the entire day yesterday following a deep thread of things to make generic fixes required for docking tests.