Proposal: Add application benchmarks for Monty
To get better insights into Monty's performance, I've vibe-coded a small benchmarking tool:
- Repository: https://github.com/jaheba/flying-circus
- Results: https://jaheba.github.io/flying-circus/
The idea is twofold:
- Compare Monty against CPython and PyPy using application workloads to better understand where Monty stands today.
- Track performance changes over time, making it easier to measure the impact of optimizations and catch potential regressions.
The tool can compare arbitrary Python runtime executables, so it's also possible to benchmark a development build of Monty against the latest release or compare two commits directly.
It measures execution time, startup time, and memory usage across different workloads, with reports highlighting improvements and regressions.
Would it make sense to integrate this into Monty's CI?
We could start by running benchmarks periodically or on demand, rather than on every PR, to establish a performance baseline and track changes over time.
What do you think? Would this be useful for Monty development?
Disclaimer: This proposal was drafted and edited with the assistance of AI.
Activity
0No activity yet.