Libuv threadpool sizing: results and tradeoffs

Where Nub landed

Nub retained CPU-based automatic threadpool sizing; it did not add a memory-based cap or revert every process to four workers. This repository records the measurements behind that decision and the limitations of the policy.

For the evaluated policy in Nub 0.9.3:

Automatic size starts at max(4, available_parallelism). Detected process/thread headroom can reduce it, with a floor of four; Windows additionally caps it at eight.
On Linux, workers beyond the first four run at nice 10 on a best-effort basis.
An explicit UV_THREADPOOL_SIZE wins. Compatibility mode leaves Node's default alone. Nub's automatically chosen value is not inherited by plain Node child processes; a child launched through Nub is sized again.
There is no automatic memory-safety guarantee. CPU availability, nice values and cgroup weights do not reveal how much memory future native operations will allocate.

The recorded decision is nubjs/nub#947. The sizing implementation and priority/child policy are pinned to the source inspected for this study.

The conclusion is a tradeoff, not a universal optimum. Larger pools produced substantial gains in several tested workloads, but also caused OOMs and reduced a neighbouring cgroup's throughput. Four avoided some failures and still failed others. The experiments did not establish a better general-purpose replacement default.

Main results

The two Linux campaigns contain 248 trials, plus four identity records. All observations below have raw records and executable fixtures in this repository; the older HTTP result is included separately with its original provenance.

Question	Observation	Limit of the conclusion
Can a larger pool help?	At 1 GiB, a 32-job scrypt batch took 721 ms with four Nub workers and 327 ms with automatic sixteen.	One workload and allocation, not a universal speedup.
Can it cause OOM despite nice 10?	At 384 MiB, the same 32-MiB-per-operation workload completed with four and was OOM-killed with automatic sixteen, 3/3 trials.	A finite burst, not an unbounded producer.
Does niceness protect a neighbouring cgroup?	On a 16-vCPU host, automatic sixteen increased Nub's PBKDF2 throughput by 69% but reduced an equal-weight sibling's throughput by 21% versus four. Extra workers were verified at nice 10.	Cgroup isolation still worked. More parallel work consumed capacity the neighbour had previously borrowed.
Does a generic memory cap solve it?	A prototype choosing one worker per 64 MiB fixed the 32-MiB-operation case, then failed with 64-MiB operations where four workers completed.	The tested constant is not a production-ready memory model. This does not disprove every possible heuristic.
Is four a memory-safe fallback?	At 384 MiB, four workers were also OOM-killed with 128-MiB scrypt operations, 3/3 trials.	Pool size alone cannot bound arbitrary operation memory.
Does the linked gzip example require a large pool to OOM?	The unbounded producer was OOM-killed with both one and four workers. Its bounded-submission controls survived the watchdog window.	That example demonstrates queued-input pressure, not an autosizing comparison.

Times are medians of three trials unless stated otherwise. See results for budgets, parameters, memory peaks, historical tail latency, and the positive and negative controls.

What the admission-control experiment does—and does not—show

The experiment was given the memory cost; Nub did not discover it. The scrypt fixture supplies an approximate 128 × N × r working-set estimate, its deliberately retained buffer size, and a chosen half-memory reserve.

Keeping sixteen workers while admitting eight known 64-MiB operations at a time completed a 1-GiB batch in 841 ms, versus 1477 ms with four workers. But this is a controlled application-level example, not a generic limiter for arbitrary Node code. The reserve is a heuristic, and the estimate excludes allocator retention, other jobs and future allocations. Even the fixed admission-four control had much higher peaks than a four-worker pool in one case.

For general image processing, native addons, mixed work and input-dependent allocations, the memory requirement may not be knowable before execution. Neither a startup launcher nor the successful controlled example establishes otherwise. Measurement can guide deployment-specific limits; it cannot turn them into a guarantee for unknown inputs.

Read or reproduce

The saved data can be validated without running workloads or inducing OOM:

python3 scripts/verify.py

Live probes require a disposable Linux VM with cgroup v2 and administrative access. They deliberately induce OOM inside constrained groups. Do not run them on a production or shared machine.

Motivation and references

The investigation started from the risks discussed in platformatic/gzip-eval-async. Those risks are real; the purpose here is to distinguish queue growth, executing native work, within-group priority and cross-group CPU sharing.