#1345

fix(memory): release the inflight lock before spawning the fetch

Openkarthik1729wants to merge intofrom

What

get_or_fetch_inner holds the inflight mutex guard — a temporary of the match scrutinee — while it calls spawner.spawn(fetch). When the tokio runtime is shutting down, spawn cancels the new task synchronously and drops the future on the spawning thread; RawFetch::drop then takes the same non-reentrant lock to dequeue itself, and the thread deadlocks against itself.

This change binds the enqueue result in a let first, so the guard is released before the spawn.

Why

We hit this in a service that uses foyer through SlateDB\x27s block cache: any test process whose runtime shut down while a block fetch was being spawned sat at 0 % CPU forever (four different test binaries, always the same stack: worker thread parked in parking_lot::RawMutex::lock_slow from RawCache::drop ← cancel_task ← Handle::spawn ← get_or_fetch_inner, test thread waiting in BlockingPool::shutdown). The same applies to any process shutdown while a fetch is in flight.

Test

closing_runtime_tests::a_fetch_spawned_on_a_closing_runtime_does_not_deadlock: a get_or_fetch issued under a handle whose runtime has been shut down must return. It fails on main (5 s watchdog) and passes with this change.

Activity

2
sorenbs3d ago

Independent confirmation from Prisma Streams, which uses foyer 0.22.3 through SlateDB 0.15's FoyerCache.

We hit the same single-thread deadlock: get_or_fetch_inner -> Handle::spawn -> bind on a closed OwnedTasks -> cancel_task -> RawFetch::drop -> inflights.lock(). It hangs one of our restart tests when a #[tokio::test] runtime drops while a cache miss is being spawned: 3 hangs in 100 runs.

  • This PR applies cleanly to v0.22.3.
  • Its test fails there without the raw.rs change (5 s watchdog) and passes with it.
  • With the patch applied through [patch.crates-io], our test passed 200 of 200 runs.

The DCO check is the only red item. Could a maintainer take a look, and would a 0.22.x patch release be possible?