One Blocking Call Starves Your Tokio Runtime

A single blocking syscall parks a Tokio worker — and every task queued behind it waits. Spot it, fix it, cap it.

Dafallah Ibrahim

— 2 min read
Share

A piece titled "Principles for Fast Tokio Applications" is on Hacker News's front page this morning, and every serious discussion of fast Tokio apps circles back to the same rule: never block a worker thread.

Most "random latency spike" bugs in async Rust aren't async problems at all. They're one synchronous call hiding inside a task, quietly parking a thread that thousands of other tasks were counting on.

Why this matters

Tokio's multithreaded runtime starts one worker per CPU core. Your 16-core box has 16 workers cooperatively polling tens of thousands of tasks. The whole model only pays off while those workers keep moving.

When a task blocks its OS thread, that worker stops polling, stops stealing work, and delays every timer and connection assigned to it. You lose 1/N of your compute instantly, and your p99 becomes "p50 plus however long the blocker ran." The logs show nothing.

How it works

Tasks don't own threads; they get polled. A worker drains its ready queue and steals from neighbors when idle. Nothing can preempt a task mid-poll — not even the scheduler.

So when a task calls std::fs::read, a synchronous SQL query, argon2 hashing, or a tight CPU loop, the worker is stuck until it returns. Tokio's cooperative budget forces a yield after 128 polls, but only for tasks that actually reach .await. A pure CPU loop never yields at all.

The fix is to move the work. tokio::task::spawn_blocking runs your closure on a separate pool (up to 512 threads by default) and hands the worker straight back.

Where this helps

  • Sync database drivers. Diesel or rusqlite calls inside an axum handler park a worker for the full query time.
  • Login flows. Argon2 is deliberately ~100ms of CPU. On the workers, one login storm eats your runtime.
  • Uploads and media. Resizing a JPEG or parsing a 50MB payload in a handler is blocking in disguise.
  • DNS. std::net::ToSocketAddrs blocks — the classic hidden offender.

Watch out

spawn_blocking is a thread handoff. Past 512 concurrent blocking tasks they queue, so cap expensive ones with a Semaphore. Don't wrap trivial ops — a thread hop for a 2µs read is a net loss. And block_in_place only exists on the multithreaded runtime.

To find stalls, run tokio-console (build with --cfg tokio_unstable) and look for tasks stuck in a single poll.

Try it yourself

Uncomment the BAD line and watch the heartbeat freeze for 300ms.

#[tokio::main(flavor = "current_thread")] // one worker: effect is obvious
async fn main() {
    tokio::spawn(async {
        let mut tick = tokio::time::interval(
            std::time::Duration::from_millis(10));
        loop { tick.tick().await; println!("tick"); }
    });

    // BAD: parks the worker for 300 ms — ticks stutter
    // std::thread::sleep(std::time::Duration::from_millis(300));

    // GOOD: same sleep, on the blocking pool — ticks stay smooth
    tokio::task::spawn_blocking(|| {
        std::thread::sleep(std::time::Duration::from_millis(300));
    }).await.unwrap();
}

TL;DR

  • What: A front-page Tokio post puts the spotlight on async Rust's one unforgivable sin: blocking a worker.
  • Why: One blocking syscall parks a per-core worker and delays every task, timer, and connection behind it.
  • Try: Audit handlers for sync I/O and CPU loops; move them to spawn_blocking, capped by a semaphore.

Read more