Rust async under the hood

Tokio is one of those libraries a lot of Rust developers use long before they really understand it. That is not a criticism. It is just what happens when the entry point is so clean. You add #[tokio::main], your async code runs, and the hard parts disappear behind a macro.

Still, the runtime is doing a huge amount of work for that one line to feel boring. It has to create tasks, schedule them across worker threads, park them when they are waiting on I/O, wake them back up at the right moment, and do all of that without turning coordination overhead into the real bottleneck.

This post is a guided tour of the moving parts that make that possible. Not every internal detail, and not a line by line source code reading. Just the pieces that matter if you want a solid mental model for how Tokio actually runs async Rust in production.

It starts with the #[tokio::main] macro

Most Tokio programs begin with the same trick:

#[tokio::main]
async fn main() {
    // async code
}

That attribute hides two steps.

First, Tokio builds a runtime. Second, it takes your async main body and runs it with block_on. In other words, the macro does not make the process magically async by itself. It creates the machinery that can poll futures, manage tasks, and interact with the operating system, then hands your top-level future to that machinery.

That distinction matters because async Rust is lazy by default. A future does nothing until something polls it. Tokio is the thing that keeps polling at the right times, on the right threads, until the work finishes.

Tasks are the runtime's unit of work

Once the runtime exists, the basic thing it manages is a task.

A Tokio task is a lightweight wrapper around a future. When you call tokio::spawn, you are not creating a new operating system thread. You are asking the runtime to track another asynchronous job that can make progress whenever it is ready to be polled.

That is the first big win. Threads are expensive. Tasks are much cheaper, which is why a single process can handle an enormous number of concurrent operations without needing one thread per request. If you are building a server, that difference is the whole game. Waiting on a database call or socket read should not cost you an entire thread.

This is also where many people's intuition starts to shift. Concurrency in Tokio is not mostly about "doing everything at once." It is about making sure expensive waiting does not pin down valuable execution resources.

The scheduler decides who runs next

If tasks are the jobs, the scheduler is traffic control.

Its job is to decide which runnable task should execute on which worker thread. That sounds simple until you remember that a busy runtime may be juggling thousands or even millions of tasks, and many of them are constantly switching between "ready to run" and "waiting on something else."

Tokio's scheduler is designed around a practical goal: keep worker threads busy without spending too much time coordinating them. A scheduler that makes perfect decisions but burns too many cycles making them would be a bad trade.

This is why Tokio leans so hard on local work, small fast queues, and only escalates to more expensive coordination when it has to.

Worker threads do the actual execution

The executor is the part that actually polls the futures inside those tasks, but in practice it helps to think of the runtime as a pool of worker threads plus the logic that feeds them work.

Each worker thread repeatedly grabs a task, polls it, and sees what happened:

1. The task completed.
2. The task can keep making progress and should be polled again later.
3. The task hit an I/O boundary or another await point and needs to sleep until something wakes it.

That loop is simple to describe, but it only works well if task handoff is cheap. If every thread had to grab a global lock for every little decision, throughput would collapse under contention.

Tokio avoids that by giving workers their own local queues first.

Local queues are the hot path

Each worker thread has a local queue that it can read from very cheaply. This is the fast path of the runtime.

If a thread already has work sitting in its own queue, it does not need to negotiate with everyone else. It just pops the next task and keeps going. That is exactly what you want in a high-throughput system. The common case should stay local and predictable.

Tokio uses a ring-buffer style structure for these local queues. The size is intentionally small enough to stay cache friendly. That is the kind of design choice you only appreciate once you realize how much time fast systems spend winning tiny battles over memory locality.

The important part is not memorizing the queue size. The important part is understanding the philosophy: keep the hot path lock free or close to it, keep it local, and keep it friendly to CPU caches.

The global queue is the shared fallback

Tokio also keeps a global queue. This is the place shared work can land when it does not fit neatly into one worker's local queue or when tasks need a common overflow path.

You can think of the queue model like this:

• Local queues are for speed.
• The global queue is for coordination.

Workers prefer their local queue first. If that is empty, they can check the global queue. Only after that do they start looking sideways at other workers.

That hierarchy is a smart compromise. It avoids forcing every task through one shared structure, but it still gives the runtime a place to put work when local queues fill up or new tasks arrive in a more centralized way.

Work stealing keeps cores from sitting idle

This is the part people usually remember, because it is both clever and easy to picture.

When one worker thread runs out of tasks, Tokio does not immediately leave that thread idle. It can steal work from another worker that still has a healthy backlog.

That matters because real workloads are uneven. One thread may get a burst of ready tasks while another hits a quiet stretch. Without work stealing, some CPU cores would sit around doing nothing while others stay overloaded. With it, the runtime can rebalance itself as the shape of the workload changes.

Work stealing is one of those ideas that sounds obvious after you hear it once. The hard part is implementing it in a way that does not erase its own benefit through synchronization cost. Tokio's scheduler became much stronger once this design matured, and it is a big part of why the runtime scales so well under pressure.

Async I/O only works because tasks can stop cleanly

Everything above explains how runnable tasks get CPU time. It does not explain what happens when a task has to wait.

That is where the reactor enters the picture.

Suppose a task needs to read from a socket, wait on a database response, or receive bytes from some other I/O source. It should not block the worker thread while waiting. If it did, async would stop being useful very quickly.

Instead, the task registers interest in that I/O event and yields control. In practical terms, the future returns Poll::Pending, and the runtime arranges for the task to be woken up later. The worker thread is then free to run something else.

This is one of the core ideas behind async Rust: waiting is normal, but blocking a thread is optional.

The reactor bridges Tokio and the operating system

The reactor, sometimes called the I/O driver, is the piece that listens for operating system events and turns them back into runnable Tokio tasks.

Here is the rough flow:

1. A task hits an await point tied to I/O.
2. Tokio registers interest in the relevant OS event.
3. The task is parked.
4. The worker thread moves on to other work.
5. The OS reports that the I/O event is ready.
6. Tokio wakes the parked task and puts it back in a runnable state.

That wakeup usually happens through a waker, which is Rust's standard mechanism for telling an executor, "this future might make progress now, poll it again."

If the scheduler is the traffic control system, the reactor is the radio tower listening for external signals. It does not run your business logic. It tells the runtime when waiting work has become active work again.

Mio sits underneath that I/O story

Tokio does not talk to Linux, macOS, and Windows event APIs directly in every corner of the runtime. It builds on mio, which provides a thin Rust layer over platform-specific event notification systems.

That abstraction is easy to gloss over, but it is doing useful work. It lets Tokio rely on a common evented I/O foundation instead of reinventing cross-platform readiness handling from scratch. The result is that the runtime can offer one coherent async model while still mapping down to very different operating system primitives underneath.

You do not need to memorize every boundary between Tokio and Mio to benefit from this. It is enough to know that Mio helps Tokio speak the operating system's language for readiness events.

One task's full trip through the runtime

Putting the pieces together helps more than studying them in isolation.

Imagine a web server built with Tokio. A new request arrives, and the runtime creates a task to handle it. That task lands in a queue and is picked up by a worker thread. The executor polls it, and the task starts running your Rust code.

Then the handler needs data from a database.

At that point, the task cannot finish yet. It registers interest in the database socket becoming ready, yields, and gets parked. The worker thread does not wait with it. It goes back to the queue and grabs other work.

Later, the database response arrives. The operating system signals that the socket is readable. The reactor notices, wakes the task, and the scheduler makes it runnable again. Maybe it resumes on the same worker thread. Maybe it resumes on a different one. Either way, it picks up where it left off, continues executing, and eventually completes the response.

That is the runtime in one loop: run, wait, wake, run again.

Why this design feels so good in servers

The obvious benefit is scale. Cheap tasks plus non-blocking I/O let a single process handle a huge number of concurrent connections.

The less obvious benefit is resource discipline. Tokio tries very hard not to waste expensive things. Threads stay busy. Waiting tasks get out of the way. Local queues reduce contention. Work stealing smooths out imbalance. The reactor keeps I/O from freezing execution. None of this is magic. It is just a lot of small decisions that all point in the same direction.

That is also why Tokio has become such a default choice in Rust backends. It gives you a runtime that is fast, but more importantly, it is fast for understandable reasons. Once you see the pieces, the performance story stops feeling mysterious.

The part worth remembering

If you only keep one mental model from this post, make it this one: Tokio is a system for making waiting cheap.

Your async code becomes tasks. Worker threads run the tasks that are ready. Tasks that need I/O step aside instead of blocking. The reactor listens for readiness events, wakes tasks when the OS says they can continue, and the scheduler gets them back onto available workers. Everything else is an optimization around that loop.

That loop is why async Rust can look calm at the surface while doing an absurd amount of coordination under the hood. And honestly, that is what makes Tokio fun to study. The API is simple enough to disappear, but the runtime underneath it is full of very sharp engineering choices.