mio — Metal I/O: Non-Blocking Networking in Rust from First Principles

26 min read • Rust Systems Networking & Async Foundations

Every async Rust runtime — Tokio, async-std, smol — is built on top of one foundational layer: the OS event system. On Linux it is epoll, on macOS it is kqueue, on Windows it is IOCP. Writing directly against these APIs is painful, platform-specific, and full of sharp edges.

mio (Metal I/O) is a tiny, cross-platform Rust library that wraps all three behind one clean API. It lets you register interest in I/O events, poll for which events are ready, and respond to them — without any async/await, without a runtime, and with essentially zero overhead. Understanding mio is understanding the engine under every Rust async framework.

We will build a working multi-client TCP echo server step by step. No prior networking or async knowledge assumed.

Part 1: Blocking vs Non-Blocking I/O

The Post Office Analogy

Blocking I/O is like standing at the post office counter waiting for the clerk to find your package. You cannot do anything else — you just wait. If you have 10,000 packages to pick up, you need 10,000 people standing in line simultaneously. That is the “one thread per connection” model: simple but expensive.

Non-blocking I/O is like leaving your phone number at the counter and going about your day. The clerk calls you when your package is ready. One person (one thread) can manage 10,000 packages because they only do work when there is work to do. That is the event-driven model mio enables.

The Cost of Blocking

Blocking TCP server — one thread per connection
use std::net::{TcpListener, TcpStream};
use std::io::{Read, Write};
use std::thread;

fn handle_client(mut stream: TcpStream) {
    let mut buf = [0u8; 1024];
    loop {
        // 'read' BLOCKS here — this thread is frozen until bytes arrive
        // If the client sends nothing for 30 seconds, this thread sits idle
        match stream.read(&mut buf) {
            Ok(0) => break,   // connection closed
            Ok(n) => { stream.write_all(&buf[..n]).unwrap(); }
            Err(_) => break,
        }
    }
}

fn main() {
    let listener = TcpListener::bind("127.0.0.1:8080").unwrap();
    for stream in listener.incoming() {
        let stream = stream.unwrap();
        // Each new connection needs a NEW OS thread
        // 10,000 connections = 10,000 threads = ~80 MB of stack memory
        thread::spawn(|| handle_client(stream));
    }
}

The problem: most of those 10,000 threads are sleeping at any given moment. Each OS thread costs ~8 KB–1 MB of stack memory and a context switch overhead. Non-blocking I/O lets one thread manage all connections, only touching ones that have data right now.

How the OS Event System Works

Modern OSes provide a mechanism to watch many file descriptors (sockets, files, pipes) and wake a thread only when one is ready. On each platform it has a different name and slightly different API:

Linux: epoll

  • epoll_create
  • epoll_ctl — register
  • epoll_wait — block
  • Edge-triggered support
  • Scales to millions of fds

macOS / BSD: kqueue

  • kqueue()
  • kevent — register
  • kevent — wait
  • Watches files, signals, timers
  • Used on iOS/macOS

Windows: IOCP

  • I/O Completion Ports
  • Completion-based model
  • CreateIoCompletionPort
  • GetQueuedCompletionStatus
  • Different mental model

mio's job: hide all three behind one API. You call Poll::new(), registry.register(), and poll.poll() and mio calls the right syscall for your platform.

Part 2: mio Core Concepts

The Airport Control Tower Analogy

Think of mio as an airport control tower. Every aircraft (I/O source: a socket, a file, a pipe) is registered with the tower with a unique flight number (Token) and tells the tower what events it cares about — ready to land (readable) or ready to depart (writable).

The controller (your event loop) calls the tower (Poll) asking “which aircraft are ready?” The tower replies with a list of Events. The controller services only those aircraft, then asks again. No aircraft is ignored, no controller is wasted waiting.

The Four Building Blocks

1. Poll — the event poller
use mio::Poll;

// Poll wraps the OS event system (epoll/kqueue/IOCP).
// Creates one kernel-level event queue.
let poll = Poll::new()?;

// To register sources we use the Registry handle
let registry = poll.registry();
// registry.register(source, token, interest)
2. Token — unique identifier per I/O source
use mio::Token;

// Token is just a newtype wrapper around usize.
// You assign one to each socket/source you register.
// When an event fires, you get back the Token to know which source is ready.

const SERVER: Token = Token(0);     // the listening socket
const CLIENT_BASE: usize = 1;       // clients get Token(1), Token(2), ...

// Typical pattern: use a slab/counter for client tokens
let client_token = Token(CLIENT_BASE + client_id);
3. Interest — what events to watch for
use mio::Interest;

// READABLE: fire when data is available to read
// WRITABLE: fire when the socket can accept more data to write
// You can combine them with the | operator

let read_only  = Interest::READABLE;
let write_only = Interest::WRITABLE;
let both       = Interest::READABLE | Interest::WRITABLE;

// On registration:
registry.register(&mut socket, token, Interest::READABLE)?;

// You can change interest later with reregister:
registry.reregister(&mut socket, token, Interest::READABLE | Interest::WRITABLE)?;
4. Events — the batch of ready notifications
use mio::Events;

// Events is a pre-allocated buffer that poll() fills with ready events.
// Capacity = maximum events returned in one poll() call.
// 128 is a good default; tune up for very high-connection-count servers.
let mut events = Events::with_capacity(128);

loop {
    // Block until at least one event fires (or timeout expires)
    poll.poll(&mut events, None)?;  // None = wait forever

    for event in events.iter() {
        println!(
            "Token {:?} is_readable={} is_writable={}",
            event.token(),
            event.is_readable(),
            event.is_writable()
        );
    }
}

Edge-Triggered vs Level-Triggered

mio uses edge-triggered notifications by default on Linux (epoll ET). This is a critical detail:

Level-Triggered (select, poll)

Fires on every call to poll() as long as the condition is true (e.g. there is still data to read). Easier to program but less efficient.

Edge-Triggered (epoll ET, mio default)

Fires once when the condition changes (e.g. new data arrives). You must read all available data in a loop until you get WouldBlock. More efficient, but you must not leave data unread.

The correct mio read pattern — loop until WouldBlock
use std::io::{self, Read};

fn read_all(stream: &mut mio::net::TcpStream, buf: &mut Vec<u8>) -> io::Result<bool> {
    let mut tmp = [0u8; 4096];
    loop {
        match stream.read(&mut tmp) {
            Ok(0) => return Ok(false),  // connection closed gracefully
            Ok(n) => buf.extend_from_slice(&tmp[..n]), // got n bytes, keep reading
            Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
                // No more data right now — we've drained the socket buffer
                // STOP reading. mio will fire the event again when more data arrives.
                return Ok(true);
            }
            Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {
                // Signal interrupted the syscall — retry immediately
                continue;
            }
            Err(e) => return Err(e),
        }
    }
}

Golden rule: when you receive a readable event, keep calling read() in a loop until you get WouldBlock. If you only read once and stop, the remaining data sits in the kernel buffer and mio will never fire the event again for that data (edge-triggered).

Part 3: Project Setup

Create project and add mio
cargo new mio-echo --bin
cd mio-echo
cargo add mio --features net,os-poll
Cargo.toml — mio feature flags explained
[dependencies]
mio = { version = "1", features = ["net", "os-poll"] }

# Feature flags:
#   net     — enables TcpListener, TcpStream, UdpSocket
#   os-poll — enables Poll (the OS event poller)
#   os-ext  — enables platform-specific extras (e.g. pipe, unix sockets)

# mio is deliberately minimal. You only pay for what you enable.

Part 4: Building a TCP Echo Server Step by Step

An echo server reads whatever a client sends and sends it right back. It is the “Hello, world!” of networking. We will build one that handles many simultaneous clients using a single thread and mio.

Step 1 — Create and Register the Listening Socket

use mio::net::TcpListener;
use mio::{Events, Interest, Poll, Token};
use std::net::SocketAddr;

const SERVER: Token = Token(0);

fn main() -> std::io::Result<()> {
    // Step 1: Create the event poller
    let mut poll = Poll::new()?;
    let mut events = Events::with_capacity(128);

    // Step 2: Create a non-blocking TCP listener
    // mio::net::TcpListener is non-blocking by default — unlike std::net::TcpListener
    let addr: SocketAddr = "127.0.0.1:9000".parse().unwrap();
    let mut server = TcpListener::bind(addr)?;

    // Step 3: Register the listener with the poll — we want to know when
    // a new connection is ready to accept (READABLE on a listener = new client)
    poll.registry().register(
        &mut server,
        SERVER,         // token: when an event fires, we get Token(0) back
        Interest::READABLE,
    )?;

    println!("Echo server listening on {}", addr);
    // ... event loop comes next
    Ok(())
}

Step 2 — Accept New Connections

When the listener becomes readable it means one or more clients are waiting. Because we are edge-triggered, we must call accept() in a loop until we get WouldBlock.

use std::collections::HashMap;
use mio::net::TcpStream;

// We keep a map from Token → TcpStream so we can look up which stream
// fired when we receive an event
let mut connections: HashMap<Token, TcpStream> = HashMap::new();
let mut next_token: usize = 1; // SERVER = 0, clients start at 1

fn accept_connections(
    server: &TcpListener,
    poll: &Poll,
    connections: &mut HashMap<Token, TcpStream>,
    next_token: &mut usize,
) -> std::io::Result<()> {
    loop {
        match server.accept() {
            Ok((mut stream, addr)) => {
                println!("New client from {}", addr);
                let token = Token(*next_token);
                *next_token += 1;

                // Register the new client socket for readable events
                poll.registry().register(
                    &mut stream,
                    token,
                    Interest::READABLE,
                )?;

                connections.insert(token, stream);
            }
            Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
                // No more pending connections — stop accepting
                break;
            }
            Err(e) => return Err(e),
        }
    }
    Ok(())
}

Step 3 — Echo Data Back to the Client

use std::io::{Read, Write};

fn handle_client(
    token: Token,
    stream: &mut TcpStream,
) -> std::io::Result<bool> {
    let mut received = Vec::with_capacity(4096);

    // Read all available data (loop until WouldBlock)
    let mut tmp = [0u8; 1024];
    loop {
        match stream.read(&mut tmp) {
            Ok(0) => return Ok(false),  // client disconnected
            Ok(n) => received.extend_from_slice(&tmp[..n]),
            Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => break,
            Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => continue,
            Err(e) => return Err(e),
        }
    }

    if received.is_empty() {
        return Ok(true);
    }

    // Echo everything back
    // write_all might not send all bytes in one call — loop until done
    let mut written = 0;
    while written < received.len() {
        match stream.write(&received[written..]) {
            Ok(0) => return Ok(false), // can't write more
            Ok(n) => written += n,
            Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
                // Socket buffer is full — in a real server you'd buffer
                // the unsent bytes and re-register for WRITABLE
                // For echo simplicity we just retry
                continue;
            }
            Err(e) => return Err(e),
        }
    }

    println!(
        "[token {:?}] echoed {} bytes: {:?}",
        token,
        written,
        std::str::from_utf8(&received).unwrap_or("<binary>")
    );
    Ok(true) // connection still alive
}

Step 4 — The Complete Event Loop

Now we wire everything together in the main event loop. This is the core pattern for any mio application: poll → dispatch → repeat.

src/main.rs — complete echo server
use mio::net::{TcpListener, TcpStream};
use mio::{Events, Interest, Poll, Token};
use std::collections::HashMap;
use std::io::{self, Read, Write};
use std::net::SocketAddr;

const SERVER: Token = Token(0);

fn main() -> io::Result<()> {
    let mut poll = Poll::new()?;
    let mut events = Events::with_capacity(128);

    let addr: SocketAddr = "127.0.0.1:9000".parse().unwrap();
    let mut server = TcpListener::bind(addr)?;
    poll.registry().register(&mut server, SERVER, Interest::READABLE)?;

    let mut connections: HashMap<Token, TcpStream> = HashMap::new();
    let mut next_id: usize = 1;

    println!("Echo server on {}", addr);
    println!("Test with: nc 127.0.0.1 9000");

    loop {
        // Block until the OS has at least one ready event
        poll.poll(&mut events, None)?;

        for event in events.iter() {
            match event.token() {
                SERVER => {
                    // New connection(s) waiting — accept all of them
                    loop {
                        match server.accept() {
                            Ok((mut stream, addr)) => {
                                let token = Token(next_id);
                                next_id += 1;
                                poll.registry().register(
                                    &mut stream,
                                    token,
                                    Interest::READABLE,
                                )?;
                                connections.insert(token, stream);
                                println!("[{:?}] connected from {}", token, addr);
                            }
                            Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => break,
                            Err(e) => return Err(e),
                        }
                    }
                }

                token => {
                    // Data ready on an existing client
                    let done = if let Some(stream) = connections.get_mut(&token) {
                        let mut buf = Vec::with_capacity(4096);
                        let mut tmp = [0u8; 1024];
                        let alive = loop {
                            match stream.read(&mut tmp) {
                                Ok(0) => break false, // EOF
                                Ok(n) => buf.extend_from_slice(&tmp[..n]),
                                Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
                                    break true
                                }
                                Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {
                                    continue
                                }
                                Err(e) => return Err(e),
                            }
                        };

                        if !buf.is_empty() {
                            stream.write_all(&buf)?;
                            println!(
                                "[{:?}] echoed {} bytes",
                                token,
                                buf.len()
                            );
                        }
                        !alive
                    } else {
                        false
                    };

                    if done {
                        // Deregister the socket before dropping it
                        if let Some(mut stream) = connections.remove(&token) {
                            poll.registry().deregister(&mut stream)?;
                            println!("[{:?}] disconnected", token);
                        }
                    }
                }
            }
        }
    }
}
Test it with netcat
# Terminal 1: run the server
cargo run --release

# Terminal 2: connect with netcat and type anything
nc 127.0.0.1 9000
hello, mio!        ← you type this
hello, mio!        ← server echoes it back

# Terminal 3: open a second simultaneous client
nc 127.0.0.1 9000
another client     ← both clients handled by the SAME single thread

Part 5: Handling Backpressure with Write Buffers

The Postal Sorting Office Analogy

Sometimes a client is slow to receive data. The postal truck (socket write buffer) is full. If you insist on delivering right now you block everyone. The solution is a sorting office (write buffer in your code): hold the outgoing mail here, register interest in WRITABLE events, and flush when the truck has space. Once flushed, deregister WRITABLE interest to avoid busy-looping on a socket that is always technically writable when idle.

Connection State with a Write Buffer

Buffered connection — the production pattern
use mio::net::TcpStream;
use mio::{Interest, Registry, Token};
use std::io::{self, Read, Write};

struct Connection {
    stream:    TcpStream,
    token:     Token,
    write_buf: Vec<u8>,   // bytes waiting to be sent
    closed:    bool,
}

impl Connection {
    fn new(stream: TcpStream, token: Token) -> Self {
        Connection { stream, token, write_buf: Vec::new(), closed: false }
    }

    /// Queue bytes to send. Attempts immediate flush; buffers the rest.
    fn send(&mut self, registry: &Registry, data: &[u8]) -> io::Result<()> {
        self.write_buf.extend_from_slice(data);
        self.flush_write_buf(registry)
    }

    /// Try to drain write_buf into the socket.
    fn flush_write_buf(&mut self, registry: &Registry) -> io::Result<()> {
        while !self.write_buf.is_empty() {
            match self.stream.write(&self.write_buf) {
                Ok(0) => { self.closed = true; break; }
                Ok(n) => { self.write_buf.drain(..n); }
                Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
                    // Socket buffer full — register WRITABLE so we get called
                    // back when there's space
                    registry.reregister(
                        &mut self.stream,
                        self.token,
                        Interest::READABLE | Interest::WRITABLE,
                    )?;
                    return Ok(());
                }
                Err(e) => return Err(e),
            }
        }

        // All bytes sent — stop watching WRITABLE to avoid busy-polling
        if self.write_buf.is_empty() {
            registry.reregister(
                &mut self.stream,
                self.token,
                Interest::READABLE,
            )?;
        }
        Ok(())
    }

    /// Handle a readable event: read all available data.
    fn read(&mut self, buf: &mut Vec<u8>) -> io::Result<bool> {
        let mut tmp = [0u8; 4096];
        loop {
            match self.stream.read(&mut tmp) {
                Ok(0) => return Ok(false),
                Ok(n) => buf.extend_from_slice(&tmp[..n]),
                Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => return Ok(true),
                Err(ref e) if e.kind() == io::ErrorKind::Interrupted => continue,
                Err(e) => return Err(e),
            }
        }
    }
}

Part 6: UDP with mio

mio works equally well with UDP. Unlike TCP there is no connection state — each datagram is independent. The pattern is simpler: one socket, one token, receive datagrams in a loop.

UDP echo server with mio
use mio::net::UdpSocket;
use mio::{Events, Interest, Poll, Token};

const SOCKET: Token = Token(0);

fn main() -> std::io::Result<()> {
    let mut poll = Poll::new()?;
    let mut events = Events::with_capacity(32);

    let mut socket = UdpSocket::bind("127.0.0.1:9001".parse().unwrap())?;
    poll.registry().register(&mut socket, SOCKET, Interest::READABLE)?;

    println!("UDP echo server on 127.0.0.1:9001");
    println!("Test with: echo 'hello' | nc -u 127.0.0.1 9001");

    let mut buf = [0u8; 2048];

    loop {
        poll.poll(&mut events, None)?;

        for event in events.iter() {
            if event.token() == SOCKET && event.is_readable() {
                // Receive datagrams in a loop until WouldBlock
                loop {
                    match socket.recv_from(&mut buf) {
                        Ok((n, sender)) => {
                            println!("UDP: {} bytes from {}", n, sender);
                            // Echo back to sender
                            socket.send_to(&buf[..n], sender)?;
                        }
                        Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => break,
                        Err(e) => return Err(e),
                    }
                }
            }
        }
    }
}

Part 7: mio vs Tokio — When to Use Which

The Car Engine Analogy

mio is the engine block. Tokio is the complete car. Most developers should drive the car — they get steering wheel, brakes, air conditioning, and a GPS. A handful of developers who build racecars, forklifts, or submarines work directly on the engine. mio is for the latter: when you need total control over the I/O layer, or when you are building a runtime itself.

PropertymioTokio
LevelLow-level (OS events)High-level (async runtime)
API styleSynchronous event loopasync / .await
ThreadingYou manage threadsThread pool managed for you
FuturesNoYes (built on mio)
Binary sizeTiny (~30 KB)Larger (~500 KB+)
ErgonomicsManual, verboseHigh — ecosystem of crates
Use whenBuilding runtimes, embedded, custom protocolsBuilding applications and services

What Tokio Does on Top of mio

Tokio wraps mio with three additional layers:

  • Task scheduler: turns async fn state machines into tasks and runs them on a thread pool. mio has no concept of tasks.
  • Waker integration: when mio fires an event, Tokio wakes the specific Future waiting on that socket and re-polls only it. In raw mio you do this manually with your Token → handler map.
  • Timer wheel: mio's poll(timeout) gives you one global timeout. Tokio adds a hashed timer wheel for millions of individual per-task timeouts.

Common Patterns & Checklist

Patterns Summary

Skeleton of any mio application
// 1. Create Poll and Events buffer
let mut poll = Poll::new()?;
let mut events = Events::with_capacity(128);

// 2. Create and register I/O sources
poll.registry().register(&mut source, TOKEN, Interest::READABLE)?;

// 3. Event loop
loop {
    // 4. Block until something is ready (pass Some(duration) for timeout)
    poll.poll(&mut events, None)?;

    // 5. Dispatch each event
    for event in events.iter() {
        match event.token() {
            LISTENER_TOKEN => { /* accept loop */ }
            client_token => {
                // 6. Read loop until WouldBlock
                // 7. Write (possibly with write buffer)
                // 8. Deregister + remove on disconnect
            }
        }
    }
}

Do This

  • ✓Loop until WouldBlock on every read and accept
  • ✓Deregister before dropping a socket (registry.deregister)
  • ✓Use a write buffer + WRITABLE interest for backpressure
  • ✓Handle Interrupted by retrying the syscall
  • ✓Use a HashMap<Token, _> or slab for O(1) connection lookup

Avoid This

  • ✗Reading only once per event (leaves data in kernel buffer)
  • ✗Registering WRITABLE permanently (causes busy-looping when the buffer is empty)
  • ✗Blocking inside the event loop (thread::sleep, blocking DNS, file I/O)
  • ✗Reusing Token values for new connections (can cause phantom events)
  • ✗Ignoring errors from register / deregister

mio is a thin, honest abstraction over the OS event system. There is no magic and very little overhead — what you write is almost exactly what the kernel executes. That directness makes it the right foundation when you need to understand exactly what is happening, build a custom protocol, or write a runtime. Once you are comfortable with the event loop pattern — register → poll → dispatch → WouldBlock — you will also understand why Tokio, async-std, and every other Rust async runtime works the way it does. Happy hacking!