dfir_rs/scheduled/net/
mod.rs

1//! This module previously contained networking code.
2//!
3//! ## How Tokio interacts with the DFIR runtime (Mingwei 2021-12-07)
4//!
5//! [Tokio](https://tokio.rs/) is a Rust async runtime. In Rust's async/await
6//! system, `Future`s must be spawned or sent to an async runtime in order to
7//! run. Tokio is the most popular provider of one of these runtimes, with
8//! [async-std](https://async.rs/) (mirrors std lib) and [smol](https://github.com/smol-rs/smol)
9//! (minimal runtime) as commonly used alternatives.
10//!
11//! Fundamentally, an async runtime's job is to poll futures (read: run tasks)
12//! when they are ready to make progress. However async runtimes also provide a
13//! `Future`s abstraction for async events such as timers, network IO, and
14//! filesystem IO. To do this, [Tokio](https://tokio.rs/) uses [Mio](https://github.com/tokio-rs/mio)
15//! which is a low-level non-blocking API for IO event notification/polling.
16//! A user of Mio can write an event loop, i.e. something like: wait for
17//! events, run computations responding to those events, repeat. Tokio provides
18//! the higher-level async/await slash `Future` abstraction on top of Mio, as
19//! well as the runtime to execute those `Future`s. Essentially, the Tokio
20//! async runtime essentially replaces the low-level event loop a user might
21//! handwrite when using Mio.
22//!
23//! For context, both Mio and Tokio provide socket/UDP/TCP-level network
24//! abstractions, which is probably the right layer for us. There are also
25//! libraries built on top of Tokio providing nice server/client HTTP APIs
26//! like [Hyper](https://hyper.rs/).
27//!
28//! The DFIR scheduled layer scheduler is essentially the same as a simple
29//! event loop: it runs subgraphs when they have data. We have also let it
30//! respond to external asynchonous events by providing a threadsafe channel
31//! through which subgraphs can be externally scheduled.
32//!
33//! In order to add networking to DFIR, in our current implementation we
34//! use Tokio and have a compatibility mechanism for working with `Future`s.
35//! A `Future` provides a `Waker` mechanism to notify when it had work to do,
36//! so we have hooked these Wakers up with DFIR's threadsafe external
37//! scheduling channel. This essentially turns DFIR into a simple async
38//! runtime.
39//!
40//! However in some situations, we still need to run futures outside of
41//! DFIR's basic runtime. It's not a goal for DFIR to provide all
42//! the features of a full runtime like Tokio. Currently for this situation we
43//! run DFIR as a task (`Future`) within the Tokio runtime. In DFIR's
44//! event loop we do all available work, then rather than block and wait for
45//! external events to schedule more tasks, we temporarily yield back to the
46//! Tokio runtime. Tokio will then respond to any outstanding events it has
47//! before once again running the DFIR scheduler task.
48//!
49//! This works perfectly well but maybe isn't the best solution long-term.
50//! In the future we may want to remove the extra Tokio runtime layer and
51//! interface with Mio directly. In this case we would have to do our own
52//! socket-style polling within the DFIR scheduler's event loop, which
53//! would require some extra work and thought. But for now interfacing with
54//! Tokio works and I don't think the overhead of the extra runtime loop is
55//! significant.