With staging enabled, a reader whose pipeline is empty handles a batch below
256 TX segments or RX packets itself, without a worker handoff or buffer swap.
The threshold is the measured crossover, not another configuration knob.
Inline work only starts once that reader's ordered work has completed, so
ownership and order hold across switches.
With several readers, transmit still takes shared tunnel credit and the same
reservations as staged batches, releasing it after the synchronous flush; a
sole reader with an empty pipeline has nothing to overtake and uses the
original inline path. Small batches a reader sends itself run on the reader's
CPU, so they can use a different XPS TX ring than its sender.
Metrics: flow.tx.inline and flow.rx.inline count reads handled on the reader
(TX inline also counts oversized reads).
With receive IRQ CPUs kept free of Nebula threads, this improved one TCP
stream over 100 GbE from 16.7 to 21.1 Gbit/s; that measurement does not isolate
the threshold. Tests cover the switch in both directions, ordering behind an
in-flight batch, a paused reader against repeated small sends on another,
RecvError handling on the listener, and send affinity across mode switches.
With tun.staged.workers set, each Linux UDP listener classifies a recvmmsg
batch, hands its data packets to that many decrypt workers, and an ordered
writer commits the batch in arrival order: replay window, roaming, firewall,
control packets and tun writes go through the same helpers inline receive
uses. Six batches per routine bound memory and backpressure. Conns that cannot
hand off their buffers stay inline.
The listener's classification is speculative. Before touching replay state the
writer resolves each data packet through its own batch-local hostmap cache,
exactly where inline receive would, so data after a close in the same batch is
still delivered while data after a removal or replacement between batches is
dropped, with the same RecvError reply inline receive sends. Known replays are
marked by a precheck and skip decryption, but keep their place so that reply
stays identical when an earlier RecvError deleted their tunnel.
New API: udp.StdConn.ListenOutHandoff, and the same on TesterConn for e2e,
runs the receive loop while handing each batch's buffers to the caller in
exchange for replacements. It is found through an unexported interface in
nebula, so udp.Conn and its other implementations are unchanged.
Metrics: flow.rx.batches, and flow.stage.{rx_read,rx_decrypt,rx_write}.q<N>.busy_ns.
Tests cover both ciphers, ordering, duplicate and corrupted packets from other
addresses, window jumps, control packets mid-batch, RecvError then replay,
tunnel replacement, firewall parity, fallback, real UDP sockets, partial
batches at close, and a handshake and traffic through e2e that checks pipeline
use on Linux and the inline fallback elsewhere. No isolated receive throughput
claim is made.
* firewall can distinguish if the host connecting has an overlapping network, is a VPN peer without an overlapping network, or is a unsafe network
* Cross stack subnet stuff (#1512)
* experiment with not filtering out non-common addresses in hostinfo.networks
* allow handshakes without overlaps
* unsafe network test
* change HostInfo.buildNetworks argument to reference the cert
* try to make certificate addition/removal reloadable in some cases
* very spicy change to respond to handshakes with cert versions we cannot match with a cert that we can indeed match
* even spicier change to rehandshake if we detect our cert is lower-version than our peer, and we have a newer-version cert available
* make tryRehandshake easier to understand
Ryan Huber