tun: Keep relayed data on the reader transmit path

The sticky gate only drains socket submissions, so moving relay data between reader and staged sender CPUs can reorder packets on independent NIC queues. Keep relayed batches on the reader with separate, shared counter reservations while preserving the direct gate and sender affinity.

Cover gate switches, oversized relay credit, mixed direct placement and stalled shared-tunnel reservations. Add Forgejo build, test, race, crypto, lint and artifact jobs on incus-docker.
This commit is contained in:
Ryan Huber committed 2026-10-18 14:59:21 +00:00
1 parent 5cbbba254e
commit a784af4026
6 files changed
+307 -26

No files matched your search

+5 -4
View File
@@ -287,11 +287,12 @@ tun:
# Small batches, below inline_below TX segments or RX packets (default 256), can skip the workers. On transmit each
# routine holds one mode, inline on the reader or fanout to the workers and the ordered sender, and switches only
# after gate_hold (default 64) reads in a row want the other one. Before going inline it waits for every batch in
# flight to reach the wire, so later packets never overtake earlier ones; flow.tx.gate_switches counts the switches.
# Choosing per read instead let small inline sends leave from another CPU than staged ones, and TCP saw reordering.
# flight to be submitted to the socket; flow.tx.gate_switches counts the switches. This does not drain NIC queues.
# Relay data always stays on the reader, so gate switches and oversized reads cannot move it between NIC queues.
# Direct data retains the adaptive gate and independent sender placement.
# On receive a listener with nothing in flight writes a small batch itself. With multiple routines, inline sends
# still take tunnel reservation credit. 0 hands every batch to the workers and the ordered sender or writer; only TX
# reads too large for a tunnel's whole credit are still sent by the reader, after it drains.
# still take tunnel reservation credit. 0 hands direct TX and all RX batches to the workers and ordered stages;
# relayed TX and reads too large for a tunnel's whole credit are still sent by the reader.
# tx: false keeps transmit inline and stages only receive.
# tx_slots and rx_slots bound the batches each routine has in flight per direction, minimum 2. A TX slot holds one
# tun read and its ciphertext, about 1.1 MiB; the default is 8, or workers + 2 if more. An RX slot holds one