tun: Hold one transmit mode per routine

With staged transmit, an idle reader encrypted and sent a small read
itself, while larger reads went to the workers and left from the
ordered sender. Each read chose on its own, so under load a routine
alternated between the two paths. Their packets left from different
CPUs, and the receiver's TCP saw them out of order: on 100 GbE, summed
over four hosts, dsack_dups rose from 0-1 on master to about 25k and
reord_seen from 0 to 13k-19k.

Each routine now holds one mode, inline on the reader or fanout to the
workers and the sender. It switches only after tun.staged.gate_hold
(default 64) reads in a row want the other mode, and before going
inline it waits until every batch in flight is on the wire, so a
switch never lets a later packet overtake an earlier one. Going to
fanout nothing is in flight. flow.tx.gate_switches counts the switches.
On the same hosts dsack_dups fell to 20-52 and reord_seen to 0-2,
while throughput matched or beat per-read mixing (many-to-one 34.9
instead of 30.7 Gbit/s).

The affinity test waited only for the previous batch's packets to be
captured before the next read, but the sender returns the slot after
its write, so the reader could still see it in flight and stage the
small read. Switching to inline now drains, which makes the inline
send deterministic there.

examples/config.yml

@@ -284,9 +284,14 @@ tun:
   # Staged reservations may use half the window, the other half absorbs inline and control traffic, relay forwarding
   # and network reordering; unlimited forwarding or delay can still exceed it. Waiting for credit on a busy tunnel
   # also delays the other tunnels of the same read, and readers are not guaranteed fairness.
-  # Idle batches below inline_below TX segments or RX packets stay on the reader, default 256. With multiple routines,
-  # those sends still take tunnel reservation credit. 0 hands every batch to the workers and the ordered sender or
-  # writer; only TX reads too large for a tunnel's whole credit are still sent by the reader, after it drains.
+  # Small batches, below inline_below TX segments or RX packets (default 256), can skip the workers. On transmit each
+  # routine holds one mode, inline on the reader or fanout to the workers and the ordered sender, and switches only
+  # after gate_hold (default 64) reads in a row want the other one. Before going inline it waits for every batch in
+  # flight to reach the wire, so later packets never overtake earlier ones; flow.tx.gate_switches counts the switches.
+  # Choosing per read instead let small inline sends leave from another CPU than staged ones, and TCP saw reordering.
+  # On receive a listener with nothing in flight writes a small batch itself. With multiple routines, inline sends
+  # still take tunnel reservation credit. 0 hands every batch to the workers and the ordered sender or writer; only TX
+  # reads too large for a tunnel's whole credit are still sent by the reader, after it drains.
   # tx: false keeps transmit inline and stages only receive.
   # tx_slots and rx_slots bound the batches each routine has in flight per direction, minimum 2. A TX slot holds one
   # tun read and its ciphertext, about 1.1 MiB; the default is 8, or workers + 2 if more. An RX slot holds one
@@ -296,6 +301,7 @@ tun:
   #  workers: 0
   #  tx: true
   #  inline_below: 256
+  #  gate_hold: 64
   #  tx_slots: 8
   #  rx_slots: 6