mirror of
https://github.com/slackhq/nebula.git
synced 2026-11-15 18:16:52 +01:00
tun: Share replay-window reservation credit across readers
Charge staged reservations to each tunnel instead of dividing a static share of the window by routines. Several readers sending on one tunnel then share its credit: the limit is half the peer replay window, assumed from ReplayWindow, which must be a power of two of at least 8192 (checked at compile time). The other half allows finite unreserved traffic (inline and control sends, relay forwarding through SendVia) and network reordering; unlimited bypass traffic or delay can still exceed it. An outstanding-count cap alone would let a fast reader repeatedly send and reacquire credit, overtaking a paused reader without bound. So once readers overlap, spent credit is only recycled when every reservation has drained. A sole ordered reader gets each completion refunded, since its later batches cannot overtake its earlier ones. A reader that fails to acquire ends sole ownership, so a busy owner cannot keep a larger waiter blocked forever. This restores progress, not FIFO fairness. All transitions are a single CAS. Waiting readers flush their own oldest batch before retrying and never hold a partial acquisition: a failed multi-tunnel (relay) acquisition rolls back. Credit is released only after the flush or an explicit drop. A read that alone exceeds a tunnel's credit, such as a tiny-MSS superpacket, is sent segment by segment, synchronously. Acquisition is per read, so pressure on one tunnel also delays the other tunnels in the same read. The combined staged transmit, receive and shared credit measured 15 to 25 Gbit/s over one tunnel with eight TCP streams between two 100 GbE hosts. Tests model overtaking, ordered reuse, a blocked reader behind a busy owner, and relay rollback with both window sizes, and run eight competing readers plus unreserved control traffic on one tunnel, checking every packet is accepted by the receiver's replay window in wire order.
This commit is contained in:
1 parent
db8f9cd2e9
commit
e4f659a207
8 files changed
+427
-28
No files matched your search
@@ -281,6 +281,9 @@ tun:
|
||||
# Receive needs a listener that can hand off its batches, which Linux has.
|
||||
# IMPORTANT: staged senders require peers with at least the 8192-entry replay window of this release. Released
|
||||
# v1.11.x peers use 1024 and are incompatible. The window is not negotiated.
|
||||
# Staged reservations may use half the window, the other half absorbs inline and control traffic, relay forwarding
|
||||
# and network reordering; unlimited forwarding or delay can still exceed it. Waiting for credit on a busy tunnel
|
||||
# also delays the other tunnels of the same read, and readers are not guaranteed fairness.
|
||||
#staged:
|
||||
# workers: 0
|
||||
|
||||
|
||||
Reference in new issue
Block a user