mirror of
https://github.com/slackhq/nebula.git
synced 2026-11-15 21:56:52 +01:00
tun: Share replay-window reservation credit across readers
Charge staged reservations to each tunnel instead of dividing a static share of the window by routines. Several readers sending on one tunnel then share its credit: the limit is half the peer replay window, assumed from ReplayWindow, which must be a power of two of at least 8192 (checked at compile time). The other half allows finite unreserved traffic (inline and control sends, relay forwarding through SendVia) and network reordering; unlimited bypass traffic or delay can still exceed it. An outstanding-count cap alone would let a fast reader repeatedly send and reacquire credit, overtaking a paused reader without bound. So once readers overlap, spent credit is only recycled when every reservation has drained. A sole ordered reader gets each completion refunded, since its later batches cannot overtake its earlier ones. A reader that fails to acquire ends sole ownership, so a busy owner cannot keep a larger waiter blocked forever. This restores progress, not FIFO fairness. All transitions are a single CAS. Waiting readers flush their own oldest batch before retrying and never hold a partial acquisition: a failed multi-tunnel (relay) acquisition rolls back. Credit is released only after the flush or an explicit drop. A read that alone exceeds a tunnel's credit, such as a tiny-MSS superpacket, is sent segment by segment, synchronously. Acquisition is per read, so pressure on one tunnel also delays the other tunnels in the same read. The combined staged transmit, receive and shared credit measured 15 to 25 Gbit/s over one tunnel with eight TCP streams between two 100 GbE hosts. Tests model overtaking, ordered reuse, a blocked reader behind a busy owner, and relay rollback with both window sizes, and run eight competing readers plus unreserved control traffic on one tunnel, checking every packet is accepted by the receiver's replay window in wire order.
This commit is contained in:
1 parent
db8f9cd2e9
commit
e4f659a207
8 files changed
+427
-28
No files matched your search
@@ -14,6 +14,9 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
// ReplayWindow is the build-time receive window and the assumed peer window
|
||||
// for TX reservation credit. Use a power of two; peers must match or have a
|
||||
// larger receive window. The bitmap costs ReplayWindow/8 bytes per tunnel.
|
||||
ReplayWindow = 8192
|
||||
|
||||
// RehandshakeAfterMessages rolls keys inside the AES-GCM data-volume margin (~2^-36 advantage at 64KB frames).
|
||||
@@ -41,6 +44,7 @@ type ConnectionState struct {
|
||||
peerCert *cert.CachedCertificate
|
||||
initiator bool
|
||||
messageCounter atomic.Uint64
|
||||
txBudget txBudget
|
||||
window *Bits
|
||||
decryptLock sync.Mutex
|
||||
writeLock sync.Mutex
|
||||
|
||||
Reference in new issue
Block a user