mirror of
https://github.com/slackhq/nebula.git
synced 2026-11-16 19:16:52 +01:00
Build multiport lanes lazily, on data-plane demand
Lanes were established for every slot as soon as the base tunnel came up, so a host with `routines: N` paid N-1 extra Noise handshakes, sessions and keepalives for every peer it talked to, including peers it exchanges a trickle with. Only routines that actually carry traffic to a peer need a lane. Add a per-slot demand flag to laneState. sendInsideMessage already loads txLanes[laneSlot] on every direct-path packet; that load missing is now the signal, and EnsureLanes only starts slots the data plane asked for. The miss path is unchanged otherwise: the packet rides the base tunnel, the same fallback used while a lane is down. noteLaneDemand load-guards its store so repeated misses on a lane-less slot are plain reads rather than a cache line ping-ponging between the routines sharing it. The demand check is EnsureLanes' last condition, so a slot that is pending or inside its backoff keeps the flag for the tick that can act on it. Consuming the flag also stops a lane whose routine went quiet from being rebuilt forever after it dies. The wire format, lane negotiation and port-offset scheme are untouched, so a lazy node interoperates with an eager one in both directions, and inbound lane handshakes from a busy peer are still accepted regardless of our own demand.
This commit is contained in:
1 parent
192d994273
commit
11528c6e02
6 files changed
+154
-11
No files matched your search
+15
-3
@@ -754,9 +754,16 @@ func (hm *HandshakeManager) maybeAllocLaneState(hostinfo *HostInfo, result *hand
|
||||
}
|
||||
|
||||
// EnsureLanes starts lane handshakes for every empty, non-pending, retry-due
|
||||
// slot of a base tunnel. Called on base handshake completion (both sides) and
|
||||
// from the connection manager's per-tunnel tick, so a dead lane re-establishes
|
||||
// within one check interval, subject to per-slot backoff.
|
||||
// slot of a base tunnel that the data plane has asked for. Called on base
|
||||
// handshake completion (both sides) and from the connection manager's
|
||||
// per-tunnel tick, so a demanded lane is established, and a dead one
|
||||
// re-established, within one check interval subject to per-slot backoff.
|
||||
//
|
||||
// Demand is what makes lanes lazy: nothing is built when the base tunnel
|
||||
// completes, only when a routine actually has traffic for the peer and finds
|
||||
// its slot empty (see noteLaneDemand). Consuming the flag here rather than
|
||||
// leaving it set also means a lane that dies after its routine went quiet
|
||||
// stays dead instead of being rebuilt forever.
|
||||
func (hm *HandshakeManager) EnsureLanes(base *HostInfo) {
|
||||
ls := base.lanes
|
||||
if ls == nil || hm.config.laneCount <= 1 {
|
||||
@@ -771,6 +778,11 @@ func (hm *HandshakeManager) EnsureLanes(base *HostInfo) {
|
||||
if ls.txLanes[i].Load() != nil || ls.txPending[i] || now.Before(ls.txRetryAt[i]) {
|
||||
continue
|
||||
}
|
||||
// Checked last: a slot that isn't startable keeps its demand for the
|
||||
// tick that can act on it.
|
||||
if !ls.takeLaneDemand(i) {
|
||||
continue
|
||||
}
|
||||
ls.txPending[i] = true
|
||||
starts = append(starts, i)
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user