Pick multiport lanes by inside flow hash, not routine index

Lanes were collapsing onto lane 0 and staying there. Nebula writes a
peer's inbound packets to the tun queue matching the socket they arrived
on, and tun_flow_update teaches the kernel to steer that flow's outbound
packets to the same queue, preferring what it learned over the hash. So
while a tunnel's lanes were still down -- which is every tunnel, for its
first moments -- all traffic arrived on socket 0, pinning every flow to
queue 0 on both hosts for as long as it stayed busy. With the lane
following the routine, that meant lane 0 forever.

Two changes break the loop:

Choose the lane from the packet's own 5-tuple hash, so lane spread no
longer depends on tun steering at all. The hash is symmetric, and the
high-addressed side rotates its choice by the low side's port offset, so
a flow's two directions land on partner lanes with exact reverse
4-tuples and each arrives through the conntrack entry the other's probe
opened.

Seed every lane as demanded at lane-set creation, so the first traffic
tick probes them all at once rather than waiting for a flow to hash onto
each. Lanes have to be up before the flows are, not after.

A routine may now send on any lane, so it holds a SendBatch per lane,
built lazily and all borrowing one shared arena -- the slab is the
expensive part, and a batch per (routine, lane) would otherwise cost
hundreds of megabytes. full() counts across the batches so the total
outstanding stays bounded as before.

That also means several routines can write to one socket, which was
already true for base traffic under multiport: the linux batchWriter's
sendmmsg scratch had no lock. Take a mutex there, once per flush.
This commit is contained in:
Wade Simmons committed 2026-09-03 14:24:53 -04:00
1 parent 4936b40169
commit 53c565eb29
7 files changed
+470 -148

No files matched your search

+10 -10
View File
@@ -110,7 +110,7 @@ func (f *Interface) consumeInsidePacket(pkt tio.Packet, fwPacket *firewall.Parse
dropReason := f.firewall.Drop(fwPacket.Packet, false, hostinfo, f.pki.GetCAPool(), localCache)
if dropReason == nil {
f.sendInsideMessage(hostinfo, pkt, nb, tx)
f.sendInsideMessage(hostinfo, pkt, &fwPacket.Packet, nb, tx)
} else {
f.rejectInside(packet, rejectBuf, q)
if f.l.Enabled(context.Background(), slog.LevelDebug) {
@@ -154,10 +154,10 @@ func (f *Interface) sendInsideEncrypt(hostinfo *HostInfo, ci *ConnectionState, l
// scratch arena: SegmentSuperpacket builds each segment's plaintext in
// segScratch[:segLen] in turn, and we encrypt directly into a fresh SendBatch slot.
//
// When routine q has a usable multiport lane to this peer, the direct path
// swaps to the lane's session and socket below. Relay and base traffic stays on
// When this flow has a usable multiport lane to this peer, the direct path swaps
// to that lane's session and socket below. Relay and base traffic stays on
// tx.base (socket 0).
func (f *Interface) sendInsideMessage(hostinfo *HostInfo, pkt tio.Packet, nb []byte, tx *txQueue) {
func (f *Interface) sendInsideMessage(hostinfo *HostInfo, pkt tio.Packet, fwPacket *firewall.Packet, nb []byte, tx *txQueue) {
ci := hostinfo.ConnectionState
if ci.eKey == nil {
return
@@ -228,21 +228,21 @@ func (f *Interface) sendInsideMessage(hostinfo *HostInfo, pkt tio.Packet, nb []b
return
}
// Direct path: prefer this routine's multiport lane once it is proven
// usable. txLane hands back the lane's session and destination together, so
// Direct path: prefer this flow's multiport lane once it is proven usable.
// txLaneForFlow hands back the lane's session and destination together, so
// there is no window where one is set and the other is not, and a demotion
// drops us back onto the base tunnel on the very next packet.
//
// A miss is also how lanes get probed in the first place: txLane raises
// A miss is also how a lane gets re-probed after a demotion: txLane raises
// demand, which the connection manager's next tick on this tunnel picks up.
// Until the lane is up the traffic rides the base tunnel, the same fallback
// a demoted lane uses.
lane := uint8(0)
if lci, laneRemote := hostinfo.lanes.txLane(tx.laneSlot); lci != nil {
lane = uint8(tx.laneSlot)
if s, lci, laneRemote := hostinfo.lanes.txLaneForFlow(fwPacket); lci != nil {
lane = uint8(s)
ci = lci
remote = laneRemote
sendBatch = tx.lane
sendBatch = tx.laneBatch(f, s)
}
err := tio.SegmentSuperpacket(pkt, func(seg []byte) error {