decodeRead failures were silently swallowed; a kernel emitting an
unnegotiated GSO type would blackhole all tun traffic with nothing in
the logs. Debug-gated per the usual idiom so the happy path pays
nothing, which means plumbing the logger down through the offload
queueset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014ugV2edVqoz3tBvq9J6yWp
Multi-disciplinary correctness review of the batched tun / GSO-GRO / sendmmsg
rework. Each fix has a regression test; the merged tree builds on
linux/darwin/openbsd/windows/freebsd/netbsd, vets clean, passes the unit and
e2e suites, and is -race clean.
Critical:
- C1 zero-length inner UDP datagram no longer panics the process (remote DoS):
the UDP coalescer routes payLen==0 to passthrough instead of seeding a GSO
slot, and WriteGSO skips empty payload iovecs as defense in depth.
- C2 segmenter no longer corrupts inner headers when gsoSize < headerLen: the
L3+L4 header is snapshotted once and each segment stamped from the copy,
replacing the destructive overlapping in-place slide (SegmentTCP + SegmentUDP).
High:
- H1 applyOuterECN updates the IPv4 header checksum (RFC 1624 incremental) when
folding outer CE into the inner ToS, so passthrough packets are no longer
dropped by the peer stack.
- H2 the GRO reject path caps the borrowed RX segment ([:n:n]) so a reject can
no longer overrun into the next coalesced segment's Nebula header. Note:
oversized ICMPv6 rejects that need >16B beyond the segment are now refused
rather than sent under GRO (safe; see TOFIX.md for the scratch-buffer follow-up).
- H3 WriteBatch falls back to per-packet WriteTo for a chunk when writeSockaddr
fails, so one bad-family destination costs only its own packet, not the batch.
- H4 UserDevice.Readers returns N distinct queue wrappers with private buffers
(sharing the pipes) so concurrent readers no longer race/overwrite borrowed
packet bytes.
- H5 Poll.Close / Offload.Close no longer null t.fd (matching master's
tunFile.Close), removing the data race with a concurrent readOne load.
Medium/Low:
- M1 the UDP GSO 127-segment gate moved from kernel >=5.5 to >=6.9 (the real
UDP_MAX_SEGMENTS 64->128 threshold), avoiding EINVAL + per-packet fallback on
5.5-6.8 kernels.
- M2 NewMultiQueueReader replays the offload mask newTun actually negotiated
instead of the TSO-only mask, so adding a queue no longer disables USO
device-wide; the advertised USO capability derives from the same mask.
- M3 the shutdown eventfd is closed in pollQueueSet.Close / offloadQueueSet.Close
(double-close guarded), fixing the per-lifecycle fd leak.
- M4 dual-stack ECN selects the cmsg by address family, not socket family: RX
parseRecvCmsg reads both IP_TOS and IPV6_TCLASS; TX writeEntryCmsg stamps
IP_TOS for v4/v4-mapped dests and IPV6_TCLASS for v6 (on-host verified).
- L1 newPoll no longer closes the fd on failure (matching newOffload), removing
the double-close on QueueSet.Add error.
remove runtime.LockOSThread() because it makes things worse now
remove the "custom" Write() method from tun_linux.go, the stdlib path via os.File performs better
We should change our guidance around number of routines, ~2 per thread (that you wish to use for Nebula) seems to be about right now
* upgrade to yaml.v3
The main nice fix here is that maps unmarshal into `map[string]any`
instead of `map[any]any`, so it cleans things up a bit.
* add config.AsBool
Since yaml.v3 doesn't automatically convert yes to bool now, for
backwards compat
* use type aliases for m
* more cleanup
* more cleanup
* more cleanup
* go mod cleanup
With the previous implementation, we check if route.MTU is greater than zero,
but it will always be because we set it to the default MTU in
parseUnsafeRoutes. This change leaves it as zero in parseUnsafeRoutes so
it can be examined later.
JackDoan