mirror of
https://github.com/slackhq/nebula.git
synced 2026-08-16 09:06:59 +02:00
datapath: fix 12 correctness findings from tun/UDP offload review
Multi-disciplinary correctness review of the batched tun / GSO-GRO / sendmmsg rework. Each fix has a regression test; the merged tree builds on linux/darwin/openbsd/windows/freebsd/netbsd, vets clean, passes the unit and e2e suites, and is -race clean. Critical: - C1 zero-length inner UDP datagram no longer panics the process (remote DoS): the UDP coalescer routes payLen==0 to passthrough instead of seeding a GSO slot, and WriteGSO skips empty payload iovecs as defense in depth. - C2 segmenter no longer corrupts inner headers when gsoSize < headerLen: the L3+L4 header is snapshotted once and each segment stamped from the copy, replacing the destructive overlapping in-place slide (SegmentTCP + SegmentUDP). High: - H1 applyOuterECN updates the IPv4 header checksum (RFC 1624 incremental) when folding outer CE into the inner ToS, so passthrough packets are no longer dropped by the peer stack. - H2 the GRO reject path caps the borrowed RX segment ([:n:n]) so a reject can no longer overrun into the next coalesced segment's Nebula header. Note: oversized ICMPv6 rejects that need >16B beyond the segment are now refused rather than sent under GRO (safe; see TOFIX.md for the scratch-buffer follow-up). - H3 WriteBatch falls back to per-packet WriteTo for a chunk when writeSockaddr fails, so one bad-family destination costs only its own packet, not the batch. - H4 UserDevice.Readers returns N distinct queue wrappers with private buffers (sharing the pipes) so concurrent readers no longer race/overwrite borrowed packet bytes. - H5 Poll.Close / Offload.Close no longer null t.fd (matching master's tunFile.Close), removing the data race with a concurrent readOne load. Medium/Low: - M1 the UDP GSO 127-segment gate moved from kernel >=5.5 to >=6.9 (the real UDP_MAX_SEGMENTS 64->128 threshold), avoiding EINVAL + per-packet fallback on 5.5-6.8 kernels. - M2 NewMultiQueueReader replays the offload mask newTun actually negotiated instead of the TSO-only mask, so adding a queue no longer disables USO device-wide; the advertised USO capability derives from the same mask. - M3 the shutdown eventfd is closed in pollQueueSet.Close / offloadQueueSet.Close (double-close guarded), fixing the per-lifecycle fd leak. - M4 dual-stack ECN selects the cmsg by address family, not socket family: RX parseRecvCmsg reads both IP_TOS and IPV6_TCLASS; TX writeEntryCmsg stamps IP_TOS for v4/v4-mapped dests and IPV6_TCLASS for v6 (on-host verified). - L1 newPoll no longer closes the fd on failure (matching newOffload), removing the double-close on QueueSet.Add error.
This commit is contained in:
@@ -439,10 +439,22 @@ func (r *Offload) WriteGSO(hdr []byte, transportHdr []byte, pays [][]byte, proto
|
||||
r.gsoIovs[1].SetLen(len(hdr))
|
||||
r.gsoIovs[2].Base = &transportHdr[0]
|
||||
r.gsoIovs[2].SetLen(len(transportHdr))
|
||||
for i, p := range pays {
|
||||
r.gsoIovs[3+i].Base = &p[0]
|
||||
r.gsoIovs[3+i].SetLen(len(p))
|
||||
// Defense in depth: an empty payload fragment can't be a valid GSO
|
||||
// segment and &p[0] would panic on it. Callers route zero-length
|
||||
// datagrams through the plain path (see UDPCoalescer.commitParsed), so
|
||||
// this should never fire, but skip empties rather than index into one.
|
||||
// `n` tracks where the next payload iovec lands, since skips make it
|
||||
// drift from 3+i.
|
||||
n := 3
|
||||
for _, p := range pays {
|
||||
if len(p) == 0 {
|
||||
continue
|
||||
}
|
||||
r.gsoIovs[n].Base = &p[0]
|
||||
r.gsoIovs[n].SetLen(len(p))
|
||||
n++
|
||||
}
|
||||
r.gsoIovs = r.gsoIovs[:n]
|
||||
|
||||
_, err := r.rawWrite(r.gsoIovs)
|
||||
return err
|
||||
@@ -454,11 +466,9 @@ func (r *Offload) Close() error {
|
||||
}
|
||||
|
||||
//shutdownFd is owned by the container, so we should not close it
|
||||
var err error
|
||||
if r.fd >= 0 {
|
||||
err = unix.Close(r.fd)
|
||||
r.fd = -1
|
||||
}
|
||||
|
||||
return err
|
||||
// Close the underlying fd but do NOT null r.fd: a reader may still be
|
||||
// loading it in readOne, and mutating the field would race that load.
|
||||
// It gets EBADF -> os.ErrClosed (or wakes via the shutdown eventfd's
|
||||
// ppoll first). closed.Swap already guarantees we only close once.
|
||||
return unix.Close(r.fd)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user