ListenOut drains the socket with recvmsg_x, xnu's private batched
recvmsg (SYS_RECVMSG_X, bsd/sys/socket_private.h), up to 64 datagrams
per syscall, and calls flush once per batch instead of once per packet.
Each msghdr_x carries its own sockaddr, so this works on the shared
unconnected socket. The send side stays on sendto: xnu only batches
sendmsg_x through the stack on connected sockets, and on an unconnected
socket like this one it falls back to one sendit per message, with
msg_name documented as unsupported.
recvmsg_x is private, so ListenOut falls back to one
ReadFromUDPAddrPort per datagram, logging once, if the kernel or a
sandbox refuses it (ENOSYS, EPERM, EOPNOTSUPP) or if a batch comes back
in a shape intact msghdr_x entries can't produce: more datagrams than
headers, a length past MTU, or a sender that isn't a matching
AF_INET/AF_INET6 sockaddr. That batch is dropped rather than delivered. IPv6 sources now arrive without their
zone, matching the Linux recvmmsg path; WriteTo already drops it.
macOS raw dnclient to a Windows peer on the same gigabit LAN, iperf3
10 s, median of 3 alternating rounds against 7cfa47d:
download, 1/4/8 streams 606/600/604 -> 869/873/863 Mbps
download CPU ~159% -> ~123%
upload 340 -> 323 Mbps
UDP 500M loss 0.32% -> 0.31%
Download is now close to the 902 Mbps the LAN gives without a tunnel.
TestMsghdrXKernelRoundTrip and FuzzMsghdrXKernelRoundTrip pin msghdrX
against the kernel's struct msghdr_x, which the SDK doesn't ship: they
round-trip bursts through recvmsg_x on loopback and check every field
the kernel writes. recvmsg_x never reports MSG_TRUNC (soreceive_m_list
doesn't set it), so a datagram past MTU arrives cut to MTU with Flags
0, the same as ReadFromUDP. TestListenOutSingleFallback covers the
fallback loop, TestCheckMsghdrX the shape check, and the round trip
asserts the check accepts every batch the kernel returns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
newTunFromFd was a stub on darwin, so an embedder handed a utun by the OS had
no way to use it. A darwin network extension is given exactly that:
NEPacketTunnelProvider creates the device and applies addressing, mtu and
routes from its network settings, and a sandboxed extension cannot change them
afterwards. Adopting that fd means everything about moving packets is shared
with newTun and only the setup differs, so the tun tracks whether the host owns
it and leaves the system routing table and interface config alone when it does.
Co-authored-by: Nate Brown <nbrown.us@gmail.com>
The goroutines Main starts stop only when its context does, and only a caller
holding the Control can arrange that. Cancelling solely on error left a config
test, which returns no Control and no error, with the lighthouse query worker
and a hostname resolver per dns named static host running for the life of the
process. Every embedder that validates a config in process rather than by
exec'ing pays for that on each config load, dnclient and the apple clients
included.
Co-authored-by: Nate Brown <nbrown.us@gmail.com>
Using <https://go.dev/doc/go1.26#goroutineleak-profiles> + Claude, I was able to run nebula's unit tests and e2e tests with the leak detector enabled.
Added a TestMain that queries pprof to see if there are any reported goroutine leaks.
I'd love to get some form of this in CI whenever go 1.26 comes out, though I'd also like to prove this is properly useful past the just five detections it got here.
<details>
<summary>TestMain</summary>
```go
package nebula
import (
"fmt"
"os"
"runtime/pprof"
"strings"
"testing"
)
// TestMain runs after all tests and checks for goroutine leaks
func TestMain(m *testing.M) {
// Run all tests
exitCode := m.Run()
// Check for goroutine leaks after all tests complete
prof := pprof.Lookup("goroutineleak")
if prof != nil {
var sb strings.Builder
if err := prof.WriteTo(&sb, 2); err != nil {
fmt.Fprintf(os.Stderr, "Failed to write goroutineleak profile: %v\n", err)
os.Exit(1)
}
content := sb.String()
leakedCount := strings.Count(content, "(leaked)")
if leakedCount > 0 {
fmt.Fprintf(os.Stderr, "\n=== GOROUTINE LEAK DETECTED ===\n")
fmt.Fprintf(os.Stderr, "Found %d leaked goroutine(s) in package nebula\n\n", leakedCount)
goros := strings.Split(content, "\n\n")
for _, goro := range goros {
if strings.Contains(goro, "(leaked)") {
fmt.Fprintln(os.Stderr, goro)
fmt.Fprintln(os.Stderr)
}
}
os.Exit(1)
} else {
fmt.Println("✓ No goroutine leaks detected in package nebula")
}
}
os.Exit(exitCode)
}
```
</details>
Also had to install go1.26rc2 and update the makefile to use that go binary + set ex:
```makefile
test-goroutineleak:
GOEXPERIMENT=goroutineleakprofile go1.26rc2 test -v ./...
```