ListenOut drains the socket with recvmsg_x, xnu's private batched
recvmsg (SYS_RECVMSG_X, bsd/sys/socket_private.h), up to 64 datagrams
per syscall, and calls flush once per batch instead of once per packet.
Each msghdr_x carries its own sockaddr, so this works on the shared
unconnected socket. The send side stays on sendto: xnu only batches
sendmsg_x through the stack on connected sockets, and on an unconnected
socket like this one it falls back to one sendit per message, with
msg_name documented as unsupported.
recvmsg_x is private, so ListenOut falls back to one
ReadFromUDPAddrPort per datagram, logging once, if the kernel or a
sandbox refuses it (ENOSYS, EPERM, EOPNOTSUPP) or if a batch comes back
in a shape intact msghdr_x entries can't produce: more datagrams than
headers, a length past MTU, or a sender that isn't a matching
AF_INET/AF_INET6 sockaddr. That batch is dropped rather than delivered. IPv6 sources now arrive without their
zone, matching the Linux recvmmsg path; WriteTo already drops it.
macOS raw dnclient to a Windows peer on the same gigabit LAN, iperf3
10 s, median of 3 alternating rounds against 7cfa47d:
download, 1/4/8 streams 606/600/604 -> 869/873/863 Mbps
download CPU ~159% -> ~123%
upload 340 -> 323 Mbps
UDP 500M loss 0.32% -> 0.31%
Download is now close to the 902 Mbps the LAN gives without a tunnel.
TestMsghdrXKernelRoundTrip and FuzzMsghdrXKernelRoundTrip pin msghdrX
against the kernel's struct msghdr_x, which the SDK doesn't ship: they
round-trip bursts through recvmsg_x on loopback and check every field
the kernel writes. recvmsg_x never reports MSG_TRUNC (soreceive_m_list
doesn't set it), so a datagram past MTU arrives cut to MTU with Flags
0, the same as ReadFromUDP. TestListenOutSingleFallback covers the
fallback loop, TestCheckMsghdrX the shape check, and the round trip
asserts the check accepts every batch the kernel returns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
remove runtime.LockOSThread() because it makes things worse now
remove the "custom" Write() method from tun_linux.go, the stdlib path via os.File performs better
We should change our guidance around number of routines, ~2 per thread (that you wish to use for Nebula) seems to be about right now
If the host OS is already big endian, we were swapping bytes when we
shouldn't have. Use the Go helper to make sure we do the endianness
correctly
Fixes: #1189
On some systems, IPv6 is disabled (for example, CIS benchmark recommends to disable it when not used), but currently all UDP connections are using AF_INET6 sockets.
When we are binding AF_INET6 socket to an address like ::ffff:1.2.3.4 (IPv4 addresses are parsed by net.ParseIP this way), we can't send or receive IPv6 packets anyway, so this will not break any scenarios.
---------
Co-authored-by: Wade Simmons <wsimmons@slack-corp.com>
* add calculated_remotes
This setting allows us to "guess" what the remote might be for a host
while we wait for the lighthouse response. For networks that hard
designed with in mind, it can help speed up handshake performance, as well as
improve resiliency in the case that all lighthouses are down.
Example:
lighthouse:
# ...
calculated_remotes:
# For any Nebula IPs in 10.0.10.0/24, this will apply the mask and add
# the calculated IP as an initial remote (while we wait for the response
# from the lighthouse). Both CIDRs must have the same mask size.
# For example, Nebula IP 10.0.10.123 will have a calculated remote of
# 192.168.1.123
10.0.10.0/24:
- mask: 192.168.1.0/24
port: 4242
* figure out what is up with this test
* add test
* better logic for sending handshakes
Keep track of the last light of hosts we sent handshakes to. Only log
handshake sent messages if the list has changed.
Remove the test Test_NewHandshakeManagerTrigger because it is faulty and
makes no sense. It relys on the fact that no handshake packets actually
get sent, but with these changes we would send packets now (which it
should!)
* use atomic.Pointer
* cleanup to make it clearer
* fix typo in example
