newTunFromFd was a stub on darwin, so an embedder handed a utun by the OS had
no way to use it. A darwin network extension is given exactly that:
NEPacketTunnelProvider creates the device and applies addressing, mtu and
routes from its network settings, and a sandboxed extension cannot change them
afterwards. Adopting that fd means everything about moving packets is shared
with newTun and only the setup differs, so the tun tracks whether the host owns
it and leaves the system routing table and interface config alone when it does.
(cherry picked from commit e20569ce51)
Co-authored-by: Nate Brown <nbrown.us@gmail.com>
Backport from #1835 to release 1.11
## What
Since the bump to netlink v1.3.0, `netlink.AddrList` returns `ErrDumpInterrupted` ("results may be incomplete or inconsistent") when the kernel sets `NLM_F_DUMP_INTR` on an address dump. `addIPs` treats any error from `AddrList` as fatal, so a transient interrupted dump aborts startup:
```
failed to start nebula: failed to get tun address list: results may be incomplete or inconsistent
```
Before v1.3.0, the pinned netlink release had no `NLM_F_DUMP_INTR` handling at all and silently returned partial results, so this condition was previously invisible.
## Root cause
The dump is racing nebula's own setup. `RTM_GETADDR` dumps addresses for the whole system, and the kernel flags the dump as interrupted if any address changes between recvmsg batches. When the cert contains an IPv6 network, the `AddrReplace` a few lines above returns while the new address is still tentative; the kernel's DAD worker flips it to preferred asynchronously ~100-400us later, and that flip lands inside the immediately-following dump. The race is entirely self-contained - no concurrent restart or interface churn is required, and the hit rate scales with how many addresses are on the host (more dump batches = wider window).
Reproduced with a standalone tool replaying this exact netlink sequence on a host with ~80 addresses: an IPv6 /80 alone interrupts 10/200 cycles and v4+v6 together 16/200, while IPv4-only runs 0/800. `ip -ts monitor addr` shows the tentative add and the async flag-clear as two events during the dump window. Hosts with DAD disabled (`net.ipv6.conf.all.accept_dad=0`, e.g. LXC containers) never reproduce, confirming the flip is the trigger. Observed in the wild as dnclient package upgrades reliably failing their post-install restart (systemd start timeout, package left half-configured).
## How
Treat `ErrDumpInterrupted` as success with a warning. netlink still returns the partial result set alongside the error, and this call site only uses the list to prune addresses that are not in the certs; a missed stale address survives until the next reload, which is strictly better than failing startup. All cert addresses were already applied via `AddrReplace` above.
Co-authored-by: John Maguire <john@defined.net>
remove runtime.LockOSThread() because it makes things worse now
remove the "custom" Write() method from tun_linux.go, the stdlib path via os.File performs better
We should change our guidance around number of routines, ~2 per thread (that you wish to use for Nebula) seems to be about right now
Recent merge of cert-v2 support introduced the ability to tunnel IPv6. However, FreeBSD's IPv6 tunneling does not work for 2 reasons:
* The ifconfig commands did not work for IPv6 addresses
* The tunnel device was not configured for link-layer mode, so it only supported IPv4
This PR improves FreeBSD tunneling support in 3 ways:
* Use ioctl instead of exec'ing ifconfig to configure the interface, with additional logic to support IPv6
* Configure the tunnel in link-layer mode, allowing IPv6 traffic
* Use readv() and writev() to communicate with the tunnel device, to avoid the need to copy the packet buffer
We switched to yaml.v3 with #1148, but missed this spot that was still
casting into `map[any]any` when yaml.v3 makes it `map[string]any`. Also
clean up a few more `interface{}` that were added as we changed them all
to `any` with #1148.
* upgrade to yaml.v3
The main nice fix here is that maps unmarshal into `map[string]any`
instead of `map[any]any`, so it cleans things up a bit.
* add config.AsBool
Since yaml.v3 doesn't automatically convert yes to bool now, for
backwards compat
* use type aliases for m
* more cleanup
* more cleanup
* more cleanup
* go mod cleanup
* firewall: add option to send REJECT replies
This change allows you to configure the firewall to send REJECT packets
when a packet is denied.
firewall:
# Action to take when a packet is not allowed by the firewall rules.
# Can be one of:
# `drop` (default): silently drop the packet.
# `reject`: send a reject reply.
# - For TCP, this will be a RST "Connection Reset" packet.
# - For other protocols, this will be an ICMP port unreachable packet.
outbound_action: drop
inbound_action: drop
These packets are only sent to established tunnels, and only on the
overlay network (currently IPv4 only).
$ ping -c1 192.168.100.3
PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
From 192.168.100.3 icmp_seq=2 Destination Port Unreachable
--- 192.168.100.3 ping statistics ---
2 packets transmitted, 0 received, +1 errors, 100% packet loss, time 31ms
$ nc -nzv 192.168.100.3 22
(UNKNOWN) [192.168.100.3] 22 (?) : Connection refused
This change also modifies the smoke test to capture tcpdump pcaps from
both the inside and outside to inspect what is going on over the wire.
It also now does TCP and UDP packet tests using the Nmap version of
ncat.
* calculate seq and ack the same was as the kernel
The logic a bit confusing, so we copy it straight from how the kernel
does iptables `--reject-with tcp-reset`:
- https://github.com/torvalds/linux/blob/v5.19/net/ipv4/netfilter/nf_reject_ipv4.c#L193-L221
* cleanup
