Files
nebula/e2e/darwin/vpnemu

vpnemu

Reproduces, on macOS and without Tailscale, the way a VPN with an exit node takes nebula's vpn network route away from its utun (#1931).

A macOS network extension never touches the routing table. Its routes are published in configd's dynamic store under State:/Network/Service/<id>/IPv4 and /IPv6, and configd's IPMonitor installs them. vpnemu creates a utun and publishes those keys itself, so configd does exactly what it would do for the real app. It publishes the routes Tailscale would with an exit node: every interface prefix that isn't a single IP, the rule in Tailscale's ipnlocal.internalAndExternalInterfacesFrom.

mode Tailscale setting published as where nebula's prefix goes
claim exit node, no LAN access AdditionalRoutes into vpnemu's utun
exclude exit node, LAN access ExcludedRoutes, service overrides primary out the next interface (en0)

configd answers EEXIST with a delete and an add, so whatever held the prefix loses it. When the service is withdrawn, configd deletes its routes by prefix alone, whoever holds them by then.

Build

It is its own module, so purego stays out of nebula's go.mod. purego reaches CoreFoundation and SystemConfiguration without cgo, so it cross compiles and runs on a mac with no developer tools.

cd e2e/darwin/vpnemu
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -o vpnemu .
# a cross compiled binary copied to a mac may need: codesign -s - -f vpnemu

Run

sudo ./vpnemu -mode claim -keep en0      # runs until ctrl-c, then withdraws and closes its utun
sudo ./vpnemu -clear                     # withdraw a service a crashed run left behind

-keep names interfaces whose prefixes are never published. Keep the one your ssh session arrives on, or claim takes your LAN into a utun nobody reads. exclude makes vpnemu primary, so anything off a kept interface's subnet, internet included, goes into its utun and is dropped while it runs.

-follow (default on) republishes when interface prefixes change, as Tailscale does on link changes. With -follow=false a route stays published after its interface is gone, which stages the route configd leaves behind for a nebula that restarts before the VPN has caught up.

A/B a nebula build

repro.sh starts nebula, then runs vpnemu, and reports egress and ping per peer before, during and after:

sudo ./repro.sh -n ./nebula -c config.yml -p 192.168.212.1 -p fd19:31::1 -m claim -k en0

Before #1931 (utun holds /24 and /64):

during   192.168.212.1   egress=utun2   ping=FAIL
after    192.168.212.1   egress=en0     ping=FAIL

With #1931 (utun holds /32 and /128, network routes added explicitly) every phase stays on nebula's utun.

Leftover route

Run an old nebula, start vpnemu -follow=false, then restart onto a new nebula while vpnemu is still up. The new nebula finds the vpn network routed to vpnemu's utun, replaces it and starts. Stop vpnemu and configd deletes the prefix, taking nebula's route with it until nebula restarts. With real Tailscale this needs nebula to restart before Tailscale notices the old utun is gone, a one time window when upgrading from a nebula without #1931.