mirror of
https://github.com/slackhq/nebula.git
synced 2026-10-04 18:56:37 +02:00
Tolerate ErrDumpInterrupted when listing tun addresses (#1842)
Backport from #1835 to release 1.11 ## What Since the bump to netlink v1.3.0, `netlink.AddrList` returns `ErrDumpInterrupted` ("results may be incomplete or inconsistent") when the kernel sets `NLM_F_DUMP_INTR` on an address dump. `addIPs` treats any error from `AddrList` as fatal, so a transient interrupted dump aborts startup: ``` failed to start nebula: failed to get tun address list: results may be incomplete or inconsistent ``` Before v1.3.0, the pinned netlink release had no `NLM_F_DUMP_INTR` handling at all and silently returned partial results, so this condition was previously invisible. ## Root cause The dump is racing nebula's own setup. `RTM_GETADDR` dumps addresses for the whole system, and the kernel flags the dump as interrupted if any address changes between recvmsg batches. When the cert contains an IPv6 network, the `AddrReplace` a few lines above returns while the new address is still tentative; the kernel's DAD worker flips it to preferred asynchronously ~100-400us later, and that flip lands inside the immediately-following dump. The race is entirely self-contained - no concurrent restart or interface churn is required, and the hit rate scales with how many addresses are on the host (more dump batches = wider window). Reproduced with a standalone tool replaying this exact netlink sequence on a host with ~80 addresses: an IPv6 /80 alone interrupts 10/200 cycles and v4+v6 together 16/200, while IPv4-only runs 0/800. `ip -ts monitor addr` shows the tentative add and the async flag-clear as two events during the dump window. Hosts with DAD disabled (`net.ipv6.conf.all.accept_dad=0`, e.g. LXC containers) never reproduce, confirming the flip is the trigger. Observed in the wild as dnclient package upgrades reliably failing their post-install restart (systemd start timeout, package left half-configured). ## How Treat `ErrDumpInterrupted` as success with a warning. netlink still returns the partial result set alongside the error, and this call site only uses the list to prune addresses that are not in the certs; a missed stale address survives until the next reload, which is strictly better than failing startup. All cert addresses were already applied via `AddrReplace` above. Co-authored-by: John Maguire <john@defined.net>
This commit is contained in:
co-authored by
John Maguire
parent
1617897043
commit
ae688d038d
+11
-1
@@ -5,6 +5,7 @@ package overlay
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
@@ -483,7 +484,16 @@ func (t *tun) addIPs(link netlink.Link) error {
|
||||
//iterate over remainder, remove whoever shouldn't be there
|
||||
al, err := netlink.AddrList(link, netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get tun address list: %s", err)
|
||||
//RTM_GETADDR dumps the whole system, so any concurrent address change
|
||||
//interrupts it - including the kernel's async tentative->preferred
|
||||
//flip of an IPv6 address the AddrReplace calls above just added,
|
||||
//which makes this a race against our own setup. Partial results are
|
||||
//still returned; the worst case is a stale address surviving until
|
||||
//the next config reload, which beats failing startup over it.
|
||||
if !errors.Is(err, netlink.ErrDumpInterrupted) {
|
||||
return fmt.Errorf("failed to get tun address list: %s", err)
|
||||
}
|
||||
t.l.Warn("tun address list dump was interrupted, stale addresses may remain")
|
||||
}
|
||||
|
||||
for i := range al {
|
||||
|
||||
Reference in New Issue
Block a user