- Don't probe lanes on a tunnel idle in both directions, and don't count lane
probes or their acks as tunnel traffic. A lane's own keepalive was keeping
idle tunnels from ever being dropped by tunnels.drop_inactive.
- Accept a recv_error from an up lane's target. The peer answers lane traffic
from the lane socket it arrived on, and the spoof guard only accepted the
base remote, so recv_errors for lane traffic were ignored.
- Keep the probe generation across a lane reset, so an ack for a probe sent
before a roam can't promote the lane on the new path.
- Bring handshake.proto in line with payload.go: field 3 is reserved and
TxLanes is field 4.
Lanes used to be separate HostInfos, each with its own handshake, its own
half-established states, its own lifetime and its own slot bookkeeping. That
bought nothing: a lane is the same tunnel over a different underlay 5-tuple.
Derive lane sessions instead. Noise leaves us with A.eKey == B.dKey, so both
sides HKDF-expand the same two keys with the same per-lane label and land on a
matched pair without exchanging anything. Lanes now cost no handshake, have no
half-established state, and die exactly when their base tunnel does. Which lane
a packet belongs to rides the low byte of the nebula header's Reserved field,
inside the AEAD's associated data.
Receiving on a lane needs no permission, since the session exists the moment the
base handshake completes. Sending on one needs proof the new 5-tuple works, so a
lane stays down until a probe on it is acked and falls back to the base tunnel
the moment it stops being acked. Probing is demand-driven off the connection
manager's per-tunnel traffic tick: a peer we exchange a trickle with never costs
more than its base tunnel, however many lanes are configured. The ack rides the
base session on purpose, so a broken reverse lane can't fail a working one.
Because the data now rides lane counters, the rehandshake, exhaustion and
swap-primary checks take the max counter across the base session and its lanes;
otherwise the base counter would sit near zero while a lane ran its keys past
the nonce ceiling.
Removes OutboundLaneTimer, EnsureLanes, startLaneHandshake, completeLane,
completeLaneResponder, makeLaneTrafficDecision and the lane fields on HostInfo.
Handshake payload field 3 (the per-lane handshake index) is permanently
reserved; peers advertise a TxLanes count instead.
Lanes were established for every slot as soon as the base tunnel came up,
so a host with `routines: N` paid N-1 extra Noise handshakes, sessions and
keepalives for every peer it talked to, including peers it exchanges a
trickle with. Only routines that actually carry traffic to a peer need a
lane.
Add a per-slot demand flag to laneState. sendInsideMessage already loads
txLanes[laneSlot] on every direct-path packet; that load missing is now
the signal, and EnsureLanes only starts slots the data plane asked for.
The miss path is unchanged otherwise: the packet rides the base tunnel,
the same fallback used while a lane is down.
noteLaneDemand load-guards its store so repeated misses on a lane-less
slot are plain reads rather than a cache line ping-ponging between the
routines sharing it. The demand check is EnsureLanes' last condition, so
a slot that is pending or inside its backoff keeps the flag for the tick
that can act on it. Consuming the flag also stops a lane whose routine
went quiet from being rebuilt forever after it dies.
The wire format, lane negotiation and port-offset scheme are untouched,
so a lazy node interoperates with an eager one in both directions, and
inbound lane handshakes from a busy peer are still accepted regardless of
our own demand.
* try to make certificate addition/removal reloadable in some cases
* very spicy change to respond to handshakes with cert versions we cannot match with a cert that we can indeed match
* even spicier change to rehandshake if we detect our cert is lower-version than our peer, and we have a newer-version cert available
* make tryRehandshake easier to understand
* Don't delete a vpnaddr if it's part of a certificate that contains a vpnaddr that's in the static host map
* remove unused arg from ConnectionManager.shouldSwapPrimary()
* Fix for relay migration on rehandshaking issue. On rehandshake, the relay tunnel doesn't migrate to the new hostinfo object correctly, due to an incorrect Nebula IP sent in the CreateRelayRequest message.
* Add a test for this case
---------
Co-authored-by: Nate Brown <nbrown.us@gmail.com>
* cache cert verification
CheckSignature and Verify are expensive methods, and certificates are
static. Cache the results.
* use atomics
* make sure public key bytes match
* add VerifyWithCache and ResetCache
* cleanup
* use VerifyWithCache
* doc
* fix panic in handleInvalidCertificate
when HandleMonitorTick fires, the hostmap can be nil which causes a panic to occur when trying to clean up the hostmap in handleInvalidCertificate. This fix just stops the invalidation from continuing if the hostmap doesn't exist.
* removed conditional for disconnectInvalid in HandleDeletionTick
This restores the hostMap.QueryVpnIP block to how it looked before #370
was merged. I'm not sure why the patch from #370 wanted to continue on
if there was no match found in the hostmap, since there isn't anything
to do at that point (the tunnel has already been closed).
This was causing a crash because the handleInvalidCertificate check
expects the hostinfo to be passed in (but it is nil since there was no
hostinfo in the hostmap).
Fixes: #657
This change adds an index based on HostInfo.remoteIndexId. This allows
us to use HostMap.QueryReverseIndex without having to loop over all
entries in the map (this can be a bottleneck under high traffic
lighthouses).
Without this patch, a high traffic lighthouse server receiving recv_error
packets and lots of handshakes, cpu pprof trace can look like this:
flat flat% sum% cum cum%
2000ms 32.26% 32.26% 3040ms 49.03% github.com/slackhq/nebula.(*HostMap).QueryReverseIndex
870ms 14.03% 46.29% 1060ms 17.10% runtime.mapiternext
Which shows 50% of total cpu time is being spent in QueryReverseIndex.
This change adds a new helper, `(*HostInfo).logger()`, that starts a new
logrus.Entry with `vpnIp` and `certName`. We don't use the helper inside
of handshake_ix though since the certificate has not been attached to
the HostInfo yet.
Fixes: #84
Wade Simmons