- Don't probe lanes on a tunnel idle in both directions, and don't count lane
probes or their acks as tunnel traffic. A lane's own keepalive was keeping
idle tunnels from ever being dropped by tunnels.drop_inactive.
- Accept a recv_error from an up lane's target. The peer answers lane traffic
from the lane socket it arrived on, and the spoof guard only accepted the
base remote, so recv_errors for lane traffic were ignored.
- Keep the probe generation across a lane reset, so an ack for a probe sent
before a roam can't promote the lane on the new path.
- Bring handshake.proto in line with payload.go: field 3 is reserved and
TxLanes is field 4.
Two routines can race on a lane's first packet and derive a session each. Only
one gets installed, and the loser was dropped along with the replay-window entry
for the packet it had just accepted, leaving that one counter replayable.
The two sessions hold identical keys, so the winner's window is the same window
in every respect that matters: mark the counter seen on it instead. Closes the
gap without putting a lock on the decrypt path.
How many lanes a tunnel has is partly the peer's call: it advertises how many it
sends on and we have to be able to receive all of them, up to the header's 256.
Deriving them all when the handshake completes meant a peer advertising a large
count cost us a replay window and two cipher states per lane, per tunnel, for
lanes it may never send on.
Derive each session on the first packet that needs it instead. The TX side asks
through laneSet.session, which installs on the spot — we only ask for lanes we
chose to send on. The RX side can't do that: anyone who can spoof a tunnel's
local index can name any lane, and installing on sight would hand them the same
allocation for free. So laneSession hands back a session without publishing it
and reports that it did; outside.go installs it only once the packet has
decrypted, which is the first moment the lane is known to be real. A spoofer
gets an HKDF per packet and nothing retained.
Only the session table is sized by the peer's advert now. txAddr, demand and
probe are sized by the lanes we will actually send on, so the peer can no longer
size our per-lane tx state either.
Also refuse relayed lane packets before the session lookup rather than after,
so a junk relay packet can't reach the derivation path at all.
Lanes used to be separate HostInfos, each with its own handshake, its own
half-established states, its own lifetime and its own slot bookkeeping. That
bought nothing: a lane is the same tunnel over a different underlay 5-tuple.
Derive lane sessions instead. Noise leaves us with A.eKey == B.dKey, so both
sides HKDF-expand the same two keys with the same per-lane label and land on a
matched pair without exchanging anything. Lanes now cost no handshake, have no
half-established state, and die exactly when their base tunnel does. Which lane
a packet belongs to rides the low byte of the nebula header's Reserved field,
inside the AEAD's associated data.
Receiving on a lane needs no permission, since the session exists the moment the
base handshake completes. Sending on one needs proof the new 5-tuple works, so a
lane stays down until a probe on it is acked and falls back to the base tunnel
the moment it stops being acked. Probing is demand-driven off the connection
manager's per-tunnel traffic tick: a peer we exchange a trickle with never costs
more than its base tunnel, however many lanes are configured. The ack rides the
base session on purpose, so a broken reverse lane can't fail a working one.
Because the data now rides lane counters, the rehandshake, exhaustion and
swap-primary checks take the max counter across the base session and its lanes;
otherwise the base counter would sit near zero while a lane ran its keys past
the nonce ceiling.
Removes OutboundLaneTimer, EnsureLanes, startLaneHandshake, completeLane,
completeLaneResponder, makeLaneTrafficDecision and the lane fields on HostInfo.
Handshake payload field 3 (the per-lane handshake index) is permanently
reserved; peers advertise a TxLanes count instead.
handleOutsideRelayPacket filled ViaSender.remoteIdx with relay.RemoteIndex,
an index from the relay peer's index space, but the rescue in
sendHandshakeResponse looks that value up in relayForByIdx, which is keyed
by local index. The lookup could never hit, so a terminal relay entry left
Disestablished by a one-sided teardown stayed Disestablished even after a
valid handshake arrived over it. The responder's first transmit then failed
to find an Established relay, deleted its only relay entry, and every
subsequent send was silently dropped until dead-tunnel detection forced a
re-handshake.
* refactor readOutsidePackets
They layout of this method is confusing and relys on certain parts to
return early for things to work correctly.
Change the ordering of the logic so that we do this:
- Handle unencrypted packets
- Decrypt packet
- Handle encrypted packets
This way, nothing can sneak through unencrypted to where it shouldn't
be.
* fix comment
* code review comments
* check for expected type/subtype
* check header version
* log header
* need to handle TestReply
* clean roaming / connectionManager
* dont need to roam here now, we do it earlier
* cleanup metrics and errors
* rxInvalid
* debug logger checks
* ErrOutOfWindow
* don't send recv errors for packets outside the connection window anymore
* Pull in fix from #1459, add my opinion on maxRecvError
* remove recv_error counter entirely
* firewall: add option to send REJECT replies
This change allows you to configure the firewall to send REJECT packets
when a packet is denied.
firewall:
# Action to take when a packet is not allowed by the firewall rules.
# Can be one of:
# `drop` (default): silently drop the packet.
# `reject`: send a reject reply.
# - For TCP, this will be a RST "Connection Reset" packet.
# - For other protocols, this will be an ICMP port unreachable packet.
outbound_action: drop
inbound_action: drop
These packets are only sent to established tunnels, and only on the
overlay network (currently IPv4 only).
$ ping -c1 192.168.100.3
PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
From 192.168.100.3 icmp_seq=2 Destination Port Unreachable
--- 192.168.100.3 ping statistics ---
2 packets transmitted, 0 received, +1 errors, 100% packet loss, time 31ms
$ nc -nzv 192.168.100.3 22
(UNKNOWN) [192.168.100.3] 22 (?) : Connection refused
This change also modifies the smoke test to capture tcpdump pcaps from
both the inside and outside to inspect what is going on over the wire.
It also now does TCP and UDP packet tests using the Nmap version of
ncat.
* calculate seq and ack the same was as the kernel
The logic a bit confusing, so we copy it straight from how the kernel
does iptables `--reject-with tcp-reset`:
- https://github.com/torvalds/linux/blob/v5.19/net/ipv4/netfilter/nf_reject_ipv4.c#L193-L221
* cleanup
By default, Nebula replies to packets it has no tunnel for with a `recv_error` packet. This packet helps speed up re-connection
in the case that Nebula on either side did not shut down cleanly. This response can be abused as a way to discover if Nebula is running
on a host though. This option lets you configure if you want to send `recv_error` packets always, never, or only to private network remotes.
valid values: always, never, private
This setting is reloadable with SIGHUP.
* don't set ConnectionState to nil
We might have packets processing in another thread, so we can't safely
just set this to nil. Since we removed it from the hostmaps, the next
packets to process should start the handshake over again.
I believe this comment is outdated or incorrect, since the next
handshake will start over with a new HostInfo, I don't think there is
any way a counter reuse could happen:
> We must null the connectionstate or a counter reuse may happen
Here is a panic we saw that I think is related:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x93a037]
goroutine 59 [running, locked to thread]:
github.com/slackhq/nebula.(*Firewall).Drop(...)
github.com/slackhq/nebula/firewall.go:380
github.com/slackhq/nebula.(*Interface).consumeInsidePacket(...)
github.com/slackhq/nebula/inside.go:59
github.com/slackhq/nebula.(*Interface).listenIn(...)
github.com/slackhq/nebula/interface.go:233
created by github.com/slackhq/nebula.(*Interface).run
github.com/slackhq/nebula/interface.go:191
* use closeTunnel
This allows you to configure remote allow lists specific to different
subnets of the inside CIDR. Example:
remote_allow_ranges:
10.42.42.0/24:
192.168.0.0/16: true
This would only allow hosts with a VPN IP in the 10.42.42.0/24 range to
have private IPs (and thus don't connect over public IPs).
The PR also refactors AllowList into RemoteAllowList and LocalAllowList to make it clearer which methods are allowed on which allow list.
Wade Simmons