mirror of
https://github.com/slackhq/nebula.git
synced 2026-11-15 19:26:52 +01:00
udp/darwin: send to busy established peers on connected sockets
On darwin an unconnected sendto runs the address and MAC policy checks and the EndpointSecurity send check for every datagram, and while xprotectd throttles it each send takes about 185us, which caps a tunnel's upload and, through its ACKs, its download. A connected socket runs those checks once, at connect, and it is the only UDP send that reports the interface queue's flow control, as ENOBUFS. Only the current direct remote of an established tunnel is eligible. udp.EstablishedPeerConn is an optional Conn interface, implemented on darwin; establishedRemotes refcounts remotes over the main hostmap and tells each writer on the first and last reference: handshake completion adds one, deleting or retiring a hostinfo removes it, SetRemote (roaming, SetRemoteIfPreferred, control, ssh) moves it. Pending and relayed hostinfos hold none, and a rehandshake's two hostinfos share one. An eligible peer that Nebula sends NEBULA_CONNSOCK_RUN (64) datagrams to within NEBULA_CONNSOCK_WINDOW (1s) gets a connected socket: SO_REUSEPORT, bound to the source address the listener's route to the peer picks on the listener's port, carrying the listener's IP_BOUND_IF. The peer sees the same address and port as from the listener, and the socket's reader delivers through the listener's reader. The listener itself never sets SO_REUSEPORT (listen routines still do), so no other uid can bind the port; with a specific listen.host and no routines connected sockets stay off, since sharing an exact address needs SO_REUSEPORT on both. All routines listening on one address share one set of connected sockets: at most NEBULA_CONNSOCKS (8), one per peer, with opens from a bucket that refills one per NEBULA_CONNSOCK_OPEN_EVERY (1s). A socket closes when its peer stops being eligible, after NEBULA_CONNSOCK_IDLE (30s) without traffic, on Rebind (of every routine) and Close, and on an error that means the connected path is gone. Closing never runs under the hostmap's or the eligibility lock; the peer can't reopen until the close finishes. A send that gets ENOBUFS or EAGAIN waits within one 10ms budget shared by every writer since the last send that went through, then drops; nothing waits in the netpoller without a deadline. When nothing is eligible or open, a send costs one atomic load before the listener path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7cfa47d569
commit
3bb5918e7c
12 files changed
+2349
-13
No files matched your search
@@ -48,6 +48,14 @@ type Conn interface {
|
||||
Close() error
|
||||
}
|
||||
|
||||
// EstablishedPeerConn is a Conn that treats the remotes of established tunnels differently from other
|
||||
// destinations, as darwin's gives busy ones connected sockets.
|
||||
type EstablishedPeerConn interface {
|
||||
// SetEstablishedPeer reports whether dst is the current remote of at least one established tunnel. Calls for
|
||||
// one dst must not race each other.
|
||||
SetEstablishedPeer(dst netip.AddrPort, established bool)
|
||||
}
|
||||
|
||||
type NoopConn struct{}
|
||||
|
||||
func (NoopConn) Rebind() error {
|
||||
|
||||
Reference in new issue
Block a user