mirror of
https://github.com/slackhq/nebula.git
synced 2026-09-30 11:16:38 +02:00
Wade Simmons
af439dadbf
Under multiport each port had exactly one socket, so each port had exactly one core. That hurt worst on the base port, which carries every handshake, every lighthouse and punch packet, every peer without multiport, and every tunnel whose lanes are down or firewalled. Bind multiport.ports consecutive ports and give each one a full group of `routines` sockets sharing it through SO_REUSEPORT, so the kernel hashes each arriving 4-tuple onto one of the group. `routines` is now per port and the total worker count is routines * multiport.ports, computed for you: with `routines: 8` and `multiport.ports: 4` you get 8 routines per port, 32 total. A routine still owns exactly one socket, which is what lets the read path own its state without locking. Sockets are laid out port-major, writers[s*routinesPerPort+r] being the r'th socket on port listen.port+s, so socket selection is a pure function of (queue, lane) with no borrowing: laneSock(q,s) = s*R + q%R, and base traffic takes laneSock(q,0). Sibling routines land on different sockets of the same port, so a lane's port is served by its whole group. multiport.ports is a new key and must be > 1; there is no default, since under these semantics defaulting it would silently multiply the worker count. The multiport decision moves above tun creation because tun queues are sized by the total. Also drop the base-port fallback for unanswered lane probes. A lane now aims only at its own paired peer port: sharing the base port's destination buys the lane a source port of its own but costs the peer the receive spread lanes exist to create, so a lane that cannot reach its port stays down and its flows ride the base tunnel.