Caleb JasikandClaude Opus 5.5
37032a22bf
On darwin an unconnected sendto runs the address and MAC policy checks and the EndpointSecurity send check for every datagram, and while xprotectd throttles it each send takes about 185us, which caps a tunnel's upload and, through its ACKs, its download. A connected socket runs those checks once, at connect, and it is the only UDP send that reports the interface queue's flow control, as ENOBUFS. Only the current direct remote of an established tunnel is eligible. udp.EstablishedPeerConn is an optional Conn interface, implemented on darwin; establishedRemotes refcounts remotes over the main hostmap and tells each writer on the first and last reference: handshake completion adds one, deleting or retiring a hostinfo removes it, SetRemote (roaming, SetRemoteIfPreferred, control, ssh) moves it. Pending and relayed hostinfos hold none, and a rehandshake's two hostinfos share one. An eligible peer that Nebula sends NEBULA_CONNSOCK_RUN (64) datagrams to within NEBULA_CONNSOCK_WINDOW (1s) gets a connected socket: SO_REUSEPORT, bound to the source address the listener's route to the peer picks on the listener's port, carrying the listener's IP_BOUND_IF. The peer sees the same address and port as from the listener, and the socket's reader delivers through the listener's reader. The listener itself never sets SO_REUSEPORT (listen routines still do), so no other uid can bind the port; with a specific listen.host and no routines connected sockets stay off, since sharing an exact address needs SO_REUSEPORT on both. All routines listening on one address share one set of connected sockets: at most NEBULA_CONNSOCKS (8), one per peer, with opens from a bucket that refills one per NEBULA_CONNSOCK_OPEN_EVERY (1s). A socket closes when its peer stops being eligible, after NEBULA_CONNSOCK_IDLE (30s) without traffic, on Rebind (of every routine) and Close, and on an error that means the connected path is gone. Closing never runs under the hostmap's or the eligibility lock; the peer can't reopen until the close finishes. A send that gets ENOBUFS or EAGAIN waits within one 10ms budget shared by every writer since the last send that went through, then drops; nothing waits in the netpoller without a deadline. When nothing is eligible or open, a send costs one atomic load before the listener path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What is Nebula?
Nebula is a scalable overlay networking tool with a focus on performance, simplicity and security. It lets you seamlessly connect computers anywhere in the world. Nebula is portable, and runs on Linux, OSX, Windows, iOS, and Android. It can be used to connect a small number of computers, but is also able to connect tens of thousands of computers.
Nebula incorporates a number of existing concepts like encryption, security groups, certificates, and tunneling. What makes Nebula different to existing offerings is that it brings all of these ideas together, resulting in a sum that is greater than its individual parts.
Further documentation can be found here.
You can read more about Nebula here.
You can also join the NebulaOSS Slack group here.
Supported Platforms
Desktop and Server
Check the releases page for downloads or see the Distribution Packages section.
- Linux - 64 and 32 bit, arm, and others
- Windows
- MacOS
- Freebsd
Distribution Packages
-
sudo pacman -S nebula -
sudo dnf install nebula -
sudo apt install nebula -
sudo apk add nebula -
brew install nebula -
docker pull nebulaoss/nebula
Mobile (source code)
Technical Overview
Nebula is a mutually authenticated peer-to-peer software-defined network based on the Noise Protocol Framework. Nebula uses certificates to assert a node's IP address, name, and membership within user-defined groups. Nebula's user-defined groups allow for provider agnostic traffic filtering between nodes. Discovery nodes (aka lighthouses) allow individual peers to find each other and optionally use UDP hole punching to establish connections from behind most firewalls or NATs. Users can move data between nodes in any number of cloud service providers, datacenters, and endpoints, without needing to maintain a particular addressing scheme.
Nebula uses Elliptic-curve Diffie-Hellman (ECDH) key exchange and AES-256-GCM in its default configuration.
Nebula was created to provide a mechanism for groups of hosts to communicate securely, even across the internet, while enabling expressive firewall definitions similar in style to cloud security groups.
Getting started (quickly)
Don't want to manage your own PKI and lighthouses? Managed Nebula from Defined Networking handles all of this for you.
To set up a Nebula network, you'll need:
1. The Nebula binaries or Distribution Packages for your specific platform. Specifically you'll need nebula-cert and the specific nebula binary for each platform you use.
2. (Optional, but you really should..) At least one discovery node with a routable IP address, which we call a lighthouse.
Nebula lighthouses allow nodes to find each other, anywhere in the world. A lighthouse is the only node in a Nebula network whose IP should not change. Running a lighthouse requires very few compute resources, and you can easily use the least expensive option from a cloud hosting provider. If you're not sure which provider to use, a number of us have used $6/mo DigitalOcean droplets as lighthouses.
Once you have launched an instance, ensure that Nebula udp traffic (default port udp/4242) can reach it over the internet.
3. A Nebula certificate authority, which will be the root of trust for a particular Nebula network.
./nebula-cert ca -name "Myorganization, Inc"
This will create files named ca.key and ca.cert in the current directory. The ca.key file is the most sensitive file you'll create, because it is the key used to sign the certificates for individual nebula nodes/hosts. Please store this file somewhere safe, preferably with strong encryption.
Be aware! By default, certificate authorities have a 1-year lifetime before expiration. See this guide for details on rotating a CA.
4. Nebula host keys and certificates generated from that certificate authority
This assumes you have four nodes, named lighthouse1, laptop, server1, host3. You can name the nodes any way you'd like, including FQDN. You'll also need to choose IP addresses and the associated subnet. In this example, we are creating a nebula network that will use 192.168.100.x/24 as its network range. This example also demonstrates nebula groups, which can later be used to define traffic rules in a nebula network.
./nebula-cert sign -name "lighthouse1" -ip "192.168.100.1/24"
./nebula-cert sign -name "laptop" -ip "192.168.100.2/24" -groups "laptop,home,ssh"
./nebula-cert sign -name "server1" -ip "192.168.100.9/24" -groups "servers"
./nebula-cert sign -name "host3" -ip "192.168.100.10/24"
By default, host certificates will expire 1 second before the CA expires. Use the -duration flag to specify a shorter lifetime.
5. Configuration files for each host
Download a copy of the nebula example configuration.
-
On the lighthouse node, you'll need to ensure
am_lighthouse: trueis set. -
On the individual hosts, ensure the lighthouse is defined properly in the
static_host_mapsection, and is added to the lighthousehostssection.
6. Copy nebula credentials, configuration, and binaries to each host
For each host, copy the nebula binary to the host, along with config.yml from step 5, and the files ca.crt, {host}.crt, and {host}.key from step 4.
DO NOT COPY ca.key TO INDIVIDUAL NODES.
7. Run nebula on each host
./nebula -config /path/to/config.yml
For more detailed instructions, find the full documentation here.
Building Nebula from source
Make sure you have go installed and clone this repo. Change to the nebula directory.
To build nebula for all platforms:
make all
To build nebula for a specific platform (ex, Windows):
make bin-windows
See the Makefile for more details on build targets
Curve P256 and FIPS 140-3 mode
The default curve used for cryptographic handshakes and signatures is Curve25519. This is the recommended setting for most users. If your deployment has certain compliance requirements, you have the option of creating your CA using nebula-cert ca -curve P256 to use NIST Curve P256. The CA will then sign certificates using ECDSA P256, and any hosts using these certificates will use P256 for ECDH handshakes.
Nebula can be built to support the FIPS 140-3 mode of Go by running either of the following make targets. (This sets GOFIPS140=v1.0.0, which must be done at compile time so that the correct AES-GCM can be used for FIPS 140-3 enforcement mode).
make fips140
make fips140 test
make release-fips140
Nebula can also be built using the BoringCrypto GOEXPERIMENT by running either of the following make targets.
make bin-boringcrypto
make release-boringcrypto
NOTE: boringcrypto support is deprecated and will be removed in the next release. Users should migrate to the native FIPS 140-3 mode described above.
This is not the recommended default deployment, but may be useful based on your compliance requirements.
Credits
Nebula was created at Slack Technologies, Inc by Nate Brown and Ryan Huber, with contributions from Oliver Fross, Alan Lam, Wade Simmons, and Lining Wang.