Install and Configure Headscale on Ubuntu 22.04|20.04

Headscale is an open source implementation of Tailscale coordination server. Tailscale has been around for some time and it allows users to create secure networks with multiple devices connected seamlessly, regardless of the physical location of these devices. In a nutshell, Tailscale makes the process of deploying and managing a VPN easier and more user-friendly.

Tailscale technology creates a secure mesh network that enables all devices connected to it communicate with each other and behave as if they are on the same local network. Headscale is completely separate from Tailscale and developed independently. In this article we shall install, configure and use Headscale to create a mesh network and connect your devices.

1. Download Headscale Apt package

Update system apt package index.

sudo apt update

Visit Headscale releases page on Github. Under tags you can get the latest stable release number.

VERSION=$(curl --silent "https://api.github.com/repos/juanfont/headscale/releases/latest"|grep '"tag_name"'|sed -E 's/.*"([^"]+)".*/\1/'|sed 's/v//')
wget https://github.com/juanfont/headscale/releases/download/v${VERSION}/headscale_${VERSION}_linux_amd64.deb

Install the package once it is downloaded using apt command.

sudo apt install -f ./headscale_${VERSION}_linux_amd64.deb

We can enable the service to start at system boot.

sudo systemctl enable headscale

2. Configure Headscale Service

You can adjust Headscale configuration settings by editing the file /etc/headscale/config.yaml

sudo vim /etc/headscale/config.yaml

Notable parameters to configure are;

# The url clients will connect to.
server_url: http://127.0.0.1:8080

# Address to listen to / bind to on the server
listen_addr: 127.0.0.1:8080

# Address to listen to /metrics, you may want
metrics_listen_addr: 127.0.0.1:9090

You can configure to listen on all interfaces.

server_url: http://0.0.0.0:8080
listen_addr: 0.0.0.0:8080

Or specific IP address.

server_url: http://192.168.20.10:8080
listen_addr: 192.168.20.10:8080

You can adjust other parameters to suit your use case and restart the service when done.

sudo systemctl restart headscale.service

Status of the service can be checked using systemctl command.

$ systemctl status headscale.service
● headscale.service - headscale coordination server for Tailscale
     Loaded: loaded (/lib/systemd/system/headscale.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-10-03 18:39:12 EAT; 8s ago
   Main PID: 5901 (headscale)
      Tasks: 8 (limit: 19092)
     Memory: 9.4M
        CPU: 68ms
     CGroup: /system.slice/headscale.service
             └─5901 /usr/bin/headscale serve

Okt 03 18:39:12 workstation systemd[1]: Started headscale coordination server for Tailscale.
Okt 03 18:39:12 workstation headscale[5901]: An updated version of Headscale has been found (0.23.0-alpha1 vs. your current v0.22.3). Check it out https://github.com/juanfont/headscale/releases
Okt 03 18:39:12 workstation headscale[5901]: 2023-10-03T18:39:12+03:00 INF Setting up a DERPMap update worker frequency=86400000
Okt 03 18:39:12 workstation headscale[5901]: 2023-10-03T18:39:12+03:00 INF listening and serving HTTP on: 0.0.0.0:9080
Okt 03 18:39:12 workstation headscale[5901]: 2023-10-03T18:39:12+03:00 INF listening and serving metrics on: 0.0.0.0:9090

Listing ports used by Headscale service.

$ ss -tunelp | egrep '9080|9090'
tcp   LISTEN 0      128                *:9080             *:*    users:(("headscale",pid=5901,fd=11)) uid:1002 ino:69679 sk:b cgroup:/system.slice/headscale.service v6only:0 <->
tcp   LISTEN 0      128                *:9090             *:*    users:(("headscale",pid=5901,fd=13)) uid:1002 ino:69680 sk:c cgroup:/system.slice/headscale.service v6only:0 <->

3. Configuring Nginx Proxy for Headscale

In this guide we’ll configure Nginx as a proxy server for Headscale. The mappings for IP address and DNS names are:

• vpn.hirebestengineers.com points to 128.140.96.199.

We can confirm the DNS record configured is working.

$ host vpn.hirebestengineers.com
vpn.hirebestengineers.com has address 128.140.96.199

Install Nginx web server in your Ubuntu system.

sudo apt install nginx

Create a virtual host for Headscale.

sudo vim /etc/nginx/conf.d/headscale.conf

Paste below information and update server_name value.

map $http_upgrade $connection_upgrade {
    default      keep-alive;
    'websocket'  upgrade;
    ''           close;
}

server {
    listen 80;
	listen [::]:80;

    server_name vpn.hirebestengineers.com;

    location / {
        proxy_pass  http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $server_name;
        proxy_redirect http:// https://;
        proxy_buffering off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
        add_header Strict-Transport-Security "max-age=15552000; includeSubDomains" always;
    }
}

Confirm nginx configurations are correct.

$ sudo nginx  -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Headscale can be configured to use domain name.

$ sudo vim /etc/headscale/config.yaml
server_url: http://vpn.hirebestengineers.com:80

You’ll need to restart Headscale service for the change to be effected.

4. Securing Headscale with SSL Certificates

We’ll configure Headscale to use TLS. This can be commercial certificates, Free Let’s Encrypt SSL or self-signed certificates.

Using Let’s Encrypt SSL

Start by installing certbot tool

sudo apt update && sudo apt install snapd
sudo snap install --classic certbot

Generate Let’s Encrypt SSL for Headscale using domain name configured in nginx headscale.conf file.

DOMAIN=vpn.hirebestengineers.com
sudo certbot --register-unsafely-without-email --agree-tos --nginx -d $DOMAIN

Successful renew output.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Account registered.
Requesting a certificate for vpn.hirebestengineers.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/vpn.hirebestengineers.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/vpn.hirebestengineers.com/privkey.pem
This certificate expires on 2024-01-02.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

Deploying certificate
Successfully deployed certificate for vpn.hirebestengineers.com to /etc/nginx/conf.d/headscale.conf
Congratulations! You have successfully enabled HTTPS on https://vpn.hirebestengineers.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Certbot tool will automatically inject SSL configurations into the file. You can confirm by viewing its contents.

cat /etc/nginx/conf.d/headscale.conf

Restart nginx web server.

sudo systemctl restart nginx

Finally update Headscale configuration and set the url to domain and https.

$ sudo vim /etc/headscale/config.yaml
server_url: https://vpn.hirebestengineers.com:443

Restart headscale service.

sudo systemctl restart headscale

Note that certificates can also be specified inside Headscale configuration file if not using Nginx proxy.

$ sudo vim /etc/headscale/config.yaml
## Use already defined certificates:
tls_cert_path: ""
tls_key_path: ""

Using custom SSL certificates

If using custom certificates you can modify

map $http_upgrade $connection_upgrade {
    default      keep-alive;
    'websocket'  upgrade;
    ''           close;
}

server {
    listen 80;
	listen [::]:80;

	listen 443      ssl http2;
	listen [::]:443 ssl http2;

    server_name your_fqdn;

    ssl_certificate <PATH_TO_CERT>;
    ssl_certificate_key <PATH_CERT_KEY>;
    ssl_protocols TLSv1.2 TLSv1.3;

    location / {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $server_name;
        proxy_redirect http:// https://;
        proxy_buffering off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
        add_header Strict-Transport-Security "max-age=15552000; includeSubDomains" always;
    }
}

5. Joining Client devices to Headscale mesh

Let’s create a user named computingforgeeks on Headscale server.

$ headscale users create computingforgeeks
User created

Install Tailscale clients

• Windows Client
• iOS
• Android
• Linux
• macOS

Install Tailscale on Linux

To install tailscale on Linux run the following commands.

curl -fsSL https://tailscale.com/install.sh | sudo sh

Checking the version of Tailscale can be done with the commands below.

$ tailscale --version
1.50.1
  tailscale commit: f45c02bfcf5ee5790c3af278c9e974c9b9b0e771
  other commit: 36a20760a45bd1936686879b34c35146cc0c4ec1
  go version: go1.21.1

Installing on macOS

You can install using Homebrew:

brew install tailscale

Or by running the script.

curl -fsSL https://tailscale.com/install.sh | sudo sh

Register a machine (normal login)

On a client machine, execute the tailscale login command:

tailscale up --login-server YOUR_HEADSCALE_URL

See example below.

# tailscale up --login-server http://vpn.hirebestengineers.com

To authenticate, visit:
	https://vpn.hirebestengineers.com:443/register/nodekey:410155d1792d0f81a5f39415a1a418f882208751570c2e5195f7a6842ca44e6a

When you open the link in your browser you’re given the commands to use in registration of the machine being added to Headscale network.

List users created in Headscale server.

$ headscale user list
ID | Name        | Created
1  | myfirstuser | 2023-10-04 14:01:42
2  | jkmutai     | 2023-10-04 14:03:31

Copy and paste the command on the Headscale server terminal while replacing USERNAME with created user.

$ headscale nodes register --user computingforgeeks --key nodekey:410155d1792d0f81a5f39415a1a418f882208751570c2e5195f7a6842ca44e6a
Machine rocky8 registered

To register a machine through headscale the command syntax is:

headscale --user <username> nodes register --key <YOU_+MACHINE_KEY>

You can now list nodes added to Headscale network.

# headscale node list
ID | Hostname | Name   | MachineKey | NodeKey | User    | IP addresses                  | Ephemeral | Last seen           | Expiration          | Online | Expired
1  | rocky8   | rocky8 | [s+TG9]    | [QQFV0] | jkmutai | 100.64.0.1, fd7a:115c:a1e0::1 | false     | 2023-10-05 16:48:58 | 0001-01-01 00:00:00 | online | no

Register machine using a pre authenticated key

You can also register a new machine using pre authenticated key.

First generate a key using the command line:

headscale --user <username> preauthkeys create --reusable --expiration 24h

Executing the command returns a pre-authenticated key used to connect a node to headscale when running the tailscale command:

tailscale up --login-server <YOUR_HEADSCALE_URL> --authkey <YOUR_AUTH_KEY>

Let’s see example below.

# On Headscale server
# headscale --user jkmutai preauthkeys create --reusable --expiration 24h
4763c4f4293b260eff230065378e5668c13db44f4569ed7b

# On Machine to be registered
# tailscale up --login-server http://vpn.hirebestengineers.com --authkey 4763c4f4293b260eff230065378e5668c13db44f4569ed7b

The machine registration command with pre-authenticated key gives no output. But you can confirm if a new nodes is added from Headscale server CLI.

# headscale node list
ID | Hostname | Name   | MachineKey | NodeKey | User    | IP addresses                  | Ephemeral | Last seen           | Expiration          | Online | Expired
1  | rocky8   | rocky8 | [s+TG9]    | [QQFV0] | jkmutai | 100.64.0.1, fd7a:115c:a1e0::1 | false     | 2023-10-05 17:05:58 | 0001-01-01 00:00:00 | online | no
2  | mail     | mail   | [V8WI2]    | [OvPLb] | jkmutai | 100.64.0.2, fd7a:115c:a1e0::2 | false     | 2023-10-05 17:06:32 | 0001-01-01 00:00:00 | online | no

6. Helpful Headscale commands

Delete a node in your network.

headscale node delete -i <ID>

Move node to another user

headscale node move  -i  <ID> -u <New-User>

Rename a machine in your network

headscale node rename  -i  <ID>  <NEW_NAME>

Expire (log out) a machine in your network

headscale node expire -i <ID>

Create a API key:

headscale apikeys create --expiration 90d

List API keys:

headscale apikeys list

Expire an API key:

headscale apikeys expire --prefix "<PREFIX>"