Date: 2026-05-04
Host: hetzner (89.167.17.183)
Backend: websockify on 0.0.0.0:6080 (noVNC)
Goal: Make https://vnc.jonahtebaa.com/ reachable only from devices on Jonah's Tailscale tailnet (tail5157c4.ts.net).
A "Tailscale required" gate had been added to the vnc.jonahtebaa.com block in the Caddyfile:
vnc.jonahtebaa.com {
@tailnet remote_ip 100.64.0.0/10 127.0.0.1/32
handle @tailnet { reverse_proxy host.docker.internal:6080 }
handle { respond "Tailscale required" 403 }
}
The gate matches the client's source IP against the Tailscale CGNAT range (100.64.0.0/10). However, every request — including ones from genuine tailnet devices — was returning 403 Tailscale required. The site was unreachable.
agent-caddy runs in Docker. Ports 80 and 443 are published with the default Docker setting userland-proxy: true. When userland proxy handles a published port, it accepts the inbound TCP connection and opens a fresh outbound connection to the container — which means the container sees the bridge gateway IP (172.x.0.1) as the source, not the real client IP.
So Caddy never observed any 100.x.x.x client; the @tailnet matcher could not match; the catch-all 403 always fired.
Verified by hitting the loopback from the host itself (127.0.0.1, which is in the allowlist) — it also returned 403, confirming the source-IP rewrite.
Three fix paths were considered:
| Option | Mechanism | Trade-off |
|---|---|---|
| A | Set userland-proxy: false in /etc/docker/daemon.json and restart docker |
Cleanest — gate works as written. But touches global daemon config and disrupts all 21 containers briefly. |
| B | Use tailscale serve to expose VNC on the tailnet hostname; redirect vnc.jonahtebaa.com to it |
No daemon changes. URL changes to *.ts.net. |
| C | Drop the IP gate, rely on existing SSO cookie | Simplest, but defeats the "tailscale-only" intent. |
Chosen: B (with the redirect approach so the bookmark vnc.jonahtebaa.com still works).
GoDaddy's DNS API is paywalled on Jonah's plan, so DNS-01 cert issuance for a custom domain on a tailnet IP was not feasible. That ruled out keeping vnc.jonahtebaa.com in the browser address bar with a valid cert.
One-time admin enable via the Tailscale console (per-tailnet feature flag):
https://login.tailscale.com/f/serve
Then on hetzner:
tailscale serve --bg --https=443 http://127.0.0.1:6080
This binds Tailscale's internal HTTPS listener on 100.78.215.80:443 (hetzner's tailnet IP) and reverse-proxies to the local websockify on 127.0.0.1:6080. Tailscale auto-issues and auto-renews a Let's Encrypt cert for ubuntu-8gb-hel1-1.tail5157c4.ts.net.
Verified:
https://ubuntu-8gb-hel1-1.tail5157c4.ts.net/ -> HTTP 200
Cert subject: ubuntu-8gb-hel1-1.tail5157c4.ts.net (Let's Encrypt E7)
Reachable from the tailnet only.
vnc.jonahtebaa.comThe original gate block was replaced with a 301 redirect to the tailnet hostname:
vnc.jonahtebaa.com {
import security_headers
redir https://ubuntu-8gb-hel1-1.tail5157c4.ts.net{uri} 301
}
Caddyfile path: /opt/agent/caddy/Caddyfile (bind-mounted into agent-caddy:/etc/caddy/Caddyfile).
Backup of original: /opt/agent/caddy/Caddyfile.bak.1777902594.
Reload (zero downtime):
docker exec agent-caddy caddy reload --config /etc/caddy/Caddyfile
vnc.jonahtebaa.com A record at GoDaddy stays at 89.167.17.183 (hetzner public). This was briefly changed to 100.78.215.80 while exploring a direct-tailnet approach, then reverted. Keeping it on the public IP is required so:
vnc.jonahtebaa.com keeps working (Let's Encrypt must reach the host on public port 80).https://vnc.jonahtebaa.com/.89.167.17.183 (public IP).agent-caddy.301 Location: https://ubuntu-8gb-hel1-1.tail5157c4.ts.net/.*.ts.net hostname only resolves on the tailnet. Tailscale routes the connection to hetzner's tailnet IP 100.78.215.80:443.127.0.0.1:6080 -> noVNC loads.If the user is not on the tailnet, step 6 fails (the .ts.net hostname has no public route), so VNC remains unreachable. The redirect itself is harmless and does not leak access.
ubuntu-8gb-hel1-1.tail5157c4.ts.net, not vnc.jonahtebaa.com. Masking would require a valid cert for vnc.jonahtebaa.com on the tailnet IP, which needs DNS-01 challenge, which needs a DNS provider with API support. GoDaddy does not provide this on Jonah's plan. Future fix: migrate jonahtebaa.com DNS to Cloudflare/Porkbun and configure Caddy DNS-01 against the new provider; then serve vnc.jonahtebaa.com directly on the tailnet IP with a valid cert.8.8.8.8, Quad9 9.9.9.9) cached the old value for a few minutes. Symptom on Mac: ERR_SSL_PROTOCOL_ERROR (Tailscale Serve rejecting SNI for vnc.jonahtebaa.com). Fix: sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder and a fresh tab.To restore the original (broken-but-original) gate:
cp /opt/agent/caddy/Caddyfile.bak.1777902594 /opt/agent/caddy/Caddyfile
docker exec agent-caddy caddy reload --config /etc/caddy/Caddyfile
tailscale serve --https=443 off
| Path | Change |
|---|---|
/opt/agent/caddy/Caddyfile |
vnc.jonahtebaa.com block replaced with redir 301 |
/opt/agent/caddy/Caddyfile.bak.1777902594 |
Backup of pre-change Caddyfile |
| Tailscale Serve config (in tailscaled state) | https=443 -> http://127.0.0.1:6080 enabled |
.ts.net hostname is renewed by tailscaled, not Caddy. No cron action required.vnc.jonahtebaa.com via HTTP-01 (DNS still public, port 80 open).gate_auth + gate_check) was removed from the vnc block when it was replaced with the redirect — Tailscale membership is now the sole access control. If layered defense is desired, the SSO gate can be re-added on the Caddy side before the redirect, or the noVNC backend can require its own auth.