portal-relay-diagnose git:20261108.f9aaa73A

Immutable. This exact content never changes and is served at /api/v1/blob/265c39d886bb92d3.

---
name: portal-relay-diagnose
description: Find which layer of a self-hosted Portal relay is failing - container and process, open ports, embedded DNS and NS/glue delegation, DNSSEC and the parent DS, TLS and SNI routing, the QUIC UDP backhaul and TCP/UDP lease ports, or approval and source policy - with bounded read-only checks, then report the cause and the fix. Use when a relay operator says the relay, its DNS, its certificates, its tunnels, or its game ports do not work. Do not use when the user is a publisher or visitor of one tunnel (portal-expose or portal-connect), for approving or banning leases (portal-relay-admin), or for first deployment (portal-relay).
license: MIT
---

# Diagnose a Portal relay

Work from the bottom layer up and stop at the first layer that fails: a DNS check means nothing while the container is down. Every check here is read-only and bounded (`curl --max-time 10`, `dig +time=3 +tries=1`, `--tail 200` on logs). Do not restart, recreate, or reconfigure anything during diagnosis; propose the fix and let the operator approve it.

Run checks from two places when possible, because a check on the relay host passes through no cloud firewall:

- **Host**: a shell on the relay machine.
- **Outside**: any other network, for example the agent's own environment.

Say which place each result came from.

## 1. Process and container (host)

- `docker compose ps portal` is running and not restarting.
- `docker compose logs portal --tail 200`: look for bind errors (`address already in use` on `443` or `53`), `permission denied` on `IDENTITY_PATH` (the image runs as uid/gid `65532`), embedded DNS start failures, and the `ds_record` line.
- `sudo ss -tulpn | grep -E ':(443|53) '`: the relay owns `443/tcp`, `53/tcp`, `53/udp`, and `443/udp` when UDP leases are on. Another process on `53` is often `systemd-resolved`.

## 2. Ports (outside)

- `curl -fsS --max-time 10 https://<relay>/api/healthz`. A timeout with the container healthy points to a cloud firewall or host firewall, not to Portal.
- `dig +time=3 +tries=1 @<relay IP> <relay domain> SOA` and the same with `+tcp`: one passing and one failing means only one of `53/udp` and `53/tcp` is open.
- Compare with the provider's firewall rules. OCI images also reject inbound in host iptables; Lightsail and Hetzner use their own firewalls in front of the host.

## 3. DNS delegation (outside)

1. `dig @<relay IP> test.<relay domain> A` answers: the embedded DNS works.
2. `dig +short NS <relay domain> @1.1.1.1` returns `ns.<relay domain>`, and `dig +short A ns.<relay domain> @1.1.1.1` (and `AAAA`) returns the relay address: the parent delegation and glue are right.
3. If 1 passes and 2 fails, the problem is at the parent zone: a missing NS, a wrong glue address (common after a public IP changed), a proxied glue record, or propagation still in progress. Read the parent's records with `dig @<parent NS> <relay domain> NS +norec`.
4. An empty AAAA on an IPv4-only relay is expected.

## 4. DNSSEC (outside)

- `dig DS <relay domain> @1.1.1.1` compared with the `ds_record` log line: key tag, algorithm `13`, digest type `2`, and digest must match exactly.
- `dig +dnssec <relay domain> SOA @1.1.1.1`: `SERVFAIL` from a validating resolver with a DS present, while `dig +cd` succeeds, means a DS mismatch or a replaced `dnssec-csk.json`. The fix is to publish the DS from the current key or to restore the old key file, never to delete the key.
- No DS at the parent means DNSSEC is simply not chained yet; it is not a failure.

## 5. TLS and SNI (outside)

- `openssl s_client -connect <relay IP>:443 -servername <relay domain> </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -dates` for the relay's own name.
- A tunnel hostname that fails the TLS handshake while the relay name works means no routable lease for that name (offline, pending approval, denied), not a certificate problem. Check it in layer 7.
- With `ACME_DNS_PROVIDER` set to an external provider, look for provider authentication errors in the logs.

## 6. Transport (outside, only when enabled)

- UDP leases need `443/udp` published and open, because the QUIC backhaul uses the public `PORTAL_URL` port.
- TCP/UDP lease ports: the Docker published range, the host firewall, and the cloud firewall must all cover the same `MIN_PORT`-`MAX_PORT` range. Probe one allocated port only, for example `nc -vz -w 5 <relay IP> <port>`. Never scan the range.
- Error codes from a publisher: `udp_disabled`, `tcp_port_disabled` (lease type off), `*_capacity_exceeded` (`max_leases` reached), and `*_port_exhausted` (range too small).

## 7. Policy (needs portal-relay-admin)

- `lease_rejected`: the lease waits for approval in `manual` mode. `lease_denied`: the identity is denied or banned.
- `source_not_allowed`: the publisher's address is outside `TUNNEL_SOURCE_CIDRS`. Behind a proxy that also needs `TRUST_PROXY_HEADERS` and `TRUSTED_PROXY_CIDRS`.
- Read the lease state through `portal-relay-admin`; do not change policy as part of diagnosis.

## Report

Give the first failing layer, the evidence (the command, where it ran, and its output), the most likely cause, and the fix as a proposal. List the layers that passed and the checks that could not run (for example no shell on the host). Do not print `ADMIN_TOKEN`, provider tokens, or the contents of `IDENTITY_PATH`.