# Troubleshooting

> The failures operators actually hit: wildcard certificates, sandbox DNS, path mode, S3 checksums, email and Node versions.

Source: https://critlab.app/docs/self-hosting/troubleshooting

The failures below are the ones installs actually run into, roughly in order of how often. Most have one cause and one fix.

## Prototypes do not load: certificate errors

**Symptom:** the dashboard works, but the prototype frame is blank or the browser reports a certificate error for a long `….s.example.com` hostname.

**Cause:** the sandbox zone has no valid wildcard certificate. Every prototype version is served from its own subdomain, so a certificate for `s.example.com` alone, or one per host, does not cover it.

**Fix:** issue a certificate for `*.s.example.com` over DNS-01 or from your CA, and attach it to the sandbox Ingress (`ingress.tls.sandboxSecretName`) or your Caddy wildcard site. Check with:

```bash
openssl s_client -connect probe.s.example.com:443 -servername probe.s.example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -subject -ext subjectAltName
```

The SAN list must include `*.s.example.com`. See [Domains and TLS](/docs/self-hosting/domains-tls).

## Prototypes do not load: DNS

**Symptom:** the frame fails with a name-resolution error.

**Fix:** add the wildcard record `*.s.example.com` pointing at your ingress. Check that an arbitrary label resolves:

```bash
dig +short anything-at-all.s.example.com
```

## Prototypes 404 inside the frame

**Causes, in order of likelihood:**

1. **web and sandbox disagree on the URL settings.** `SANDBOX_URL_MODE`, `SANDBOX_ZONE`, `SANDBOX_SCHEME` and `SANDBOX_TOKEN_SECRET` must be identical on both. A URL one issues is otherwise one the other cannot read, and every mismatch looks the same: a 404.
2. **The zone value is wrong.** `SANDBOX_ZONE` is the bare zone (`s.example.com`), with no `*.`, no scheme and no trailing slash.
3. **A proxy rewrites the Host header.** The sandbox recognises a version by its hostname. In Caddy, do not add `header_up Host {http.request.host}` to the sandbox site; the placeholder drops the port and the sandbox silently falls back to path-mode routing.
4. **The sandbox cannot reach Postgres.** It looks up each version's file list there, read-only. Check its log for a connection or auth error; on the Helm chart the sandbox reads the same `DATABASE_URL` as web (or `sandbox.database.existingSecret` for a narrower role) — see [Database and internal lookup for the sandbox](/docs/self-hosting/helm#database-and-internal-lookup-for-the-sandbox).

## A prototype renders unstyled or blank in path mode

With `SANDBOX_URL_MODE=path`, packages that load assets from root-absolute URLs (`/assets/index.js` — Vite, Astro, Create React App, Next.js static export) request them from the sandbox host's root and get 404s. This is the known limitation of path mode, not a bug. Either rebuild the package with relative asset paths (Vite `base: "./"`) or switch to host mode with a wildcard certificate.

## Uploads fail with 413, or larger uploads fail silently

Your ingress rejects the request body before critlab sees it; ingress-nginx allows 1 MB by default. Set `ingress.controller` to your controller and keep `web.ingress.maxBodySize` at 110m or above. If you raised `MAX_PACKAGE_BYTES_PRO`, raise the body size with it (the chart checks).

## Nobody receives sign-in codes

See [Email](/docs/self-hosting/email). In short: without `SMTP_URL` the codes are printed to web's log; with a wrong `SMTP_URL` the send error is in web's log; a relay may reject the default `MAIL_FROM` of `critlab@localhost`.

## Uploads fail with `BadDigest` on SeaweedFS

Only affects your own tooling that uploads to SeaweedFS through presigned URLs: create the AWS SDK v3 client with `requestChecksumCalculation: "WHEN_REQUIRED"`. critlab itself does not presign uploads.

## A workspace's own bucket is refused

- `region_mismatch`: an EU workspace connected a bucket in a US region. The owner can allow cross-region storage in the storage settings.
- Private address refused: the bucket endpoint resolves to a private network address. Set `STORAGE_ALLOW_PRIVATE_ENDPOINTS=1` on web and sandbox if that is intended.
- Verification failed: the probe names the failing step and the store's own error (`AccessDenied`, `NoSuchBucket`, `SignatureDoesNotMatch`, `PermanentRedirect`).

## Presence and live comments do not update

- The WebSocket at `/ws` is cut by the ingress: set a long read timeout (the chart sets one hour for ingress-nginx) and make sure the controller passes the `Upgrade` header.
- `REALTIME_URL` must be the URL browsers use, `wss://…` behind TLS; it is also part of the app's CSP.
- More than one realtime replica needs `REDIS_URL`. Without Redis, comment changes are not pushed live at all and appear when a viewer refocuses the tab.

## Services exit at startup

| Message | Fix |
| --- | --- |
| `SANDBOX_TOKEN_SECRET is missing or shorter than 32 characters` | Set it, identically, on web, sandbox and realtime. |
| `Missing required env var S3_…` | Set the `S3_*` variables on web and sandbox. |
| `DATABASE_URL is not set` | Set it on web (its `migrate` init container reads it too) and sandbox. |
| `AUTH_SECRET is not set` | Set it on web. |

## Running from source: Node.js version

Use Node.js 22.23.4 or newer, which the images use and the repository pins in `.nvmrc` (older releases carry the October 2026 security issues, and critlab logs a warning at start-up on them). Node 24 has crashed the development server and long test runs with segmentation faults in V8 and zlib.

## Still stuck

Collect web, sandbox and realtime logs from the minute the problem happened, the licence state from Settings → License (a platform admin sees it) if a feature is refused, and your values file with secrets removed, and contact Supermode.