diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md new file mode 100644 index 0000000..3a2575a --- /dev/null +++ b/DEPLOYMENT.md @@ -0,0 +1,287 @@ +# Deployment Guide — theta42/proxy + +The proxy is an OpenID Connect-protected reverse proxy (OpenResty front + Node +management app + Redis) that is **both** an OIDC client of an SSO Manager *and* +a direct LDAP client for user lookups. Two deployment methods: + +1. **Docker** — a single all-in-one image bundling OpenResty + the app + Redis + (`docker compose up`). +2. **Bare metal** — `ops/install.sh` on Debian/Ubuntu (installs Node.js, + OpenResty, Lua modules, Redis, and a systemd unit). + +## How configuration works + +The app loads configuration via [`@simpleworkjs/conf`](https://www.npmjs.com/package/@simpleworkjs/conf), +which deep-merges, in order: + +1. `conf/base.js` (committed, generic defaults) +2. `conf/.js` (optional) +3. `conf/secrets.js` (gitignored — secrets + per-deployment values) +4. **`app_*` environment variables** — the highest-precedence layer + +Any env var whose name starts with `app_` overrides the merged config. The rest +of the name is split on **double-underscore** (`__`) into a nested path. Values +are `JSON.parse`-coerced when possible and kept as raw strings otherwise. + +| Env var | Sets | Notes | +|---------|------|-------| +| `app_oidc__issuer` | `conf.oidc.issuer` | browser-facing SSO URL | +| `app_oidc__authorizationEndpoint` | `conf.oidc.authorizationEndpoint` | browser-facing | +| `app_oidc__tokenEndpoint` | `conf.oidc.tokenEndpoint` | server-to-server; can be internal | +| `app_oidc__userinfoEndpoint` | `conf.oidc.userinfoEndpoint` | server-to-server; can be internal | +| `app_oidc__endSessionEndpoint` | `conf.oidc.endSessionEndpoint` | browser-facing | +| `app_oidc__clientId` / `app_oidc__clientSecret` | OIDC client creds | register in the SSO first | +| `app_oidc__redirectUri` | `conf.oidc.redirectUri` | must match the SSO client exactly | +| `app_oidc__enabled` | `conf.oidc.enabled` | boolean | +| `app_ldap__url` | `conf.ldap.url` | `ldaps://…:636` or `ldap://…:389` | +| `app_ldap__bindDN` / `app_ldap__bindPassword` | LDAP service account | don't reuse the admin DN | +| `app_ldap__searchBase` / `app_ldap__userFilter` | user search | | +| `app_ldap__tlsOptions__rejectUnauthorized` | `conf.ldap.tlsOptions.rejectUnauthorized` | `false` for self-signed LDAPS | +| `app_ldap__tlsOptions__ca` | `conf.ldap.tlsOptions.ca` | path to a CA cert for strict trust | +| `app_auth__adminUsers` | `conf.auth.adminUsers` | local anti-lockout admin (uid) | +| `app_auth__adminGroups` | `conf.auth.adminGroups` | SSO/LDAP groups that are global admin (JSON array) | +| `app_auth__localAdminPass` | `conf.auth.localAdminPass` | initial password for the local anti-lockout admin (used once, on first creation only — defaults to the username itself if unset) | +| `app_redis__prefix` | `conf.redis.prefix` | default `proxy_` | + +See [`docs/docker.md`](docs/docker.md) for a shorter, container-focused version +of this reference. + +> **Requires `@simpleworkjs/conf` >= 1.1.0.** The Docker image will not honor +> `app_*` env vars on 1.0.0. The lock is already on `^1.1.0`; if you regenerate it: +> ```bash +> cd nodejs && npm install @simpleworkjs/conf@^1.1.0 +> ``` + +--- + +## Method 1: Docker (all-in-one) + +The image (`Dockerfile`) bundles OpenResty + the app + Redis in one container, +mirroring the bare-metal `ops/install.sh` layout. `docker-entrypoint.sh`: +generates the fallback SSL cert, parameterizes the OpenResty `resolver`/ +`set_real_ip_from` directives, starts Redis + the node app, and execs OpenResty +in the foreground under `dumb-init`. + +### Setup + +The bundled `docker-compose.yml` reads the OIDC + LDAP + auth wiring from a +bind-mounted `./config/proxy-secrets.js` (not from `app_*` env). Copy the +example, fill in your secrets, then build + start: + +```bash +mkdir -p config && chmod 700 config +cp secrets.js.example config/proxy-secrets.js +$EDITOR config/proxy-secrets.js # set oidc.clientId/clientSecret, ldap.bindPassword, ... +docker compose up -d --build +``` + +`docker-entrypoint.sh` sets `CONF_SECRETS=/config/proxy-secrets.js` so +`@simpleworkjs/conf` reads it directly. No `app_*` env is passed — `app_*` env +would override the file (env beats secrets.js in `@simpleworkjs/conf`), so the +file is kept authoritative. `RESOLVER` / `REAL_IP_FROM` / `NODE_ENV` / +`NODE_PORT` are OpenResty-runtime / process env, not `app_*` config, so they +stay in the compose. + +> Running the unified `theta-env` stack? Its `setup.sh` generates +> `./config/proxy-secrets.js` (+ `./config/sso-secrets.js`) for you and +> registers the OAuth client with the SSO — see the theta-env README. + +### Access + +- Proxy (public): `https:///` — OpenResty front, auto-SSL (Let's Encrypt) +- Management UI / API: `http://127.0.0.1:3000/` (bound to localhost; the front + proxies the UI under its own TLS) +- Health: `http://127.0.0.1:3000/health` → `{"status":"ok"}` + +### API tokens (personal access tokens) + +Any logged-in user can mint a long-lived bearer token to call the management +API from scripts/CI/other services, without an OIDC browser session. Tokens are +self-service and authenticate **as their creator**: the creator's groups are +snapshotted at mint time (mirroring how the proxy's browser session captures +groups at login — the proxy never re-queries the IdP), and the existing authz +layer (`Permission.effectiveFor` / `roles.resolveEffective`) applies unchanged. +Local groups and owned-domain rights are recomputed live each request; only the +SSO/LDAP group membership is the mint-time snapshot. + +Create one in the UI under **API Tokens** (the token string is shown **once**), +then use it as a bearer token: + +```bash +curl -H "Authorization: Bearer prx__" https://proxy.example.com/api/host +``` + +Format: `prx__` — the `id` is the lookup key, the `secret` is +bcrypt-hashed and never stored in plaintext. Rotate or revoke from the same page +(immediate effect). Optional expiry (in days) at creation. Tokens persist in the +bundled Redis (AOF — see *Backups and restore*), so they survive rebuilds. + +The token carries the creator's effective rights: a global admin's token can +manage Hosts/Users/Groups; a domain manager's token can manage their own +domains but `requireAdmin` routes return 403. To tighten permissions after group +changes, revoke and re-mint the token. + +### OpenResty runtime env + +| Variable | Default | Description | +|----------|---------|-------------| +| `RESOLVER` | `127.0.0.11` | DNS for upstream names in Host records (Docker's embedded DNS) | +| `REAL_IP_FROM` | _empty_ | Trusted CIDR for `X-Real-IP`. Empty = proxy is the front (removes the real_ip block). Set to an upstream proxy's CIDR if one sits in front. | + +### Auto-SSL / Let's Encrypt + +`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB +persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs +survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is +required for HTTP-01 challenges (mapped in the compose). + +### Backups and restore + +**What lives where** + +| State | Location | Persisted? | +|-------|----------|------------| +| Host records, permissions, DNS creds, local users | `proxy-data` volume (`/data`, Redis) | yes (AOF + RDB) | +| Let's Encrypt certs (auto-ssl) | `proxy-data` volume (`/data`, Redis) | yes — same Redis | +| nginx response cache / logs | `proxy-cache` / `proxy-logs` volumes | yes (volume) | +| Secrets (OIDC client secret, LDAP bind password) | `./config/proxy-secrets.js` (bind mount) | your responsibility — back up off-host | + +**Automatic snapshots** — when run as part of the unified `theta-env` stack, +`setup.sh` snapshots Redis + `./config/` to `./backups//` before every +rebuild and keeps the last `BACKUP_KEEP` (default 5). Standalone deployments +should run `ops/backup.sh` the same way (on a cron/systemd timer, or by hand +before an upgrade): + +```bash +./ops/backup.sh # keeps the last 5 by default +./ops/backup.sh 10 # or override retention +BACKUP_KEEP=10 ./ops/backup.sh +``` + +It snapshots Redis (`BGSAVE`, falling back to a synchronous `SAVE` if that +doesn't complete quickly) and `./config/` to `./backups//`, +pruning older backups beyond the retention count — the same approach +`theta-env`'s `setup.sh` uses, just scoped to this one container. Equivalent +manual steps, if you'd rather not use the script: + +```bash +# Redis — hot snapshot: trigger a save, then copy the RDB out +docker compose exec proxy redis-cli BGSAVE +docker compose cp proxy:/data/dump.rdb proxy-redis-$(date +%F).rdb + +# Secrets — copy the config dir (holds OIDC client secret, LDAP bind password) +cp -a ./config config-backup-$(date +%F) && chmod 700 config-backup-$(date +%F) +``` +Store the backup **off the host** — it contains secrets and the whole +Host/permission/user dataset. + +**Restore — Redis (full proxy state + certs)** + +```bash +cp -a config-backup- ./config && chmod 700 ./config +docker compose up -d +docker compose stop proxy +# AOF wins on startup — delete it so the RDB is loaded instead (see note). +docker compose run --rm --no-deps --entrypoint sh proxy -c \ + 'rm -f /data/appendonly.aof /data/appendonly.aof.*' +docker compose cp proxy-redis-.rdb proxy:/data/dump.rdb +docker compose start proxy +``` + +> **AOF vs RDB (important):** with `--appendonly yes`, Redis loads +> `appendonly.aof` on startup and **ignores** `dump.rdb` if the AOF exists. To +> restore from an RDB snapshot you **must delete the AOF first** (the runbook +> does this); Redis then loads the RDB and writes a fresh AOF. Verify: +> `docker compose exec proxy redis-cli DBSIZE`. +> +> Restoring Redis restores cert state **at the snapshot time** — certs issued +> after the snapshot are lost and will be re-issued on next request. + +**Upgrades** + +```bash +./setup.sh # backs up, then rebuilds — proxy-data keeps Redis state +# (standalone) docker compose pull && docker compose up -d +``` +Host records, permissions, DNS creds, local users, and Let's Encrypt certs all +survive the rebuild because they live on the `proxy-data` volume, not in the +image. **Migrations note:** if a release ships a `nodejs/migrations/` script, +run it after upgrading (it transforms in-Redis records); see the release notes. + +### Logs + +OpenResty runs in the foreground and the Node app in the background, both +writing to the container's stdout/stderr. nginx access/error logs go to files +(`/var/log/nginx`, on the `proxy-logs` volume), so they do **not** appear in +`docker logs`. + +```bash +docker compose logs -f proxy # app + OpenResty (stdout/stderr) +docker compose exec proxy tail -f /var/log/nginx/error.log # nginx errors +docker compose exec proxy tail -f /var/log/nginx/access.log # nginx access +docker compose logs --tail=200 --since=10m proxy # recent context +``` + +--- + +## Method 2: Bare metal (Debian/Ubuntu) + +`ops/install.sh` is an idempotent installer: it installs Node.js 22.x, OpenResty +(from openresty.org), Lua modules (luarocks), Redis, force-syncs the repo to +`/opt/theta42/proxy`, symlinks the OpenResty + systemd config from the repo, and +starts `proxy.service`. Re-run it to update — it prints the version you're +updating from and to (or "Already up to date" if there's nothing new). + +```bash +wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash +``` + +or, if you already have the repo checked out: + +```bash +sudo ./ops/install.sh +``` + +Configuration is file-based: on first run the installer seeds +`/etc/proxy/secrets.js` from `secrets.js.example` (placeholders you must fill +in — OIDC + LDAP values, see `nodejs/conf/base.js` for the shape). Edit it, +then `sudo systemctl restart proxy`. Later runs never touch an existing +secrets file. + +--- + +## Fronting an SSO Manager + +The proxy is a natural front for [`theta42/sso-manager-node`](https://github.com/theta42/sso-manager-node): +it terminates TLS for the SSO's UI and protects it with OIDC login, while also +binding to the SSO's LDAP directly for user lookups. To run both together: + +1. **One Docker network** so the proxy can reach the SSO internally at + `http://sso-manager:3001` (token/userinfo, server-to-server) and + `ldaps://sso-manager:636` (LDAP). +2. **Set the SSO's `OAUTH_ISSUER`** to the browser-facing HTTPS URL the proxy + serves the SSO at (e.g. `https://sso.example.com`). +3. **Register the proxy as an OIDC client** in the SSO, with a `redirectUri` + matching the proxy's callback (`https://proxy.example.com/api/auth/oidc/callback`). +4. **LDAP**: point `app_ldap__url` at `ldaps://sso-manager:636` and create a + dedicated service account (`cn=ldapclient,ou=people,…`) — don't reuse the + admin DN. For the SSO's self-signed LDAPS cert, set + `app_ldap__tlsOptions__rejectUnauthorized=false` (or mount the cert and use + `app_ldap__tlsOptions__ca=`). + +The [`theta42/theta-env`](https://github.com/theta42/theta-env) unified repo +automates all four steps with `./setup.sh`. + +--- + +## Security notes + +1. **Never commit `secrets.js`** — it's in `.gitignore`. +2. **Bind the management port to localhost** (the compose does: `127.0.0.1:3000`). + The OpenResty front proxies the UI under TLS; don't expose 3000 to the LAN. +3. **`REAL_IP_FROM` empty by default** — the proxy trusts no one to set `X-Real-IP` + (it's the front). Only set it if a trusted proxy sits in front. +4. **LDAPS for any LDAP that crosses the network.** Use `ldaps://`/StartTLS; plain + `ldap://` is fine only on a private docker network. +5. The image runs OpenResty workers as `nobody` and the node app as root (matches + the bare-metal systemd unit). Harden to a non-root user for production if needed. \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..8203639 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,39 @@ +# Documentation + +This directory contains the GitHub Pages documentation site for the Proxy project. + +**Live site:** https://theta42.github.io/proxy/ + +## Pages + +- `index.md` - Home page with project overview +- `installation.md` - Installation and setup guide +- `api.md` - Complete API reference +- `architecture.md` - System architecture and design +- `contributing.md` - Development and contribution guide + +## Local Preview + +To preview the site locally: + +```bash +# Install Jekyll (one-time setup) +gem install jekyll bundler + +# Run local server +cd docs +jekyll serve + +# View at http://localhost:4000/proxy/ +``` + +## Theme + +The site uses the Cayman theme (`jekyll-theme-cayman`). Configuration is in `_config.yml`. + +## Updating Documentation + +1. Edit markdown files in this directory +2. Commit and push to master branch +3. GitHub Pages automatically rebuilds (may take 1-2 minutes) +4. Changes visible at https://theta42.github.io/proxy/ diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 0000000..560f548 --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1,47 @@ +title: Proxy +description: A reverse proxy and HTTPS termination service built on OpenResty/nginx, with an OIDC + LDAP-aware management API and web GUI. +url: "https://theta42.github.io" +baseurl: "/proxy" +logo: /assets/img/theta42.svg +lang: en_US + +plugins: + - jekyll-seo-tag + - jekyll-sitemap + +github: + repository_url: https://github.com/theta42/proxy + zip_url: https://github.com/theta42/proxy/archive/refs/heads/master.zip + tar_url: https://github.com/theta42/proxy/archive/refs/heads/master.tar.gz + repository_name: theta42/proxy + +nav: + - title: Home + page: / + icon: fa-house + - title: Installation + page: /installation.html + icon: fa-download + - title: Architecture + page: /architecture.html + icon: fa-sitemap + - title: API + page: /api.html + icon: fa-code + - title: Docker + page: /docker.html + icon: fa-box + - title: Contributing + page: /contributing.html + icon: fa-code-branch + - title: Changelog + url: https://github.com/theta42/proxy/blob/master/CHANGELOG.md + icon: fa-list + +defaults: + - scope: + path: "" + type: "pages" + values: + layout: default + image: /assets/img/theta42.svg diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html new file mode 100644 index 0000000..132c524 --- /dev/null +++ b/docs/_layouts/default.html @@ -0,0 +1,82 @@ + + + + + + + + {% seo title=false %} + {% if page.title %}{{ page.title }} · {% endif %}{{ site.title }} + + + + + + + + + +
+
+
+
+
+
+ {{ content }} +
+
+
+
+
+
+ + + + + + diff --git a/docs/api.md b/docs/api.md new file mode 100755 index 0000000..a497b71 --- /dev/null +++ b/docs/api.md @@ -0,0 +1,972 @@ +--- +layout: default +title: API Reference +description: The proxy's management REST API — hosts, DNS providers, users, groups, and permissions. +--- + +# API Documentation + +[← Back to Home](index.html) + +All API endpoints require authentication unless otherwise noted. Three +authentication methods are supported: + +- **`auth-token` header** — a browser-session token from `POST /api/auth/login` + or the OIDC flow (below). +- **`Authorization: Bearer ` header** — a self-service API token (PAT, + see [API Tokens](#api-tokens)), for scripts/CI without a browser session. +- **OIDC (browser)** — if the proxy is configured as an OIDC client of an SSO + (`app_oidc__*` / `conf.oidc`, see [DEPLOYMENT.md](https://github.com/theta42/proxy/blob/master/DEPLOYMENT.md)), + users can log in via `GET /api/auth/oidc/start` instead of posting a + username/password. + +The proxy can also be configured as a **direct LDAP client** (`app_ldap__*` / +`conf.ldap`) for looking up/validating users, independent of the OIDC flow — +see DEPLOYMENT.md for the full configuration reference. + +Authenticated requests also carry **RBAC** (role-based access control): +global admins can manage everything; other users are scoped to `viewer` or +`manager` rights on specific domains via [Permissions](#permissions) and +[Groups](#groups). + +Base URL: `https://your-proxy-host.com/api` + +--- + +## Authentication + +### Login + +**POST** `/api/auth/login` + +Authenticate a user and receive an auth token. + +```bash +curl -H "Content-Type: application/json" \ + -X POST \ + -d '{"username": "myuser", "password": "mypassword"}' \ + https://proxy-host.com/api/auth/login +``` + +**Responses:** +- `200` `{"login": true, "token": "027d3964-7d81-4462-a6f9-2c1f9b40b4be", "message": "myuser logged in!"}` +- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` + +### Logout + +**ALL** `/api/auth/logout` + +Invalidate the current auth token. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/auth/logout +``` + +**Responses:** +- `200` `{"message": "Bye"}` + +### OIDC Login (start) + +**GET** `/api/auth/oidc/start` + +Begin the OIDC authorization-code flow: creates a PKCE + state challenge and +redirects the browser to the configured SSO's authorize endpoint. Only +available when `conf.oidc.enabled` is true. + +**Query Parameters:** +- `redirect` - Internal path to return to after login (optional; sanitized to same-origin) + +```bash +curl -i "https://proxy-host.com/api/auth/oidc/start?redirect=/hosts" +``` + +**Responses:** +- `302` Redirect to the SSO's authorization endpoint +- `404` `{"name": "OidcDisabled", "message": "OIDC login is not enabled."}` + +### OIDC Callback + +**GET** `/api/auth/oidc/callback` + +Redirect target for the SSO after login. Validates the one-time `state`, +exchanges the authorization `code` for tokens, reads identity from the +userinfo endpoint, establishes a session, and redirects the browser back to +the login page with the app's own `auth-token` in a URL fragment. + +**Query Parameters:** +- `code` (required) - Authorization code from the SSO +- `state` (required) - State value from the `start` step + +```bash +# Not called directly — the SSO redirects the browser here after login. +``` + +**Responses:** +- `302` Redirect to `/login#token=...&redirect=...` +- `400` `{"name": "OidcCallbackInvalid", "message": "Missing code or state."}` or expired/unknown state + +--- + +## API Tokens + +Self-service personal access tokens (PATs) for scripting/CI without a browser +session. Every endpoint is owner-scoped: a user only sees/manages tokens they +created. Mounted at `/api/api-token`. + +### List API Tokens + +**GET** `/api/api-token` + +List the current user's API tokens. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/api-token +``` + +**Responses:** +- `200` `{"results": [{"id": "...", "name": "ci", ...}, ...]}` + +### Create API Token + +**POST** `/api/api-token` + +Create a new API token. The raw token string is only returned once, at +creation. + +**Parameters:** +- `name` (required) - Display name +- `description` (optional) +- `expires_in_days` (optional) - `0` or omitted means no expiry + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "ci", "expires_in_days": 90}' \ + https://proxy-host.com/api/api-token +``` + +**Responses:** +- `200` `{"results": {...}, "token": "prx__", "message": "API token 'ci' created. Save it now — it will not be shown again."}` + +### Get API Token + +**GET** `/api/api-token/:id` + +Get a token's metadata (not the raw secret, which is never stored/returned again). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/api-token/ +``` + +**Responses:** +- `200` `{"results": {...}}` +- `403` Not your token + +### Update API Token + +**PUT** `/api/api-token/:id` + +Update a token's name/description/expiry. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"name": "ci-updated"}' \ + https://proxy-host.com/api/api-token/ +``` + +**Responses:** +- `200` `{"results": {...}, "message": "API token 'ci-updated' updated."}` + +### Delete (Revoke) API Token + +**DELETE** `/api/api-token/:id` + +Revoke a token immediately. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/api-token/ +``` + +**Responses:** +- `200` `{"id": "", "message": "API token 'ci' revoked."}` + +### Rotate API Token + +**POST** `/api/api-token/:id/rotate` + +Issue a new secret for an existing token (same id, new raw value shown once). + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/api-token//rotate +``` + +**Responses:** +- `200` `{"token": "prx__", "message": "API token 'ci' rotated. Save it — it will not be shown again."}` + +--- + +## Users + +All user endpoints require authentication. `GET /me` and `PUT /password` +(self-service) work for any authenticated user; everything else (listing, +creating, deleting users, resetting another user's password) requires global +admin. + +### List Users + +**GET** `/api/user` + +Get list of all users. Admin only. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/user +``` + +**Query Parameters:** +- `detail` - Include full user details (optional) + +**Responses:** +- `200` `{"results": ["user1", "user2"]}` +- `200` `{"results": [{"username": "user1", ...}, ...]}` (with `?detail=true`) +- `403` Not an admin + +### Get Current User + +**GET** `/api/user/me` + +Get the currently authenticated user's identity and effective RBAC rights +(drives the web UI's nav/button gating). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/user/me +``` + +**Responses:** +- `200` `{"username": "myuser", "groups": [...], "localGroups": [...], "externalGroups": [...], "isAdmin": false, "global": null, "domains": {...}}` + +### Create User + +**POST** `/api/user` + +Create a new local user. Admin only. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"username": "newuser", "password": "newpassword"}' \ + https://proxy-host.com/api/user +``` + +**Responses:** +- `200` User created successfully +- `403` Not an admin +- `409` Username already exists +- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error (also returned for weak passwords) + +### Delete User + +**DELETE** `/api/user/:username` + +Delete a user account. Admin only. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/user/olduser +``` + +**Responses:** +- `200` `{"username": "olduser", "results": ...}` +- `403` Not an admin +- `404` User not found + +### Change Password (Self) + +**PUT** `/api/user/password` + +Change the password for the currently authenticated user. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"password": "newpassword"}' \ + https://proxy-host.com/api/user/password +``` + +**Responses:** +- `200` `{"results": ...}` Password changed successfully +- `422` Weak password rejected by the password policy + +### Change Password (Other User) + +**PUT** `/api/user/password/:username` + +Change the password for another user. Admin only. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"password": "newpassword"}' \ + https://proxy-host.com/api/user/password/otheruser +``` + +**Responses:** +- `200` `{"results": ...}` Password changed successfully +- `403` Not an admin +- `404` User not found + +--- + +## Permissions + +RBAC: grants a `viewer` or `manager` role to a user or group, either globally +or scoped to one domain. Global-admin-only. Mounted at `/api/permission`. + +### List Permissions + +**GET** `/api/permission` + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/permission +``` + +**Responses:** +- `200` `{"results": [{"id": "...", "subjectType": "user", "subject": "alice", "role": "manager", "scope": "domain", "domain": "example.com", ...}, ...]}` + +### List Permission Subjects + +**GET** `/api/permission/subjects` + +Autocomplete source for the "Subject" field: known usernames plus known group +names (local groups, groups already used in permissions, and groups from +`conf.auth.adminGroups` / `conf.auth.groupRoleMap`). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/permission/subjects +``` + +**Responses:** +- `200` `{"users": ["alice", "bob"], "groups": ["ops", "sre"]}` + +### Create Permission + +**POST** `/api/permission` + +Grant a role to a subject. + +**Parameters:** +- `subjectType` (required) - `user` or `group` +- `subject` (required) - username or group name +- `role` (required) - `viewer` or `manager` +- `scope` (required) - `global` or `domain` +- `domain` (required if `scope` is `domain`) + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"subjectType": "user", "subject": "alice", "role": "manager", "scope": "domain", "domain": "example.com"}' \ + https://proxy-host.com/api/permission +``` + +**Responses:** +- `200` `{"message": "Granted manager to user \"alice\" on example.com.", ...}` +- `422` Validation error + +### Delete Permission + +**DELETE** `/api/permission/:id` + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/permission/ +``` + +**Responses:** +- `200` `{"message": "Permission removed."}` + +--- + +## Groups + +Local groups (independent of any SSO/LDAP groups) used as subjects for +permission grants. Global-admin-only. Mounted at `/api/group`. + +### List Groups + +**GET** `/api/group` + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/group +``` + +**Responses:** +- `200` `{"results": [{"name": "ops", "members": ["alice", "bob"], ...}, ...]}` + +### Create Group + +**POST** `/api/group` + +**Parameters:** +- `name` (required) +- `members` (optional) - array of usernames + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "ops", "members": ["alice"]}' \ + https://proxy-host.com/api/group +``` + +**Responses:** +- `200` `{"message": "Group \"ops\" created.", ...}` + +### Delete Group + +**DELETE** `/api/group/:name` + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/group/ops +``` + +**Responses:** +- `200` `{"message": "Group \"ops\" removed."}` + +### Add Group Member + +**POST** `/api/group/:name/members` + +**Parameters:** +- `username` (required) + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"username": "bob"}' \ + https://proxy-host.com/api/group/ops/members +``` + +**Responses:** +- `200` `{"message": "Added \"bob\" to \"ops\".", ...}` + +### Remove Group Member + +**DELETE** `/api/group/:name/members/:username` + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/group/ops/members/bob +``` + +**Responses:** +- `200` `{"message": "Removed \"bob\" from \"ops\".", ...}` + +--- + +## Hosts + +Manage proxy host configurations. + +### List Hosts + +**GET** `/api/host` + +Get list of all configured hosts. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host +``` + +**Query Parameters:** +- `detail` - Include full host details (optional) + +**Responses:** +- `200` `{"results": ["example.com", "*.wildcard.com"]}` +- `200` `{"results": [{"host": "example.com", "ip": "192.168.1.10", ...}, ...]}` (with `?detail=true`) + +### Get Host + +**GET** `/api/host/:host` + +Get configuration for a specific host. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"item": "example.com", "results": {"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, ...}}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` + +### Lookup Host + +**GET** `/api/host/lookup/:domain` + +Test the host lookup algorithm (supports wildcard matching). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/lookup/sub.example.com +``` + +**Responses:** +- `200` `{"string": "sub.example.com", "results": {"host": "*.example.com", ...}}` +- `200` `{"string": "sub.example.com", "results": null}` (no match) + +### Get Lookup Tree + +**GET** `/api/host/lookupobj` + +Get the internal lookup tree structure (for debugging). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/lookupobj +``` + +**Responses:** +- `200` `{"results": {"com": {"example": {...}}}}` + +### Create Host + +**POST** `/api/host` + +Add a new host configuration. + +**Parameters:** +- `host` (required) - Domain name (e.g., `example.com`, `*.example.com`) +- `ip` (required) - Target IP address or FQDN +- `targetPort` (required) - Target port number (1-65535) +- `forcessl` (optional) - Force HTTPS redirect (default: true) +- `targetssl` (optional) - Use HTTPS to backend (default: false) +- `challengeType` (optional) - For wildcards: `DNS-01-wildcard` or `wildcardChild` + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, "forcessl": true, "targetssl": false}' \ + https://proxy-host.com/api/host +``` + +**Responses:** +- `200` `{"message": "\"example.com\" added.", "host": "example.com", ...}` +- `409` `{"name": "HostNameUsed", "message": "Host already exists"}` +- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error + +### Update Host + +**PUT** `/api/host/:host` + +Update an existing host configuration. + +**Parameters:** Same as Create Host (all optional) + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"ip": "192.168.1.20", "targetPort": 9000}' \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"message": "\"example.com\" updated.", ...}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` +- `422` Validation error + +### Delete Host + +**DELETE** `/api/host/:host` + +Remove a host configuration. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"message": "example.com deleted", ...}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` + +### Clear Host Cache + +**DELETE** `/api/host/cache` + +Remove all cached wildcard-subdomain host lookups. Cache entries are created on +demand when a wildcard host serves a subdomain; clearing them forces the next +request for each subdomain to be resolved fresh through the lookup tree. +Admin only. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/host/cache +``` + +**Responses:** +- `200` `{"message": "Cleared 3 cached hosts.", "count": 3}` + +### Renew Wildcard Certificate + +**PUT** `/api/host/:host/renew` + +Manually trigger wildcard certificate renewal. + +```bash +curl -H "auth-token: your-token-here" \ + -X PUT \ + https://proxy-host.com/api/host/*.example.com/renew +``` + +**Responses:** +- `200` `{"message": "Requesting wildcard cert for *.example.com"}` +- `404` Host not found + +--- + +## DNS Providers + +Manage DNS provider integrations for wildcard SSL certificates. + +### List DNS Providers + +**GET** `/api/dns` + +Get list of configured DNS providers. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns +``` + +**Query Parameters:** +- `detail` - Include full provider details (optional) + +**Responses:** +- `200` `{"results": ["provider-id-1", "provider-id-2"]}` + +### List Available Provider Types + +**OPTIONS** `/api/dns` + +Get list of supported DNS provider types and their configuration requirements. + +```bash +curl -H "auth-token: your-token-here" \ + -X OPTIONS \ + https://proxy-host.com/api/dns +``` + +**Responses:** +- `200` `{"results": [{"name": "Cloudflare", "fields": {...}}, {"name": "DigitalOcean", ...}, {"name": "PorkBun", ...}, {"name": "DuckDns", ...}]}` + +### Create DNS Provider + +**POST** `/api/dns` + +Configure a new DNS provider. + +**Cloudflare:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My Cloudflare", "dnsProvider": "Cloudflare", "token": "your-api-token"}' \ + https://proxy-host.com/api/dns +``` + +**DigitalOcean:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My DO", "dnsProvider": "DigitalOcean", "token": "your-api-token"}' \ + https://proxy-host.com/api/dns +``` + +**PorkBun:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My PorkBun", "dnsProvider": "PorkBun", "apiKey": "pk_xxx", "secretApiKey": "sk_xxx"}' \ + https://proxy-host.com/api/dns +``` + +**DuckDNS (free):** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My DuckDNS", "dnsProvider": "DuckDns", "token": "your-duckdns-token", "subdomains": "myhost,myhost2"}' \ + https://proxy-host.com/api/dns +``` + +`subdomains` is a comma-separated list of the subdomains you've registered at +[duckdns.org](https://www.duckdns.org) (e.g. `myhost` for +`myhost.duckdns.org`), since DuckDNS has no API to list them for you. +DuckDNS only supports one A/AAAA record and one TXT record per domain (no +arbitrary sub-records) — enough for dynamic DNS and DNS-01 wildcard certs. + +**Responses:** +- `200` `{"message": "\"provider-id\" added.", ...}` +- `422` Validation error or invalid API credentials + +### Get DNS Provider + +**GET** `/api/dns/:id` + +Get a specific DNS provider configuration. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"item": "provider-id", "results": {...}}` +- `404` Provider not found + +### Update DNS Provider + +**PUT** `/api/dns/:id` + +Update DNS provider configuration. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"name": "Updated Name"}' \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"message": "\"provider-id\" updated.", ...}` +- `404` Provider not found + +### Delete DNS Provider + +**DELETE** `/api/dns/:id` + +Remove a DNS provider and all associated domains. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"message": "provider-id deleted", ...}` +- `404` Provider not found + +### List Domains + +**GET** `/api/dns/domain` + +List all domains from all configured providers. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/domain +``` + +**Query Parameters:** +- `detail` - Include full domain details (optional) + +**Responses:** +- `200` `{"results": ["example.com", "test.com"]}` + +### Get Domain + +**GET** `/api/dns/domain/:domain` + +Get details for a specific domain. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/domain/example.com +``` + +**Responses:** +- `200` `{"results": [{"domain": "example.com", "zoneId": "...", ...}]}` +- `404` Domain not found + +### Refresh Domains + +**POST** `/api/dns/domain/refresh/:providerId` + +Refresh the domain list from a DNS provider's API. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/dns/domain/refresh/provider-id +``` + +**Responses:** +- `200` `{"results": ...}` Updated domain list +- `404` Provider not found + +### Dynamic DNS + +A-records kept automatically pointed at this box's public (WAN) IP. All +`/api/dns/dynamic*` routes are viewer/manager scoped to the record's domain +(via [Permissions](#permissions)), not admin-only like the rest of `/api/dns`. + +#### Get Current Public IP + +**GET** `/api/dns/dynamic/ip` + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/dynamic/ip +``` + +**Responses:** +- `200` `{"ip": "203.0.113.5"}` + +#### List Dynamic Records + +**GET** `/api/dns/dynamic` + +Lists records the caller may view (their own/granted domains, or all for admins). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/dynamic +``` + +**Responses:** +- `200` `{"results": [{"id": "...", "domain": "example.com", "name": "home", "last_status": "ok", ...}, ...]}` + +#### Create Dynamic Record + +**POST** `/api/dns/dynamic` + +Requires `manager` rights on the target domain. Applies the record immediately +against the current public IP (best-effort — failures are recorded in +`last_status` and retried by the scheduler). + +**Parameters:** +- `domain` (required) +- `name` (required) - sub-label, or `@` for the apex + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"domain": "example.com", "name": "home"}' \ + https://proxy-host.com/api/dns/dynamic +``` + +**Responses:** +- `200` `{"message": "\"home.example.com\" added.", ...}` +- `403` Missing `manager` rights on the domain +- `422` Validation error + +#### Refresh Dynamic Record + +**POST** `/api/dns/dynamic/:id/refresh` + +Force an immediate refresh of one record against the current public IP. +Requires `manager` rights on the record's domain. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/dns/dynamic//refresh +``` + +**Responses:** +- `200` `{"message": "Refreshed \"home.example.com\".", "result": {...}}` +- `403` Missing `manager` rights on the domain + +#### Delete Dynamic Record + +**DELETE** `/api/dns/dynamic/:id` + +Stop managing a record. Requires `manager` rights on the record's domain. +Leaves the provider's A record in place at its last value. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/dns/dynamic/ +``` + +**Responses:** +- `200` `{"message": "home.example.com removed.", ...}` +- `403` Missing `manager` rights on the domain + +--- + +## Certificates + +Retrieve SSL certificate information. + +### Get Certificate + +**GET** `/api/cert/:host` + +Get the SSL certificate for a host. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/cert/example.com +``` + +**Responses:** +- `200` Certificate data including `cert_pem`, `fullchain_pem`, `privkey_pem`, expiry information +- `404` Certificate not found + +--- + +## Error Responses + +All endpoints may return the following error responses: + +- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - Authentication required or invalid +- `404` `{"name": "NotFound", "message": "..."}` - Resource not found +- `422` `{"name": "ObjectValidateError", "message": [...], "keys": [...]}` - Validation errors +- `500` Internal server error + +## Notes + +- All timestamps are in milliseconds since epoch +- Authenticated endpoints accept either the `auth-token` header (browser + session / OIDC login) or an `Authorization: Bearer ` API token +- Host names support wildcards: `*` (single level) and `**` (multi-level) +- DNS providers are validated on creation - invalid API credentials will be rejected +- Wildcard certificates are automatically renewed 30 days before expiration diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..29f7a04 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,291 @@ +--- +layout: default +title: Architecture +description: How the proxy's OIDC client, LDAP client, and OpenResty routing fit together. +--- + +# Architecture + +[← Back to Home](index.html) + +> Looking for a plainer explanation of hosts, HTTPS, or the local +> permission model instead of internals? See +> [Hosts & HTTPS](concepts-hosts.html) and +> [Users, Groups & Permissions](concepts-access.html). + +## System Overview + +The proxy system consists of three main components working together to provide high-performance reverse proxying with automated SSL management. + +``` +┌──────────────────────────────────────────────────────────────┐ +│ Internet │ +└─────────────────────────┬────────────────────────────────────┘ + │ HTTPS/HTTP + ▼ +┌──────────────────────────────────────────────────────────────┐ +│ OpenResty/Nginx │ +│ ┌────────────────┐ ┌──────────────┐ ┌─────────────────┐ │ +│ │ SSL Termination│ │ Host Routing │ │ Request Proxying│ │ +│ │ (lua-resty- │ │ (targetinfo. │ │ │ │ +│ │ auto-ssl) │ │ lua) │ │ │ │ +│ └────────────────┘ └──────┬───────┘ └─────────────────┘ │ +└────────────┬──────────────────┼───────────────────────────┬──┘ + │ │ │ + Let's Encrypt 1. Check Redis FIRST Backend + HTTP-01 2. Unix Socket (fallback) Services + │ │ │ + ▼ ▼ ▼ +┌──────────────────────┐ ┌──────────────────────────────────┐ +│ Redis │ │ Node.js Application │ +│ (Primary Cache) │ │ ┌──────────────┐ ┌─────────┐ │ +│ - Host configs ◄────┼──┼──┤ Services │ │ Routes │ │ +│ - User accounts │ │ │ - host_lookup│ │ - /api/*│ │ +│ - SSL certs │ │ │ - scheduler │ │ │ │ +│ - Auth tokens │ │ └──────────────┘ └─────────┘ │ +└──────────────────────┘ └─────────┬────────────────────────┘ + │ + ▼ + ┌──────────────────────┐ + │ DNS Providers │ + │ - Cloudflare │ + │ - DigitalOcean │ + │ - PorkBun │ + │ - DuckDNS (free) │ + │ (DNS-01 challenges) │ + └──────────────────────┘ +``` + +## Component Details + +### OpenResty/Nginx (Frontend) + +**Responsibilities:** +- Accept incoming HTTP/HTTPS requests +- SSL termination using lua-resty-auto-ssl +- Host-based routing decisions (Redis-first lookup) +- Proxy requests to backend services + +**Key Features:** +- HTTP-01 ACME challenge handling for automatic SSL +- Redis-first host lookup with Node.js fallback via Unix socket +- High-performance event-driven architecture +- Support for WebSocket connections +- Continues serving cached hosts even if Node.js is down + +**Configuration Files:** +- `/etc/openresty/nginx.conf` - Main configuration +- `/etc/openresty/autossl.conf` - Let's Encrypt integration +- `/etc/openresty/sites-enabled/000-proxy` - Proxy configuration +- `/usr/local/openresty/lualib/targetinfo.lua` - Host lookup module + +### Node.js Application (Backend) + +**Responsibilities:** +- API for host/user/DNS management +- Wildcard SSL certificate orchestration +- Host lookup tree maintenance +- User authentication and authorization + +**Directory Structure:** +``` +nodejs/ +├── bin/www # Application entry point +├── conf/ # Configuration (base.js, environment overlays, secrets.js) +├── controller/ # App-level wiring (pubsub, startup) +├── migrations/ # One-off Redis data migration scripts +├── models/ # Data models +│ ├── host.js # Host configuration and lookup +│ ├── auth.js # Authentication logic +│ ├── user.js # User management +│ └── dns_provider/ # DNS provider implementations +├── routes/ # API endpoints +│ ├── host.js # Host CRUD operations +│ ├── dns.js # DNS provider management +│ ├── user.js # User management +│ ├── auth.js # Authentication (login + OIDC) +│ ├── permission.js # RBAC permission management +│ ├── group.js # Local group management +│ └── api_token.js # Self-service API (PAT) tokens +├── services/ # Background services +│ ├── host_lookup.js # Unix socket server +│ └── host_scheduler.js # Cert renewal scheduler +├── middleware/ # Express middleware +│ └── auth.js # Authentication middleware +└── utils/ # Utility modules + └── unix_socket_json.js # Unix socket server +``` + +### Redis (Data Store) + +**ORM:** [model-redis](https://www.npmjs.com/package/model-redis) - A lightweight Redis ORM for Node.js with schema validation, relationships, and automatic key management. + +**Stored Data:** +- Host configurations (domain, IP, port, SSL settings) +- User accounts and hashed passwords +- Authentication tokens +- SSL certificates (for wildcard domains) +- DNS provider credentials +- Domain-to-provider mappings + +**Key Prefixes:** +``` +proxy_Host_ # Host configuration +proxy_User_ # User account +proxy_AuthToken_ # Auth tokens +proxy_DnsProvider_ # DNS provider +proxy_Domain_ # Domain info +:latest # SSL certificate cache +``` + +## Request Flow + +### Standard HTTP/HTTPS Request + +1. **Client** sends HTTPS request to `app.example.com` +2. **OpenResty** receives request, terminates SSL +3. **Lua script** (`targetinfo.lua`) queries **Redis first** for host config +4. If **found in Redis**, jump to step 7 (Node.js not involved) +5. If **not in Redis**, Lua queries Node.js via Unix socket as fallback +6. **Node.js** performs host lookup (supports wildcards), caches result in Redis +7. **OpenResty** proxies request to backend service using target IP and port +8. **Response** proxied back to client + +**Resilience**: If Node.js goes down, all hosts already cached in Redis continue to work. Only new/uncached hosts will fail until Node.js recovers. + +### Wildcard SSL Certificate Request + +1. **User** creates wildcard host (`*.example.com`) via API +2. **Node.js** validates domain has DNS provider configured +3. **Let's Encrypt** DNS-01 challenge initiated +4. **DNS provider** API creates TXT record (`_acme-challenge.example.com`) +5. **Let's Encrypt** validates TXT record +6. **Certificate** generated and stored in Redis +7. **DNS provider** cleans up TXT record +8. **Background scheduler** monitors expiration, renews 30 days before expiry + +## Host Lookup Algorithm + +The lookup tree enables sophisticated domain matching: + +``` +Input: "api.v1.example.com" + +Tree Structure: +{ + "com": { + "example": { + "*": { // Matches api.example.com + "#record": {...} + }, + "v1": { + "api": { // Matches api.v1.example.com (exact) + "#record": {...} + } + } + } + } +} + +Priority: Exact > Single wildcard (*) > Double wildcard (**) +``` + +**Wildcard Types:** +- `example.com` - Exact match only +- `*.example.com` - Matches `sub.example.com` (single level) +- `**.example.com` - Matches any depth (`sub.deep.example.com`) +- `api.*.example.com` - Matches `api.v1.example.com`, `api.v2.example.com` + +## Security Architecture + +### Authentication Flow + +1. User sends credentials to `/api/auth/login` +2. Credentials validated against stored hash (bcrypt) +3. Token generated and stored in Redis with TTL +4. Token returned to client +5. Subsequent requests include token in `auth-token` header +6. Middleware validates token before processing request + +### SSL Certificate Security + +- **Private keys** stored only in Redis (memory/disk based on config) +- **Fallback certificates** used when SNI unavailable +- **Let's Encrypt** rate limiting respected +- **DNS provider credentials** marked as `isPrivate` (not returned in API) + +### Unix Socket Communication + +- Socket file: `/var/run/proxy_lookup.socket` +- Permissions: `777` (container-safe, single-use deployment) +- Protocol: JSON over Unix stream socket +- Buffer handling: Accumulates partial messages until complete JSON + +## Performance Optimizations + +### Caching Strategy + +The system uses a multi-tier caching approach: + +1. **Redis (L1 Cache)** - OpenResty checks Redis FIRST for every request + - Primary host configuration storage + - Survives Node.js restarts/failures + - Shared across all OpenResty workers + +2. **Node.js Lookup Tree (L2 Cache)** - In-memory host lookup with wildcard matching + - Only queried when Redis has no entry + - Rebuilt automatically when hosts change + - Supports complex wildcard resolution + +3. **Wildcard Parent Caching** - Resolved wildcard matches stored back to Redis + - Subsequent requests to `api.example.com` hit Redis directly + - No repeated wildcard resolution needed + +### Unix Socket vs HTTP API + +Unix socket chosen over HTTP for host lookups: +- **Lower latency** - No TCP overhead +- **Higher throughput** - No HTTP parsing +- **Simpler** - Direct JSON communication +- **Secure** - Filesystem permissions, no network exposure + +## Scalability Considerations + +### Current Architecture + +- **Single instance** - OpenResty + Node.js + Redis on one server +- **Vertical scaling** - Add CPU/RAM as needed +- **Limitations** - Unix socket ties OpenResty to Node.js on same host + +### Future Scaling Options + +- **Redis cluster** - Distribute data storage +- **Multiple OpenResty instances** - Load balance incoming requests +- **Stateless Node.js** - Run multiple API instances +- **Replace Unix socket** - Use TCP/HTTP for cross-host communication +- **Separate cert management** - Dedicated service for wildcard SSL + +## Monitoring and Observability + +### Logs + +- **OpenResty**: `/var/log/nginx/access.log`, `/var/log/nginx/error.log` +- **Node.js**: `journalctl -u proxy.service` +- **Redis**: `redis-cli MONITOR` + +### Health Checks + +- Node.js API: `curl http://localhost:3000/api/host` +- Redis: `redis-cli PING` +- OpenResty: `systemctl status openresty` +- Unix socket: `ls -la /var/run/proxy_lookup.socket` + +### Metrics to Monitor + +- Request rate and response times +- SSL certificate expiration dates +- Redis memory usage +- Host lookup cache hit rate +- Background service execution times + +[← Back to Home](index.html) diff --git a/docs/assets/css/style.css b/docs/assets/css/style.css new file mode 100644 index 0000000..e24a5de --- /dev/null +++ b/docs/assets/css/style.css @@ -0,0 +1,116 @@ +/* theta42 docs site — shares the in-app dark navbar/footer + card look + (Bootstrap 5 + Font Awesome, same as the running apps) rather than a + generic Jekyll theme. */ + +body { + background-color: #f4f5f6; +} + +.navbar-brand img { + filter: drop-shadow(0 0 2px rgba(0, 0, 0, .4)); +} + +.navbar-nav .nav-link.active { + color: #fff; + font-weight: 600; +} + +/* Markdown content typography, scoped to the card body so it doesn't leak + into the nav/footer. */ +.site-content h1:first-child { + margin-top: 0; +} + +.site-content h1, +.site-content h2, +.site-content h3 { + font-weight: 700; +} + +.site-content h2 { + margin-top: 2.5rem; + padding-bottom: .4rem; + border-bottom: 1px solid #e9ecef; +} + +.site-content h3 { + margin-top: 1.75rem; +} + +.site-content a { + color: #a3671f; + text-decoration-color: rgba(163, 103, 31, .35); +} + +.site-content a:hover { + color: #8a5a16; +} + +.site-content pre { + background-color: #212529; + color: #f8f9fa; + padding: 1rem 1.25rem; + border-radius: .375rem; + overflow-x: auto; +} + +.site-content code { + color: #a3671f; + background-color: #f4f0e8; + padding: .15em .4em; + border-radius: .25rem; + font-size: .875em; +} + +.site-content pre code { + color: inherit; + background: none; + padding: 0; +} + +.site-content table { + display: block; + overflow-x: auto; + width: 100%; + border-collapse: collapse; + margin: 1.25rem 0; +} + +.site-content table th, +.site-content table td { + border: 1px solid #dee2e6; + padding: .5rem .75rem; + text-align: left; +} + +.site-content table th { + background-color: #f8f9fa; +} + +.site-content blockquote { + border-left: 4px solid #C59341; + padding: .5rem 1rem; + margin: 1.25rem 0; + background-color: #f8f6f1; + color: #495057; +} + +.site-content img { + max-width: 100%; + height: auto; +} + +/* Screenshot grids in the markdown use width="49%" inline attrs for a + two-up desktop layout -- stack them on narrow screens instead of + squeezing to illegibility. */ +@media (max-width: 576px) { + .site-content img[width] { + width: 100% !important; + margin-bottom: .75rem; + } +} + +.site-content hr { + margin: 2rem 0; + border-top: 1px solid #e9ecef; +} diff --git a/docs/assets/img/favicon.svg b/docs/assets/img/favicon.svg new file mode 100644 index 0000000..35e1881 --- /dev/null +++ b/docs/assets/img/favicon.svg @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/docs/assets/img/theta42.svg b/docs/assets/img/theta42.svg new file mode 100644 index 0000000..e598305 --- /dev/null +++ b/docs/assets/img/theta42.svg @@ -0,0 +1,51 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 42 + + diff --git a/docs/concepts-access.md b/docs/concepts-access.md new file mode 100644 index 0000000..76c9aa8 --- /dev/null +++ b/docs/concepts-access.md @@ -0,0 +1,75 @@ +--- +layout: default +title: Users, Groups & Permissions +description: A plain-language guide to local admin accounts, groups, and the domain-scoped permission model in theta42/proxy. +--- + +# Users, Groups & Permissions + +This page explains, in plain language, who can manage what in this app. For +the deeper system-design detail, see [Architecture](architecture.html). + +## Two different ways to log in + +Most people who use apps you've proxied through this app never see this +app's own login at all — they use whatever authentication you set up on +the *individual host* (basic auth, or single sign-on through your SSO +Manager). This page is about a different, smaller group: the people who +manage the proxy itself — adding hosts, registering DNS providers, and so +on. + +There are two ways someone gets into the proxy's own management UI: + +- **A local account**, created on the **Users** page — a username and + password specific to this app. +- **Single sign-on**, if you've connected this proxy to an SSO Manager (or + another OIDC provider) — the same login your other connected apps use. + +Either way, once logged in, what they're actually *allowed to do* here is +controlled by permissions, described below. + +## Groups + +A **group** here is just a named list of local usernames, used to grant +the same permission to several people at once instead of one at a time. +If you're using SSO instead of local accounts, group membership normally +comes from your identity provider instead — local groups exist mainly for +the local-account case. + +## Permissions: scope + role + +Each **permission** entry grants one subject (a user or a group) one +**role**, at one **scope** — the two are independent choices: + +**Scope** — *where* the role applies: + +- **Domain** — only hosts under one specific domain (e.g. someone can + manage everything under `example.com`, but can't see or touch a + completely different domain you also proxy). +- **Global** — everywhere, across every domain this proxy manages. + +**Role** — *what* they can do within that scope: + +- **Viewer** — read-only. Can see hosts and their settings, but not + change anything. +- **Manager** — full control over hosts (create, edit, delete) within + that scope. +- **Admin** — same host control as Manager, **plus**, but *only when + granted at Global scope*, the ability to manage other people's + permissions, DNS providers, and local user accounts. An Admin role + granted at Domain scope instead of Global behaves exactly like Manager + for that one domain — it does not unlock those extra admin-only pages. + +In practice: give someone **Manager** on just the domain(s) they're +responsible for to delegate day-to-day host management without handing +them the keys to everything. Reserve **Global Admin** for people who +should be able to change anything, anywhere, including who else has +access. + +## Want more detail? + +This page doesn't cover the exact permission-checking implementation or +how SSO group membership maps into this system internally — for that, see +[Architecture](architecture.html). + +[← Back to Home](index.html) diff --git a/docs/concepts-api-tokens.md b/docs/concepts-api-tokens.md new file mode 100644 index 0000000..5efe7d4 --- /dev/null +++ b/docs/concepts-api-tokens.md @@ -0,0 +1,60 @@ +--- +layout: default +title: API Tokens +description: A plain-language guide to personal access tokens in theta42/proxy. +--- + +# API Tokens + +This page explains what an API token is and when you'd want one. For the +full list of API endpoints a token can call, see the +[API reference](api.html). + +## What's an API token, in plain terms? + +Normally, you interact with this app by logging in through a web browser. +An **API token** (also called a personal access token, or PAT) is an +alternative way in — a long, random string that a script, a scheduled job, +or another program can use instead of a username and password, to act on +your behalf without a human typing a login in each time. + +If you've ever set up a script to talk to GitHub, GitLab, or a similar +service using a "token" instead of your real password, this is the same +idea. + +## When would you actually need one? + +Most people never need to create one of these — you'll only want a token +if you're automating something, for example: + +- A script that registers or updates hosts automatically (say, spinning up + a new service and wanting the proxy entry created for it without a + manual step). +- A monitoring or backup job that checks this app's health via its API. +- A configuration-management tool that keeps your host list in sync with + something else. + +If you're not doing any of that, you don't need an API token — just log in +normally through the web UI. + +## How it works + +Create a token from your Profile page, give it a name so you remember what +it's for later, and optionally an expiry. You'll be shown the token's +value **exactly once** — copy it somewhere safe immediately, because it +can't be viewed again afterward (only revoked or rotated). Whatever script +or tool you're using it with sends it along with each request, the same +way a browser sends your login session. + +A token acts **as you**, with **your** [permissions](concepts-access.html) +— if you're only a Manager on one domain, a token you create can't touch +any other domain either. If you ever suspect a token has leaked (ended up +somewhere it shouldn't have, like a public script or log file), revoke it +immediately from your Profile page; it stops working right away. + +## Want more detail? + +This page doesn't attempt to list every API endpoint or show request/ +response examples — for that, see the full [API reference](api.html). + +[← Back to Home](index.html) diff --git a/docs/concepts-dns.md b/docs/concepts-dns.md new file mode 100644 index 0000000..23e0d62 --- /dev/null +++ b/docs/concepts-dns.md @@ -0,0 +1,49 @@ +--- +layout: default +title: DNS Providers +description: A plain-language guide to why theta42/proxy needs a DNS provider, and only for wildcard certificates. +--- + +# DNS Providers + +This page explains, in plain language, what a "DNS provider" is for in this +app and when you actually need one. For setup steps, see +[Installation](installation.html). + +## Do you need this at all? + +**Only if you want a [wildcard host](concepts-hosts.html)** (something like +`*.example.com` covering every subdomain with one certificate). A normal, +single-name host doesn't need a DNS provider configured at all — skip this +page entirely if that's all you're setting up. + +## Why a wildcard cert needs this extra step + +To prove you actually own `example.com` before issuing a certificate that +covers *every* possible subdomain of it, Let's Encrypt needs to see a +specific, temporary DNS record appear on that domain — something only the +real owner of the domain could add. A normal single-host certificate +doesn't need this because it can prove ownership a simpler way (by +responding to a web request instead). + +So: to get a wildcard certificate, this app needs to be able to add (and +later remove) that one temporary DNS record on your domain automatically, +which means it needs your domain registrar or DNS host's API credentials — +that's what registering a **DNS provider** here does. + +## What you're actually giving it access to + +A DNS provider entry only needs enough access to add/remove TXT records — +it's not given your registrar account's full login, and it can't do +anything to your domain besides that one narrow task (and, for some +providers, keeping a dynamic A record updated if you use that feature +separately). Check your specific provider's page in the +[Installation guide](installation.html) for exactly what kind of +credential to generate and how narrowly you can scope it. + +## Want more detail? + +For exact setup steps per provider (Cloudflare, DigitalOcean, Porkbun, +DuckDNS, etc.), see [Installation](installation.html). + +[← Back to Home](index.html) diff --git a/docs/concepts-hosts.md b/docs/concepts-hosts.md new file mode 100644 index 0000000..504d2e1 --- /dev/null +++ b/docs/concepts-hosts.md @@ -0,0 +1,81 @@ +--- +layout: default +title: Hosts & HTTPS +description: A plain-language guide to hosts, HTTPS certificates, and wildcards in theta42/proxy. +--- + +# Hosts & HTTPS + +This page explains, in plain language, what a "host" is and how this app +gets you working HTTPS without you having to think about certificates. For +the deeper system-design detail, see [Architecture](architecture.html); for +step-by-step setup, see [Installation](installation.html). + +## What's a "host"? + +A **host** is one entry telling the proxy: "when someone requests *this* +public address, send them to *that* server." For example: requests for +`photos.example.com` get sent to the little box in your closet running your +photo app on port 8080. Each app or service you want to reach from outside +your network — a home automation dashboard, a media server, this proxy's +own management UI — gets its own host entry. + +Two settings on a host are easy to mix up: + +- **Incoming host name** — the public address people type in their + browser (`photos.example.com`). +- **Target IP/port** — where the proxy actually sends the request behind + the scenes (`10.0.0.5:8080`, or a hostname like `photo-server`). + +Everything else on the host form (traffic limits, access rules, +authentication) is optional — a bare host with just those two fields +already works. + +## HTTPS certificates: mostly automatic + +Every public website needs an HTTPS certificate so browsers show the lock +icon instead of a scary warning. This app gets one for you automatically +from [Let's Encrypt](https://letsencrypt.org) the first time a host is +actually requested — you don't manually request, install, or renew +anything for a normal host. This happens behind the scenes using a method +called **HTTP-01**, and it's the default for every new host. + +## Wildcards: one certificate for a whole family of hosts + +Sometimes you want *every* subdomain under one name to work — `app1.`, +`app2.`, `anything.example.com` — without registering each one by hand and +waiting for its own certificate. That's what a **wildcard** host does: a +single host entry named `*.example.com` gets one certificate that covers +the whole family at once. Setting one up needs one extra piece of +information the automatic method above doesn't need — see +[DNS Providers](concepts-dns.html) for why. + +Once a wildcard exists, you have two ways to actually use it: + +- **Register nothing else, and turn on "Match any subdomain"** on the + wildcard host itself — *any* subdomain that doesn't already have its own + entry gets automatically routed to the wildcard's target the first time + it's requested. Convenient, but it means literal typos and random scan + traffic get routed too, not just the subdomains you meant to use. +- **Register each subdomain as its own host, as a "Parent Wildcard" + child** — more setup, but each subdomain can point at a different + target/server while still reusing the one wildcard certificate instead + of getting its own. This is the recommended default and is what + "Match only subdomains defined here" (the host form's default) does. + +You'll see the **"Parent Wildcard"** option light up automatically on the +host form whenever the name you're entering already has a matching +wildcard available to reuse — including the wildcard's own bare base +domain (e.g. `example.com` itself, not just `something.example.com`). + +## Load Balancing + +If you have multiple servers running the same application, you can load balance traffic across them. When editing a host, you can specify **Additional Targets** (one `IP:port` per line). The proxy will automatically distribute incoming requests across your primary target and all additional targets using a round-robin strategy, providing simple high availability and load distribution without extra configuration. + +## Want more detail? + +This page skips the system-internals (Redis, OpenResty, the lookup service) +and the exact install steps. For those, see +[Architecture](architecture.html) and [Installation](installation.html). + +[← Back to Home](index.html) diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..08aa80f --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,344 @@ +--- +layout: default +title: Contributing +description: How to contribute to the proxy — dev setup, tests, and code conventions. +--- + +# Contributing Guide + +[← Back to Home](index.html) + +Thank you for considering contributing to the Proxy project! This guide will help you get started. + +## Development Setup + +### Prerequisites + +- Node.js 18+ (18.x, 20.x, or 22.x recommended) +- Redis server +- Git + +### Local Development + +1. **Clone the repository** + ```bash + git clone https://github.com/theta42/proxy.git + cd proxy/nodejs + ``` + +2. **Install dependencies** + ```bash + npm install + ``` + +3. **Start Redis** (if not already running) + ```bash + redis-server + ``` + +4. **Run in development mode** + ```bash + npm run dev + ``` + + This starts the Node.js API with nodemon for auto-reload on file changes. + +5. **Access the API** + - API: `http://localhost:3000/api` + - Web UI: `http://localhost:3000` + +## Testing + +The project uses Node.js built-in test runner (requires Node 18+). + +### Running Tests + +```bash +# Run all tests +npm test + +# Run only unit tests +npm run test:unit + +# Run only integration tests +npm run test:integration + +# Watch mode for development +npm run test:watch +``` + +### Test Structure + +``` +test/ +├── unit/ # Unit tests for isolated components +│ ├── basicauth.test.js +│ ├── callback_queue.test.js +│ ├── dynamic_record.test.js +│ ├── host_features.test.js +│ ├── host_lookup.test.js +│ ├── hostname_validate.test.js +│ ├── host_sso.test.js +│ ├── oidc.test.js +│ ├── password_policy.test.js +│ ├── roles.test.js +│ ├── safe_redirect.test.js +│ ├── unix_socket.test.js +│ └── wildcard_matchany.test.js +├── integration/ # Integration tests +│ └── dns_provider.test.js +└── helpers/ # Test utilities + └── dns_provider_contract.js +``` + +### Writing Tests + +We test **custom logic**, not third-party libraries: + +**DO test:** +- Host lookup algorithm +- Socket buffering logic +- DNS provider contracts +- Custom utility functions + +**DON'T test:** +- Express.js routing +- Redis ORM +- External DNS APIs (use mocks instead) + +### Adding DNS Provider Tests + +When adding a new DNS provider, you **must** add contract tests: + +```javascript +describe('NewProvider Provider', () => { + const NewProvider = require('../../models/dns_provider/newprovider'); + + test('should meet DNS provider contract', () => { + const mockCredentials = {api_key: 'mock-key'}; + const instance = validateDnsProviderContract(NewProvider, mockCredentials); + assert.ok(instance); + }); + + test('should have valid method signatures', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateMethodSignatures(instance); + }); + + test('should validate key mapping', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateKeyMapping(instance); + }); + + test('should validate type checking', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateTypeChecking(instance); + }); +}); +``` + +See `test/integration/dns_provider.test.js` for examples. + +## Code Style + +### General Guidelines + +- Use strict mode: `'use strict';` +- Use tabs for indentation +- Clear, descriptive variable names +- Comment complex logic +- No trailing whitespace + +### File Organization + +```javascript +'use strict'; + +// 1. Node.js built-ins +const fs = require('fs'); +const path = require('path'); + +// 2. Third-party modules +const express = require('express'); +const redis = require('redis'); + +// 3. Local modules +const {Host} = require('./models'); +const middleware = require('./middleware/auth'); + +// 4. Code... +``` + +### Naming Conventions + +- Classes: `PascalCase` +- Functions: `camelCase` +- Constants: `UPPER_SNAKE_CASE` +- Private methods: `__privateMethod` (double underscore prefix) + +## Project Structure + +Understanding the codebase: + +``` +nodejs/ +├── conf/ # Configuration (base.js, environment overlays, secrets.js) +├── controller/ # App-level wiring (pubsub, startup) +├── migrations/ # One-off Redis data migration scripts +├── models/ # Data models (Host, User, DNS providers) +├── routes/ # API route handlers +├── services/ # Background services (lookup, scheduler) +├── middleware/ # Express middleware +├── utils/ # Utility functions +├── public/ # Static web assets +├── views/ # EJS templates +└── test/ # Test suite +``` + +## Pull Request Process + +### Before Submitting + +1. **Run tests** - Ensure all tests pass + ```bash + npm test + ``` + +2. **Test locally** - Verify your changes work + ```bash + npm run dev + ``` + +3. **Update documentation** - Keep docs in sync with code changes + +4. **Commit messages** - Use clear, descriptive messages + ``` + Add DNS provider for Route53 + + - Implement Route53 DNS API client + - Add contract tests for Route53 + - Update documentation with Route53 setup + ``` + +### Submitting a PR + +1. **Fork the repository** + +2. **Create a feature branch** + ```bash + git checkout -b feature/my-new-feature + ``` + +3. **Make your changes** + +4. **Commit your changes** + ```bash + git add . + git commit -m "Description of changes" + ``` + +5. **Push to your fork** + ```bash + git push origin feature/my-new-feature + ``` + +6. **Open a Pull Request** on GitHub + +### PR Requirements + +- All tests must pass (CI/CD runs automatically) +- Tests run on Node.js 18.x, 20.x, and 22.x +- No merge conflicts with `master` +- Code follows project conventions +- New features include tests +- Documentation updated if needed + +### CI/CD Process + +When you open a PR: +1. GitHub Actions automatically runs tests +2. Tests execute on multiple Node.js versions +3. PR cannot be merged until all checks pass +4. Review from maintainers +5. Merge to master + +## Data Models + +The project uses [model-redis](https://www.npmjs.com/package/model-redis) as the ORM for Redis data storage. All models extend the `Table` class and use a declarative schema via `_keyMap`. + +**Example Model:** +```javascript +const Table = require('../utils/redis_model'); + +class Host extends Table { + static _key = 'host'; // Primary key field + static _keyMap = { + 'host': {isRequired: true, type: 'string', min: 3, max: 500}, + 'ip': {isRequired: true, type: 'string', min: 3, max: 500}, + 'targetPort': {isRequired: true, type: 'number', min: 0, max: 65535}, + 'forcessl': {default: true, type: 'boolean'}, + 'created_on': {default: () => Date.now(), type: 'number'} + }; +} +``` + +**Learn more:** [model-redis documentation](https://www.npmjs.com/package/model-redis) + +## Adding Features + +### Adding a DNS Provider + +1. **Create provider file** in `models/dns_provider/yourprovider.js` + +2. **Extend DnsApi base class** + ```javascript + const {DnsApi} = require('./common'); + + class YourProvider extends DnsApi { + static _keyMap = { + api_key: {isRequired: true, type: 'string', isPrivate: true} + }; + + // Implement required methods + async listDomains() { } + async getRecords(domain, options) { } + async createRecord(domain, options) { } + async deleteRecords(domain, options) { } + } + ``` + +3. **Add to provider list** in `models/dns_provider.js` + +4. **Add contract tests** in `test/integration/dns_provider.test.js` + +5. **Test your provider** + ```bash + npm run test:integration + ``` + +### Adding API Endpoints + +1. **Add route** in appropriate file (`routes/`) +2. **Update API documentation** (`nodejs/api.md` and `docs/api.md` — keep them in sync) +3. **Test the endpoint** manually and add integration tests if needed + +## Getting Help + +- **Questions?** Open a [GitHub Discussion](https://github.com/theta42/proxy/discussions) +- **Bug reports** Use [GitHub Issues](https://github.com/theta42/proxy/issues) +- **Security issues** Email maintainers directly (see package.json) + +## Code of Conduct + +- Be respectful and inclusive +- Focus on constructive feedback +- Help others learn and grow +- Follow the project's technical direction + +## License + +By contributing, you agree that your contributions will be licensed under the MIT License. + +--- + +[← Back to Home](index.html) | [View on GitHub](https://github.com/theta42/proxy) diff --git a/docs/docker.md b/docs/docker.md new file mode 100644 index 0000000..83b6a3c --- /dev/null +++ b/docs/docker.md @@ -0,0 +1,141 @@ +--- +layout: default +title: Docker +description: Running the proxy's all-in-one Docker image — OpenResty, the management app, and Redis in one container. +--- + +# Docker Deployment + +[← Back to Home](index.html) + +The proxy ships as a single all-in-one Docker image bundling **OpenResty + the +Node management app + Redis** in one container, mirroring the bare-metal +[`ops/install.sh`](https://github.com/theta42/proxy/blob/master/ops/install.sh) +layout. This is the easiest way to run the proxy standalone, or as part of the +unified [theta-env](https://github.com/theta42/theta-env) stack. + +## Quick start (standalone) + +```bash +git clone https://github.com/theta42/proxy.git +cd proxy +mkdir -p config && chmod 700 config +cp secrets.js.example config/proxy-secrets.js # set OIDC/LDAP wiring +$EDITOR config/proxy-secrets.js +docker compose up -d --build +``` + +- Proxy (public, auto-SSL): `https:///` +- Management UI / API: `http://127.0.0.1:3000/` (bound to localhost) +- Health: `http://127.0.0.1:3000/health` → `{"status":"ok"}` + +## How configuration works + +The app loads config via [`@simpleworkjs/conf`](https://www.npmjs.com/package/@simpleworkjs/conf), +which deep-merges, in order: + +1. `conf/base.js` (committed defaults) +2. `conf/.js` (optional) +3. `conf/secrets.js` (gitignored) +4. **`app_*` environment variables** — the highest-precedence layer + +The bundled `docker-compose.yml` mounts `./config/proxy-secrets.js` at `/config`, +and `docker-entrypoint.sh` sets `CONF_SECRETS=/config/proxy-secrets.js` so the +app reads the OIDC + LDAP + auth wiring from the file. **No `app_*` env is +passed** — `app_*` env beats `secrets.js`, so the file is authoritative only if +the matching `app_*` env is absent. See `secrets.js.example` for the shape. + +Any env var starting with `app_` overrides the merged config; the rest of the +name splits on **double-underscore** (`__`) into a nested path. Values are +`JSON.parse`-coerced when possible, kept as strings otherwise. `app_*` env is +still supported for advanced/standalone use — add the vars to the compose +`environment:` block yourself (the bundled compose no longer sets them). + +> **Requires `@simpleworkjs/conf` >= 1.1.0.** The `app_*` env layer is not +> honored on 1.0.0. The lock is already on `^1.1.0`. + +### Key `app_*` variables + +| Env var | Sets | +|---------|------| +| `app_oidc__issuer` | `conf.oidc.issuer` (browser-facing SSO URL) | +| `app_oidc__authorizationEndpoint` | `conf.oidc.authorizationEndpoint` | +| `app_oidc__tokenEndpoint` | `conf.oidc.tokenEndpoint` (server-to-server; can be internal) | +| `app_oidc__userinfoEndpoint` | `conf.oidc.userinfoEndpoint` (server-to-server) | +| `app_oidc__endSessionEndpoint` | `conf.oidc.endSessionEndpoint` | +| `app_oidc__clientId` / `app_oidc__clientSecret` | OIDC client creds | +| `app_oidc__redirectUri` | `conf.oidc.redirectUri` (must match the SSO client) | +| `app_oidc__enabled` | `conf.oidc.enabled` (boolean) | +| `app_ldap__url` | `conf.ldap.url` (`ldaps://…:636` or `ldap://…:389`) | +| `app_ldap__bindDN` / `app_ldap__bindPassword` | LDAP service account | +| `app_ldap__searchBase` / `app_ldap__userFilter` | user search | +| `app_ldap__tlsOptions__rejectUnauthorized` | `false` for self-signed LDAPS | +| `app_ldap__tlsOptions__ca` | path to a CA cert for strict trust | +| `app_auth__adminUsers` | local anti-lockout admin (uid) | +| `app_auth__adminGroups` | SSO/LDAP groups that are global admin (JSON array) | +| `app_redis__prefix` | `conf.redis.prefix` (default `proxy_`) | + +See [`DEPLOYMENT.md`](https://github.com/theta42/proxy/blob/master/DEPLOYMENT.md) +for the complete reference. + +## OpenResty runtime env + +| Variable | Default | Description | +|----------|---------|-------------| +| `RESOLVER` | `127.0.0.11` | DNS for upstream names in Host records (Docker's embedded DNS) | +| `REAL_IP_FROM` | _empty_ | Trusted CIDR for `X-Real-IP`. Empty = the proxy is the front (removes the real_ip block). Set to an upstream proxy's CIDR if one sits in front. | + +## Auto-SSL / Let's Encrypt + +`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB +persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs +survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is +required for HTTP-01 challenges (mapped in the compose). Back up + restore Redis +to back up + restore cert state (see *Backups and restore* in `DEPLOYMENT.md`). + +## Fronting an SSO Manager + +The proxy is a natural front for +[`theta42/sso-manager-node`](https://github.com/theta42/sso-manager-node): it +terminates TLS for the SSO's UI and protects it with OIDC login, while also +binding to the SSO's LDAP directly for user lookups. To run both together: + +1. **One Docker network** so the proxy reaches the SSO internally at + `http://sso-manager:3001` (token/userinfo) and `ldaps://sso-manager:636`. +2. **Set the SSO's `app_oauth__issuer`** to the browser-facing HTTPS URL the + proxy serves the SSO at (e.g. `https://sso.example.com`). +3. **Register the proxy as an OIDC client** in the SSO, with `redirectUri` + matching `https://proxy.example.com/api/auth/oidc/callback`. +4. **LDAP**: point `app_ldap__url` at `ldaps://sso-manager:636`, create a + dedicated service account (`cn=ldapclient,ou=people,…`), and for the SSO's + self-signed LDAPS cert set `app_ldap__tlsOptions__rejectUnauthorized=false` + (or mount the cert and use `app_ldap__tlsOptions__ca=`). + +The [`theta42/theta-env`](https://github.com/theta42/theta-env) unified repo +automates all four steps with `./setup.sh` — see +[theta-env docs](https://theta42.github.io/theta-env/). + +## API tokens (personal access tokens) + +Any logged-in user can mint a long-lived bearer token to call the management API +from scripts/CI without an OIDC browser session. Self-service; authenticates as +the creator (groups snapshotted at mint; authz layer unchanged). + +Create one under **API Tokens** in the UI (shown once), then: + +```bash +curl -H "Authorization: Bearer prx__" https://proxy.example.com/api/host +``` + +Rotate/revoke from the same page (immediate effect). Optional expiry at +creation. The token carries the creator's rights (admin → full mgmt API; +domain manager → those domains; `requireAdmin` routes 403). To tighten after +group changes, revoke + re-mint. Tokens persist in Redis (AOF) and survive +rebuilds. + +## Bare metal + +Prefer a systemd install? See the [Installation Guide](installation.html) for +the `ops/install.sh` automated installer on Debian/Ubuntu. + +[← Back to Home](index.html) \ No newline at end of file diff --git a/docs/images/host-auth-basic.png b/docs/images/host-auth-basic.png new file mode 100644 index 0000000..f4d8026 Binary files /dev/null and b/docs/images/host-auth-basic.png differ diff --git a/docs/images/host-auth-sso.png b/docs/images/host-auth-sso.png new file mode 100644 index 0000000..cccb8ec Binary files /dev/null and b/docs/images/host-auth-sso.png differ diff --git a/docs/images/hosts.png b/docs/images/hosts.png new file mode 100644 index 0000000..cb6cf08 Binary files /dev/null and b/docs/images/hosts.png differ diff --git a/docs/images/load-balancing.png b/docs/images/load-balancing.png new file mode 100644 index 0000000..d21f2b9 Binary files /dev/null and b/docs/images/load-balancing.png differ diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..1178b61 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,79 @@ +--- +layout: default +title: Home +description: A reverse proxy and HTTPS termination service built on OpenResty/nginx, with automatic Let's Encrypt certs, OIDC login, and direct LDAP access control per host. +--- + +# Proxy + +A reverse proxy and HTTPS termination service built on OpenResty/nginx, with a +management API and web GUI. It puts any of your apps behind single sign-on +(OIDC) and can also look users up directly in LDAP — so the same people who +log in to your SSO are the people allowed to reach your proxied apps. + +Automatic HTTPS from Let's Encrypt (including wildcards), routing by hostname, +and per-host access control tied to your identity provider — managed from a +web UI or a REST API, with no downtime on config changes. + +Part of the theta42 self-hosted identity stack, alongside +[SSO Manager](https://theta42.github.io/sso-manager-node/) and +[theta-env](https://theta42.github.io/theta-env/) (the two composed with one +command). + +## Screenshots + +Host list +Per-host SSO auth + +Basic auth and SSO are mutually exclusive per host, with per-user password +management once basic auth is enabled: + +Per-host basic auth + +*(click any screenshot to view full size)* + +## Why this over the alternatives + +Nginx Proxy Manager, Traefik, and Caddy are all good reverse proxies with +auto-HTTPS. This one is built around identity: it is both an **OIDC client** +of an SSO provider (for browser login) **and** a direct **LDAP client** (for +user lookups and per-host access control), so access decisions come from your +real user directory, not a static allow-list or a separate auth proxy bolted +on top. The trade-off is that it expects an OIDC/LDAP identity source to point +at — it is not a standalone auth server. Pair it with +[SSO Manager](https://theta42.github.io/sso-manager-node/) (bundled OpenLDAP + +OIDC) for a self-hosted SSO + proxy stack, or point it at any OIDC provider + +LDAP directory you already run. + +## Features + +- Automated HTTPS via Let's Encrypt — HTTP-01 and DNS-01 (wildcard) challenges +- Multiple DNS providers (Cloudflare, DigitalOcean, PorkBun, DuckDNS — free) +- Dynamic host routing with wildcard domain matching (`*`, `**`) +- **Multi-target load balancing** — configure multiple backend targets per host with built-in round-robin load balancing +- **OIDC login** and **direct LDAP lookups**, independently of each other +- Per-host **basic auth** as an alternative to SSO (mutually exclusive, so + it's never ambiguous which one gated a request) +- **Role-based access control** — global admins, local groups, and + per-domain permissions (viewer/manager) +- Self-service API tokens for scripting/CI without a browser session +- Web UI and a full REST API + +## Get it + +```bash +git clone https://github.com/theta42/proxy.git +cd proxy && docker compose up -d --build +``` + +That's the standalone quick start. For the full set of install options (Docker, +bare-metal, or as part of the combined SSO + proxy stack), configuration +reference, and API docs, see the +**[GitHub repository](https://github.com/theta42/proxy)**. + +## Related projects + +- **[SSO Manager](https://theta42.github.io/sso-manager-node/)** — the OIDC + provider + LDAP directory this proxy is designed to sit in front of. +- **[theta-env](https://theta42.github.io/theta-env/)** — runs this proxy and + SSO Manager together with one command. diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000..7fbacdf --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,283 @@ +--- +layout: default +title: Installation +description: Installing the proxy — Docker, bare metal, or as part of the unified theta-env stack. +--- + +# Installation Guide + +[← Back to Home](index.html) + +> Looking for a plainer explanation of hosts, HTTPS, and DNS providers +> instead of install steps? See [Hosts & HTTPS](concepts-hosts.html) and +> [DNS Providers](concepts-dns.html). + +## Quick Install (Recommended) + +For modern Debian-based systems (Ubuntu 20.04+, Debian 11+): + +```bash +wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash +``` + +This automated installer will: +- Install Node.js 22.x +- Install OpenResty and required dependencies +- Install and configure Redis +- Set up SSL fallback certificates +- Install Lua dependencies +- Clone and install the proxy application +- Configure systemd service +- Start the proxy service + +## Manual Installation + +> **Recommended path:** `ops/install.sh` (above) is idempotent and safe to +> re-run — it symlinks the OpenResty/systemd config from the repo checkout, +> so future updates stay in sync automatically (`git pull` + re-run). The +> manual steps below *copy* those same files instead of symlinking them, so +> they will **not** auto-track later changes to `ops/nginx_conf/` or +> `ops/proxy.service` — you'd need to re-copy them by hand after every +> update. Prefer the manual path only if `install.sh` doesn't fit your +> distribution. + +### System Requirements + +- Modern Linux distribution (Ubuntu 20.04+, Debian 11+, or equivalent) +- Root access +- Inbound internet access for Let's Encrypt validation +- Minimum 1GB RAM, 10GB disk space + +### Step 1: Install Dependencies + +**Ubuntu/Debian:** +```bash +apt install libpam0g-dev build-essential redis-server luarocks -y +``` + +### Step 2: Install Node.js 22.x + +```bash +curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \ + sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg + +NODE_MAJOR=22 +echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | \ + sudo tee /etc/apt/sources.list.d/nodesource.list + +apt update && apt install nodejs -y +``` + +Verify installation: +```bash +node --version # Should show v22.x.x +npm --version +``` + +### Step 3: Install OpenResty + +openresty.org ships distinct trees (and components) for Debian and Ubuntu. The +Debian tree is published only up to **bookworm** (there is no trixie block) and +uses the **`openresty`** component; Ubuntu uses the host codename and **`main`**. +So on a Debian 13 (trixie) host, point at the `bookworm` distribution (binary- +compatible, same OpenSSL 3 era). + +```bash +. /etc/os-release +CODENAME="$(lsb_release -sc)" +case "$ID" in + debian) + OR_PATH=package/debian + OR_COMPONENT=openresty + # Debian tree only publishes up to bookworm; fall back to it for trixie+. + case "$CODENAME" in jessie|stretch|buster|bullseye|bookworm) OR_DISTRO="$CODENAME";; *) OR_DISTRO=bookworm;; esac + ;; + *) + OR_PATH=package/ubuntu + OR_DISTRO="$CODENAME" + OR_COMPONENT=main + ;; +esac + +wget -O - https://openresty.org/package/pubkey.gpg | \ + sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg + +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/${OR_PATH} ${OR_DISTRO} ${OR_COMPONENT}" | \ + sudo tee /etc/apt/sources.list.d/openresty.list +``` + +> **Debian 13 (trixie):** apt's sequoia GPG backend rejects SHA-1 signatures by +> default, and the OpenResty signing key is still SHA-1, so `apt update` will +> refuse the repo. Extend the SHA-1 acceptance window before updating: +> ```bash +> sudo mkdir -p /etc/crypto-policies/back-ends +> sudo cp /usr/share/apt/default-sequoia.config /etc/crypto-policies/back-ends/apt-sequoia.config +> sudo sed -i 's/2026-02-01/2028-02-01/' /etc/crypto-policies/back-ends/apt-sequoia.config +> ``` +> (The `default-sequoia.config` file only ships on Debian 13+, so this is a no-op +> on older releases. `ops/install.sh` applies this automatically.) + +```bash +apt update && apt install openresty -y +``` + +### Step 4: Install Lua Dependencies + +```bash +luarocks install lua-resty-auto-ssl +luarocks install luasocket +``` + +### Step 5: SSL Configuration + +Create fallback SSL certificates: + +```bash +mkdir -p /etc/ssl/ + +openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \ + -subj '/CN=sni-support-required-for-valid-ssl' \ + -keyout /etc/ssl/resty-auto-ssl-fallback.key \ + -out /etc/ssl/resty-auto-ssl-fallback.crt +``` + +### Step 6: Configure OpenResty + +Clone the repository and copy configuration files: + +```bash +mkdir -p /opt/theta42 +cd /opt/theta42 +git clone https://github.com/theta42/proxy.git +cd proxy + +# Copy nginx configs +mkdir -p /etc/openresty/sites-enabled/ +cp ops/nginx_conf/nginx.conf /etc/openresty/nginx.conf +cp ops/nginx_conf/autossl.conf /etc/openresty/autossl.conf +cp ops/nginx_conf/proxy.conf /etc/openresty/sites-enabled/000-proxy +cp ops/nginx_conf/targetinfo.lua /usr/local/openresty/lualib/targetinfo.lua +``` + +### Step 7: Install Application + +```bash +cd /opt/theta42/proxy/nodejs +npm install +``` + +### Step 7b: Configure Secrets + +```bash +mkdir -p /etc/proxy +cp /opt/theta42/proxy/secrets.js.example /etc/proxy/secrets.js +chmod 600 /etc/proxy/secrets.js +$EDITOR /etc/proxy/secrets.js # set oidc.clientId/clientSecret, ldap.bindPassword, ... +``` + +`@simpleworkjs/conf` reads this file via the `CONF_SECRETS` env var, which the +systemd unit below sets to `/etc/proxy/secrets.js`. + +### Step 8: Configure Systemd Service + +```bash +cp /opt/theta42/proxy/ops/proxy.service /etc/systemd/system/proxy.service +systemctl daemon-reload +systemctl enable proxy.service +systemctl start proxy.service +``` + +Verify service is running: +```bash +systemctl status proxy.service +``` + +### Step 9: Initial Setup + +The proxy API will be available on port 3000 by default. You'll need to: + +1. Create your first user account +2. Configure DNS providers (for wildcard SSL) +3. Add your first host + +See the [API Reference](api.html) for details. + +## Configuration + +### Environment Variables + +- `NODE_ENV` - Set to `production` for production deployments +- `NODE_PORT` - Override default port (default: 3000) + +### Redis Configuration + +The proxy uses Redis with the prefix `proxy_`. To change this, edit `nodejs/conf/base.js`: + +```javascript +redis: { + prefix: 'proxy_' +} +``` + +### OpenResty Configuration + +Key configuration files in `/etc/openresty/`: +- `nginx.conf` - Main nginx configuration +- `autossl.conf` - Let's Encrypt HTTP-01 challenge handler +- `sites-enabled/000-proxy` - Proxy server configuration + +### Unix Socket + +The proxy communicates with OpenResty via Unix socket at: +``` +/var/run/proxy_lookup.socket +``` + +This path is configurable in `nodejs/conf/base.js`. + +## Troubleshooting + +### Service won't start + +Check logs: +```bash +journalctl -u proxy.service -f +``` + +Common issues: +- Port 3000 already in use +- Redis not running: `systemctl status redis-server` +- Permission issues: Service must run as root for user management + +### SSL certificates not working + +Check OpenResty logs: +```bash +tail -f /var/log/nginx/error.log +``` + +Common issues: +- Firewall blocking ports 80/443 +- DNS not pointing to server +- Let's Encrypt rate limits exceeded + +### Host lookup not working + +Check Unix socket: +```bash +ls -la /var/run/proxy_lookup.socket +# Should show srwxrwxrwx (socket permissions) +``` + +Test lookup: +```bash +echo '{"domain":"example.com"}' | nc -U /var/run/proxy_lookup.socket +``` + +## Next Steps + +- [Configure DNS Providers](api.html#dns-providers) for wildcard SSL +- [Add your first host](api.html#hosts) +- [Set up the web interface](index.html) + +[← Back to Home](index.html) diff --git a/docs/robots.txt b/docs/robots.txt new file mode 100644 index 0000000..bd503ab --- /dev/null +++ b/docs/robots.txt @@ -0,0 +1,4 @@ +User-agent: * +Allow: / + +Sitemap: https://theta42.github.io/proxy/sitemap.xml