diff --git a/CHANGELOG.md b/CHANGELOG.md index 064af47..367e1ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,7 @@ -# v1.13.2 +## v1.13.3 +- fix: remove missing DEPLOYMENT.md and docs/ from Docker build context + +## v1.13.2 - chore: Update CI pipeline integration # Changelog diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md deleted file mode 100644 index 3a2575a..0000000 --- a/DEPLOYMENT.md +++ /dev/null @@ -1,287 +0,0 @@ -# Deployment Guide — theta42/proxy - -The proxy is an OpenID Connect-protected reverse proxy (OpenResty front + Node -management app + Redis) that is **both** an OIDC client of an SSO Manager *and* -a direct LDAP client for user lookups. Two deployment methods: - -1. **Docker** — a single all-in-one image bundling OpenResty + the app + Redis - (`docker compose up`). -2. **Bare metal** — `ops/install.sh` on Debian/Ubuntu (installs Node.js, - OpenResty, Lua modules, Redis, and a systemd unit). - -## How configuration works - -The app loads configuration via [`@simpleworkjs/conf`](https://www.npmjs.com/package/@simpleworkjs/conf), -which deep-merges, in order: - -1. `conf/base.js` (committed, generic defaults) -2. `conf/.js` (optional) -3. `conf/secrets.js` (gitignored — secrets + per-deployment values) -4. **`app_*` environment variables** — the highest-precedence layer - -Any env var whose name starts with `app_` overrides the merged config. The rest -of the name is split on **double-underscore** (`__`) into a nested path. Values -are `JSON.parse`-coerced when possible and kept as raw strings otherwise. - -| Env var | Sets | Notes | -|---------|------|-------| -| `app_oidc__issuer` | `conf.oidc.issuer` | browser-facing SSO URL | -| `app_oidc__authorizationEndpoint` | `conf.oidc.authorizationEndpoint` | browser-facing | -| `app_oidc__tokenEndpoint` | `conf.oidc.tokenEndpoint` | server-to-server; can be internal | -| `app_oidc__userinfoEndpoint` | `conf.oidc.userinfoEndpoint` | server-to-server; can be internal | -| `app_oidc__endSessionEndpoint` | `conf.oidc.endSessionEndpoint` | browser-facing | -| `app_oidc__clientId` / `app_oidc__clientSecret` | OIDC client creds | register in the SSO first | -| `app_oidc__redirectUri` | `conf.oidc.redirectUri` | must match the SSO client exactly | -| `app_oidc__enabled` | `conf.oidc.enabled` | boolean | -| `app_ldap__url` | `conf.ldap.url` | `ldaps://…:636` or `ldap://…:389` | -| `app_ldap__bindDN` / `app_ldap__bindPassword` | LDAP service account | don't reuse the admin DN | -| `app_ldap__searchBase` / `app_ldap__userFilter` | user search | | -| `app_ldap__tlsOptions__rejectUnauthorized` | `conf.ldap.tlsOptions.rejectUnauthorized` | `false` for self-signed LDAPS | -| `app_ldap__tlsOptions__ca` | `conf.ldap.tlsOptions.ca` | path to a CA cert for strict trust | -| `app_auth__adminUsers` | `conf.auth.adminUsers` | local anti-lockout admin (uid) | -| `app_auth__adminGroups` | `conf.auth.adminGroups` | SSO/LDAP groups that are global admin (JSON array) | -| `app_auth__localAdminPass` | `conf.auth.localAdminPass` | initial password for the local anti-lockout admin (used once, on first creation only — defaults to the username itself if unset) | -| `app_redis__prefix` | `conf.redis.prefix` | default `proxy_` | - -See [`docs/docker.md`](docs/docker.md) for a shorter, container-focused version -of this reference. - -> **Requires `@simpleworkjs/conf` >= 1.1.0.** The Docker image will not honor -> `app_*` env vars on 1.0.0. The lock is already on `^1.1.0`; if you regenerate it: -> ```bash -> cd nodejs && npm install @simpleworkjs/conf@^1.1.0 -> ``` - ---- - -## Method 1: Docker (all-in-one) - -The image (`Dockerfile`) bundles OpenResty + the app + Redis in one container, -mirroring the bare-metal `ops/install.sh` layout. `docker-entrypoint.sh`: -generates the fallback SSL cert, parameterizes the OpenResty `resolver`/ -`set_real_ip_from` directives, starts Redis + the node app, and execs OpenResty -in the foreground under `dumb-init`. - -### Setup - -The bundled `docker-compose.yml` reads the OIDC + LDAP + auth wiring from a -bind-mounted `./config/proxy-secrets.js` (not from `app_*` env). Copy the -example, fill in your secrets, then build + start: - -```bash -mkdir -p config && chmod 700 config -cp secrets.js.example config/proxy-secrets.js -$EDITOR config/proxy-secrets.js # set oidc.clientId/clientSecret, ldap.bindPassword, ... -docker compose up -d --build -``` - -`docker-entrypoint.sh` sets `CONF_SECRETS=/config/proxy-secrets.js` so -`@simpleworkjs/conf` reads it directly. No `app_*` env is passed — `app_*` env -would override the file (env beats secrets.js in `@simpleworkjs/conf`), so the -file is kept authoritative. `RESOLVER` / `REAL_IP_FROM` / `NODE_ENV` / -`NODE_PORT` are OpenResty-runtime / process env, not `app_*` config, so they -stay in the compose. - -> Running the unified `theta-env` stack? Its `setup.sh` generates -> `./config/proxy-secrets.js` (+ `./config/sso-secrets.js`) for you and -> registers the OAuth client with the SSO — see the theta-env README. - -### Access - -- Proxy (public): `https:///` — OpenResty front, auto-SSL (Let's Encrypt) -- Management UI / API: `http://127.0.0.1:3000/` (bound to localhost; the front - proxies the UI under its own TLS) -- Health: `http://127.0.0.1:3000/health` → `{"status":"ok"}` - -### API tokens (personal access tokens) - -Any logged-in user can mint a long-lived bearer token to call the management -API from scripts/CI/other services, without an OIDC browser session. Tokens are -self-service and authenticate **as their creator**: the creator's groups are -snapshotted at mint time (mirroring how the proxy's browser session captures -groups at login — the proxy never re-queries the IdP), and the existing authz -layer (`Permission.effectiveFor` / `roles.resolveEffective`) applies unchanged. -Local groups and owned-domain rights are recomputed live each request; only the -SSO/LDAP group membership is the mint-time snapshot. - -Create one in the UI under **API Tokens** (the token string is shown **once**), -then use it as a bearer token: - -```bash -curl -H "Authorization: Bearer prx__" https://proxy.example.com/api/host -``` - -Format: `prx__` — the `id` is the lookup key, the `secret` is -bcrypt-hashed and never stored in plaintext. Rotate or revoke from the same page -(immediate effect). Optional expiry (in days) at creation. Tokens persist in the -bundled Redis (AOF — see *Backups and restore*), so they survive rebuilds. - -The token carries the creator's effective rights: a global admin's token can -manage Hosts/Users/Groups; a domain manager's token can manage their own -domains but `requireAdmin` routes return 403. To tighten permissions after group -changes, revoke and re-mint the token. - -### OpenResty runtime env - -| Variable | Default | Description | -|----------|---------|-------------| -| `RESOLVER` | `127.0.0.11` | DNS for upstream names in Host records (Docker's embedded DNS) | -| `REAL_IP_FROM` | _empty_ | Trusted CIDR for `X-Real-IP`. Empty = proxy is the front (removes the real_ip block). Set to an upstream proxy's CIDR if one sits in front. | - -### Auto-SSL / Let's Encrypt - -`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB -persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs -survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is -required for HTTP-01 challenges (mapped in the compose). - -### Backups and restore - -**What lives where** - -| State | Location | Persisted? | -|-------|----------|------------| -| Host records, permissions, DNS creds, local users | `proxy-data` volume (`/data`, Redis) | yes (AOF + RDB) | -| Let's Encrypt certs (auto-ssl) | `proxy-data` volume (`/data`, Redis) | yes — same Redis | -| nginx response cache / logs | `proxy-cache` / `proxy-logs` volumes | yes (volume) | -| Secrets (OIDC client secret, LDAP bind password) | `./config/proxy-secrets.js` (bind mount) | your responsibility — back up off-host | - -**Automatic snapshots** — when run as part of the unified `theta-env` stack, -`setup.sh` snapshots Redis + `./config/` to `./backups//` before every -rebuild and keeps the last `BACKUP_KEEP` (default 5). Standalone deployments -should run `ops/backup.sh` the same way (on a cron/systemd timer, or by hand -before an upgrade): - -```bash -./ops/backup.sh # keeps the last 5 by default -./ops/backup.sh 10 # or override retention -BACKUP_KEEP=10 ./ops/backup.sh -``` - -It snapshots Redis (`BGSAVE`, falling back to a synchronous `SAVE` if that -doesn't complete quickly) and `./config/` to `./backups//`, -pruning older backups beyond the retention count — the same approach -`theta-env`'s `setup.sh` uses, just scoped to this one container. Equivalent -manual steps, if you'd rather not use the script: - -```bash -# Redis — hot snapshot: trigger a save, then copy the RDB out -docker compose exec proxy redis-cli BGSAVE -docker compose cp proxy:/data/dump.rdb proxy-redis-$(date +%F).rdb - -# Secrets — copy the config dir (holds OIDC client secret, LDAP bind password) -cp -a ./config config-backup-$(date +%F) && chmod 700 config-backup-$(date +%F) -``` -Store the backup **off the host** — it contains secrets and the whole -Host/permission/user dataset. - -**Restore — Redis (full proxy state + certs)** - -```bash -cp -a config-backup- ./config && chmod 700 ./config -docker compose up -d -docker compose stop proxy -# AOF wins on startup — delete it so the RDB is loaded instead (see note). -docker compose run --rm --no-deps --entrypoint sh proxy -c \ - 'rm -f /data/appendonly.aof /data/appendonly.aof.*' -docker compose cp proxy-redis-.rdb proxy:/data/dump.rdb -docker compose start proxy -``` - -> **AOF vs RDB (important):** with `--appendonly yes`, Redis loads -> `appendonly.aof` on startup and **ignores** `dump.rdb` if the AOF exists. To -> restore from an RDB snapshot you **must delete the AOF first** (the runbook -> does this); Redis then loads the RDB and writes a fresh AOF. Verify: -> `docker compose exec proxy redis-cli DBSIZE`. -> -> Restoring Redis restores cert state **at the snapshot time** — certs issued -> after the snapshot are lost and will be re-issued on next request. - -**Upgrades** - -```bash -./setup.sh # backs up, then rebuilds — proxy-data keeps Redis state -# (standalone) docker compose pull && docker compose up -d -``` -Host records, permissions, DNS creds, local users, and Let's Encrypt certs all -survive the rebuild because they live on the `proxy-data` volume, not in the -image. **Migrations note:** if a release ships a `nodejs/migrations/` script, -run it after upgrading (it transforms in-Redis records); see the release notes. - -### Logs - -OpenResty runs in the foreground and the Node app in the background, both -writing to the container's stdout/stderr. nginx access/error logs go to files -(`/var/log/nginx`, on the `proxy-logs` volume), so they do **not** appear in -`docker logs`. - -```bash -docker compose logs -f proxy # app + OpenResty (stdout/stderr) -docker compose exec proxy tail -f /var/log/nginx/error.log # nginx errors -docker compose exec proxy tail -f /var/log/nginx/access.log # nginx access -docker compose logs --tail=200 --since=10m proxy # recent context -``` - ---- - -## Method 2: Bare metal (Debian/Ubuntu) - -`ops/install.sh` is an idempotent installer: it installs Node.js 22.x, OpenResty -(from openresty.org), Lua modules (luarocks), Redis, force-syncs the repo to -`/opt/theta42/proxy`, symlinks the OpenResty + systemd config from the repo, and -starts `proxy.service`. Re-run it to update — it prints the version you're -updating from and to (or "Already up to date" if there's nothing new). - -```bash -wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash -``` - -or, if you already have the repo checked out: - -```bash -sudo ./ops/install.sh -``` - -Configuration is file-based: on first run the installer seeds -`/etc/proxy/secrets.js` from `secrets.js.example` (placeholders you must fill -in — OIDC + LDAP values, see `nodejs/conf/base.js` for the shape). Edit it, -then `sudo systemctl restart proxy`. Later runs never touch an existing -secrets file. - ---- - -## Fronting an SSO Manager - -The proxy is a natural front for [`theta42/sso-manager-node`](https://github.com/theta42/sso-manager-node): -it terminates TLS for the SSO's UI and protects it with OIDC login, while also -binding to the SSO's LDAP directly for user lookups. To run both together: - -1. **One Docker network** so the proxy can reach the SSO internally at - `http://sso-manager:3001` (token/userinfo, server-to-server) and - `ldaps://sso-manager:636` (LDAP). -2. **Set the SSO's `OAUTH_ISSUER`** to the browser-facing HTTPS URL the proxy - serves the SSO at (e.g. `https://sso.example.com`). -3. **Register the proxy as an OIDC client** in the SSO, with a `redirectUri` - matching the proxy's callback (`https://proxy.example.com/api/auth/oidc/callback`). -4. **LDAP**: point `app_ldap__url` at `ldaps://sso-manager:636` and create a - dedicated service account (`cn=ldapclient,ou=people,…`) — don't reuse the - admin DN. For the SSO's self-signed LDAPS cert, set - `app_ldap__tlsOptions__rejectUnauthorized=false` (or mount the cert and use - `app_ldap__tlsOptions__ca=`). - -The [`theta42/theta-env`](https://github.com/theta42/theta-env) unified repo -automates all four steps with `./setup.sh`. - ---- - -## Security notes - -1. **Never commit `secrets.js`** — it's in `.gitignore`. -2. **Bind the management port to localhost** (the compose does: `127.0.0.1:3000`). - The OpenResty front proxies the UI under TLS; don't expose 3000 to the LAN. -3. **`REAL_IP_FROM` empty by default** — the proxy trusts no one to set `X-Real-IP` - (it's the front). Only set it if a trusted proxy sits in front. -4. **LDAPS for any LDAP that crosses the network.** Use `ldaps://`/StartTLS; plain - `ldap://` is fine only on a private docker network. -5. The image runs OpenResty workers as `nobody` and the node app as root (matches - the bare-metal systemd unit). Harden to a non-root user for production if needed. \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index 9ef111e..ace8a77 100644 --- a/Dockerfile +++ b/Dockerfile @@ -115,8 +115,6 @@ COPY nodejs/api.md ./api.md # docs/ mirrors the repo's own top-level docs/ folder. COPY README.md /README.md COPY CHANGELOG.md /CHANGELOG.md -COPY DEPLOYMENT.md /DEPLOYMENT.md -COPY docs /docs # Baked commit hash from the gitinfo stage (see build_info.js). COPY --from=gitinfo /commit.txt ./.build_commit diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 8203639..0000000 --- a/docs/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# Documentation - -This directory contains the GitHub Pages documentation site for the Proxy project. - -**Live site:** https://theta42.github.io/proxy/ - -## Pages - -- `index.md` - Home page with project overview -- `installation.md` - Installation and setup guide -- `api.md` - Complete API reference -- `architecture.md` - System architecture and design -- `contributing.md` - Development and contribution guide - -## Local Preview - -To preview the site locally: - -```bash -# Install Jekyll (one-time setup) -gem install jekyll bundler - -# Run local server -cd docs -jekyll serve - -# View at http://localhost:4000/proxy/ -``` - -## Theme - -The site uses the Cayman theme (`jekyll-theme-cayman`). Configuration is in `_config.yml`. - -## Updating Documentation - -1. Edit markdown files in this directory -2. Commit and push to master branch -3. GitHub Pages automatically rebuilds (may take 1-2 minutes) -4. Changes visible at https://theta42.github.io/proxy/ diff --git a/docs/_config.yml b/docs/_config.yml deleted file mode 100644 index 560f548..0000000 --- a/docs/_config.yml +++ /dev/null @@ -1,47 +0,0 @@ -title: Proxy -description: A reverse proxy and HTTPS termination service built on OpenResty/nginx, with an OIDC + LDAP-aware management API and web GUI. -url: "https://theta42.github.io" -baseurl: "/proxy" -logo: /assets/img/theta42.svg -lang: en_US - -plugins: - - jekyll-seo-tag - - jekyll-sitemap - -github: - repository_url: https://github.com/theta42/proxy - zip_url: https://github.com/theta42/proxy/archive/refs/heads/master.zip - tar_url: https://github.com/theta42/proxy/archive/refs/heads/master.tar.gz - repository_name: theta42/proxy - -nav: - - title: Home - page: / - icon: fa-house - - title: Installation - page: /installation.html - icon: fa-download - - title: Architecture - page: /architecture.html - icon: fa-sitemap - - title: API - page: /api.html - icon: fa-code - - title: Docker - page: /docker.html - icon: fa-box - - title: Contributing - page: /contributing.html - icon: fa-code-branch - - title: Changelog - url: https://github.com/theta42/proxy/blob/master/CHANGELOG.md - icon: fa-list - -defaults: - - scope: - path: "" - type: "pages" - values: - layout: default - image: /assets/img/theta42.svg diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html deleted file mode 100644 index 132c524..0000000 --- a/docs/_layouts/default.html +++ /dev/null @@ -1,82 +0,0 @@ - - - - - - - - {% seo title=false %} - {% if page.title %}{{ page.title }} · {% endif %}{{ site.title }} - - - - - - - - - -
-
-
-
-
-
- {{ content }} -
-
-
-
-
-
- - - - - - diff --git a/docs/api.md b/docs/api.md deleted file mode 100755 index a497b71..0000000 --- a/docs/api.md +++ /dev/null @@ -1,972 +0,0 @@ ---- -layout: default -title: API Reference -description: The proxy's management REST API — hosts, DNS providers, users, groups, and permissions. ---- - -# API Documentation - -[← Back to Home](index.html) - -All API endpoints require authentication unless otherwise noted. Three -authentication methods are supported: - -- **`auth-token` header** — a browser-session token from `POST /api/auth/login` - or the OIDC flow (below). -- **`Authorization: Bearer ` header** — a self-service API token (PAT, - see [API Tokens](#api-tokens)), for scripts/CI without a browser session. -- **OIDC (browser)** — if the proxy is configured as an OIDC client of an SSO - (`app_oidc__*` / `conf.oidc`, see [DEPLOYMENT.md](https://github.com/theta42/proxy/blob/master/DEPLOYMENT.md)), - users can log in via `GET /api/auth/oidc/start` instead of posting a - username/password. - -The proxy can also be configured as a **direct LDAP client** (`app_ldap__*` / -`conf.ldap`) for looking up/validating users, independent of the OIDC flow — -see DEPLOYMENT.md for the full configuration reference. - -Authenticated requests also carry **RBAC** (role-based access control): -global admins can manage everything; other users are scoped to `viewer` or -`manager` rights on specific domains via [Permissions](#permissions) and -[Groups](#groups). - -Base URL: `https://your-proxy-host.com/api` - ---- - -## Authentication - -### Login - -**POST** `/api/auth/login` - -Authenticate a user and receive an auth token. - -```bash -curl -H "Content-Type: application/json" \ - -X POST \ - -d '{"username": "myuser", "password": "mypassword"}' \ - https://proxy-host.com/api/auth/login -``` - -**Responses:** -- `200` `{"login": true, "token": "027d3964-7d81-4462-a6f9-2c1f9b40b4be", "message": "myuser logged in!"}` -- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - -### Logout - -**ALL** `/api/auth/logout` - -Invalidate the current auth token. - -```bash -curl -H "auth-token: your-token-here" \ - -X POST \ - https://proxy-host.com/api/auth/logout -``` - -**Responses:** -- `200` `{"message": "Bye"}` - -### OIDC Login (start) - -**GET** `/api/auth/oidc/start` - -Begin the OIDC authorization-code flow: creates a PKCE + state challenge and -redirects the browser to the configured SSO's authorize endpoint. Only -available when `conf.oidc.enabled` is true. - -**Query Parameters:** -- `redirect` - Internal path to return to after login (optional; sanitized to same-origin) - -```bash -curl -i "https://proxy-host.com/api/auth/oidc/start?redirect=/hosts" -``` - -**Responses:** -- `302` Redirect to the SSO's authorization endpoint -- `404` `{"name": "OidcDisabled", "message": "OIDC login is not enabled."}` - -### OIDC Callback - -**GET** `/api/auth/oidc/callback` - -Redirect target for the SSO after login. Validates the one-time `state`, -exchanges the authorization `code` for tokens, reads identity from the -userinfo endpoint, establishes a session, and redirects the browser back to -the login page with the app's own `auth-token` in a URL fragment. - -**Query Parameters:** -- `code` (required) - Authorization code from the SSO -- `state` (required) - State value from the `start` step - -```bash -# Not called directly — the SSO redirects the browser here after login. -``` - -**Responses:** -- `302` Redirect to `/login#token=...&redirect=...` -- `400` `{"name": "OidcCallbackInvalid", "message": "Missing code or state."}` or expired/unknown state - ---- - -## API Tokens - -Self-service personal access tokens (PATs) for scripting/CI without a browser -session. Every endpoint is owner-scoped: a user only sees/manages tokens they -created. Mounted at `/api/api-token`. - -### List API Tokens - -**GET** `/api/api-token` - -List the current user's API tokens. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/api-token -``` - -**Responses:** -- `200` `{"results": [{"id": "...", "name": "ci", ...}, ...]}` - -### Create API Token - -**POST** `/api/api-token` - -Create a new API token. The raw token string is only returned once, at -creation. - -**Parameters:** -- `name` (required) - Display name -- `description` (optional) -- `expires_in_days` (optional) - `0` or omitted means no expiry - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "ci", "expires_in_days": 90}' \ - https://proxy-host.com/api/api-token -``` - -**Responses:** -- `200` `{"results": {...}, "token": "prx__", "message": "API token 'ci' created. Save it now — it will not be shown again."}` - -### Get API Token - -**GET** `/api/api-token/:id` - -Get a token's metadata (not the raw secret, which is never stored/returned again). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/api-token/ -``` - -**Responses:** -- `200` `{"results": {...}}` -- `403` Not your token - -### Update API Token - -**PUT** `/api/api-token/:id` - -Update a token's name/description/expiry. - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X PUT \ - -d '{"name": "ci-updated"}' \ - https://proxy-host.com/api/api-token/ -``` - -**Responses:** -- `200` `{"results": {...}, "message": "API token 'ci-updated' updated."}` - -### Delete (Revoke) API Token - -**DELETE** `/api/api-token/:id` - -Revoke a token immediately. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/api-token/ -``` - -**Responses:** -- `200` `{"id": "", "message": "API token 'ci' revoked."}` - -### Rotate API Token - -**POST** `/api/api-token/:id/rotate` - -Issue a new secret for an existing token (same id, new raw value shown once). - -```bash -curl -H "auth-token: your-token-here" \ - -X POST \ - https://proxy-host.com/api/api-token//rotate -``` - -**Responses:** -- `200` `{"token": "prx__", "message": "API token 'ci' rotated. Save it — it will not be shown again."}` - ---- - -## Users - -All user endpoints require authentication. `GET /me` and `PUT /password` -(self-service) work for any authenticated user; everything else (listing, -creating, deleting users, resetting another user's password) requires global -admin. - -### List Users - -**GET** `/api/user` - -Get list of all users. Admin only. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/user -``` - -**Query Parameters:** -- `detail` - Include full user details (optional) - -**Responses:** -- `200` `{"results": ["user1", "user2"]}` -- `200` `{"results": [{"username": "user1", ...}, ...]}` (with `?detail=true`) -- `403` Not an admin - -### Get Current User - -**GET** `/api/user/me` - -Get the currently authenticated user's identity and effective RBAC rights -(drives the web UI's nav/button gating). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/user/me -``` - -**Responses:** -- `200` `{"username": "myuser", "groups": [...], "localGroups": [...], "externalGroups": [...], "isAdmin": false, "global": null, "domains": {...}}` - -### Create User - -**POST** `/api/user` - -Create a new local user. Admin only. - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"username": "newuser", "password": "newpassword"}' \ - https://proxy-host.com/api/user -``` - -**Responses:** -- `200` User created successfully -- `403` Not an admin -- `409` Username already exists -- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error (also returned for weak passwords) - -### Delete User - -**DELETE** `/api/user/:username` - -Delete a user account. Admin only. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/user/olduser -``` - -**Responses:** -- `200` `{"username": "olduser", "results": ...}` -- `403` Not an admin -- `404` User not found - -### Change Password (Self) - -**PUT** `/api/user/password` - -Change the password for the currently authenticated user. - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X PUT \ - -d '{"password": "newpassword"}' \ - https://proxy-host.com/api/user/password -``` - -**Responses:** -- `200` `{"results": ...}` Password changed successfully -- `422` Weak password rejected by the password policy - -### Change Password (Other User) - -**PUT** `/api/user/password/:username` - -Change the password for another user. Admin only. - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X PUT \ - -d '{"password": "newpassword"}' \ - https://proxy-host.com/api/user/password/otheruser -``` - -**Responses:** -- `200` `{"results": ...}` Password changed successfully -- `403` Not an admin -- `404` User not found - ---- - -## Permissions - -RBAC: grants a `viewer` or `manager` role to a user or group, either globally -or scoped to one domain. Global-admin-only. Mounted at `/api/permission`. - -### List Permissions - -**GET** `/api/permission` - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/permission -``` - -**Responses:** -- `200` `{"results": [{"id": "...", "subjectType": "user", "subject": "alice", "role": "manager", "scope": "domain", "domain": "example.com", ...}, ...]}` - -### List Permission Subjects - -**GET** `/api/permission/subjects` - -Autocomplete source for the "Subject" field: known usernames plus known group -names (local groups, groups already used in permissions, and groups from -`conf.auth.adminGroups` / `conf.auth.groupRoleMap`). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/permission/subjects -``` - -**Responses:** -- `200` `{"users": ["alice", "bob"], "groups": ["ops", "sre"]}` - -### Create Permission - -**POST** `/api/permission` - -Grant a role to a subject. - -**Parameters:** -- `subjectType` (required) - `user` or `group` -- `subject` (required) - username or group name -- `role` (required) - `viewer` or `manager` -- `scope` (required) - `global` or `domain` -- `domain` (required if `scope` is `domain`) - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"subjectType": "user", "subject": "alice", "role": "manager", "scope": "domain", "domain": "example.com"}' \ - https://proxy-host.com/api/permission -``` - -**Responses:** -- `200` `{"message": "Granted manager to user \"alice\" on example.com.", ...}` -- `422` Validation error - -### Delete Permission - -**DELETE** `/api/permission/:id` - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/permission/ -``` - -**Responses:** -- `200` `{"message": "Permission removed."}` - ---- - -## Groups - -Local groups (independent of any SSO/LDAP groups) used as subjects for -permission grants. Global-admin-only. Mounted at `/api/group`. - -### List Groups - -**GET** `/api/group` - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/group -``` - -**Responses:** -- `200` `{"results": [{"name": "ops", "members": ["alice", "bob"], ...}, ...]}` - -### Create Group - -**POST** `/api/group` - -**Parameters:** -- `name` (required) -- `members` (optional) - array of usernames - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "ops", "members": ["alice"]}' \ - https://proxy-host.com/api/group -``` - -**Responses:** -- `200` `{"message": "Group \"ops\" created.", ...}` - -### Delete Group - -**DELETE** `/api/group/:name` - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/group/ops -``` - -**Responses:** -- `200` `{"message": "Group \"ops\" removed."}` - -### Add Group Member - -**POST** `/api/group/:name/members` - -**Parameters:** -- `username` (required) - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"username": "bob"}' \ - https://proxy-host.com/api/group/ops/members -``` - -**Responses:** -- `200` `{"message": "Added \"bob\" to \"ops\".", ...}` - -### Remove Group Member - -**DELETE** `/api/group/:name/members/:username` - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/group/ops/members/bob -``` - -**Responses:** -- `200` `{"message": "Removed \"bob\" from \"ops\".", ...}` - ---- - -## Hosts - -Manage proxy host configurations. - -### List Hosts - -**GET** `/api/host` - -Get list of all configured hosts. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/host -``` - -**Query Parameters:** -- `detail` - Include full host details (optional) - -**Responses:** -- `200` `{"results": ["example.com", "*.wildcard.com"]}` -- `200` `{"results": [{"host": "example.com", "ip": "192.168.1.10", ...}, ...]}` (with `?detail=true`) - -### Get Host - -**GET** `/api/host/:host` - -Get configuration for a specific host. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/host/example.com -``` - -**Responses:** -- `200` `{"item": "example.com", "results": {"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, ...}}` -- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` - -### Lookup Host - -**GET** `/api/host/lookup/:domain` - -Test the host lookup algorithm (supports wildcard matching). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/host/lookup/sub.example.com -``` - -**Responses:** -- `200` `{"string": "sub.example.com", "results": {"host": "*.example.com", ...}}` -- `200` `{"string": "sub.example.com", "results": null}` (no match) - -### Get Lookup Tree - -**GET** `/api/host/lookupobj` - -Get the internal lookup tree structure (for debugging). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/host/lookupobj -``` - -**Responses:** -- `200` `{"results": {"com": {"example": {...}}}}` - -### Create Host - -**POST** `/api/host` - -Add a new host configuration. - -**Parameters:** -- `host` (required) - Domain name (e.g., `example.com`, `*.example.com`) -- `ip` (required) - Target IP address or FQDN -- `targetPort` (required) - Target port number (1-65535) -- `forcessl` (optional) - Force HTTPS redirect (default: true) -- `targetssl` (optional) - Use HTTPS to backend (default: false) -- `challengeType` (optional) - For wildcards: `DNS-01-wildcard` or `wildcardChild` - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, "forcessl": true, "targetssl": false}' \ - https://proxy-host.com/api/host -``` - -**Responses:** -- `200` `{"message": "\"example.com\" added.", "host": "example.com", ...}` -- `409` `{"name": "HostNameUsed", "message": "Host already exists"}` -- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error - -### Update Host - -**PUT** `/api/host/:host` - -Update an existing host configuration. - -**Parameters:** Same as Create Host (all optional) - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X PUT \ - -d '{"ip": "192.168.1.20", "targetPort": 9000}' \ - https://proxy-host.com/api/host/example.com -``` - -**Responses:** -- `200` `{"message": "\"example.com\" updated.", ...}` -- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` -- `422` Validation error - -### Delete Host - -**DELETE** `/api/host/:host` - -Remove a host configuration. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/host/example.com -``` - -**Responses:** -- `200` `{"message": "example.com deleted", ...}` -- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` - -### Clear Host Cache - -**DELETE** `/api/host/cache` - -Remove all cached wildcard-subdomain host lookups. Cache entries are created on -demand when a wildcard host serves a subdomain; clearing them forces the next -request for each subdomain to be resolved fresh through the lookup tree. -Admin only. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/host/cache -``` - -**Responses:** -- `200` `{"message": "Cleared 3 cached hosts.", "count": 3}` - -### Renew Wildcard Certificate - -**PUT** `/api/host/:host/renew` - -Manually trigger wildcard certificate renewal. - -```bash -curl -H "auth-token: your-token-here" \ - -X PUT \ - https://proxy-host.com/api/host/*.example.com/renew -``` - -**Responses:** -- `200` `{"message": "Requesting wildcard cert for *.example.com"}` -- `404` Host not found - ---- - -## DNS Providers - -Manage DNS provider integrations for wildcard SSL certificates. - -### List DNS Providers - -**GET** `/api/dns` - -Get list of configured DNS providers. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns -``` - -**Query Parameters:** -- `detail` - Include full provider details (optional) - -**Responses:** -- `200` `{"results": ["provider-id-1", "provider-id-2"]}` - -### List Available Provider Types - -**OPTIONS** `/api/dns` - -Get list of supported DNS provider types and their configuration requirements. - -```bash -curl -H "auth-token: your-token-here" \ - -X OPTIONS \ - https://proxy-host.com/api/dns -``` - -**Responses:** -- `200` `{"results": [{"name": "Cloudflare", "fields": {...}}, {"name": "DigitalOcean", ...}, {"name": "PorkBun", ...}, {"name": "DuckDns", ...}]}` - -### Create DNS Provider - -**POST** `/api/dns` - -Configure a new DNS provider. - -**Cloudflare:** -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "My Cloudflare", "dnsProvider": "Cloudflare", "token": "your-api-token"}' \ - https://proxy-host.com/api/dns -``` - -**DigitalOcean:** -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "My DO", "dnsProvider": "DigitalOcean", "token": "your-api-token"}' \ - https://proxy-host.com/api/dns -``` - -**PorkBun:** -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "My PorkBun", "dnsProvider": "PorkBun", "apiKey": "pk_xxx", "secretApiKey": "sk_xxx"}' \ - https://proxy-host.com/api/dns -``` - -**DuckDNS (free):** -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"name": "My DuckDNS", "dnsProvider": "DuckDns", "token": "your-duckdns-token", "subdomains": "myhost,myhost2"}' \ - https://proxy-host.com/api/dns -``` - -`subdomains` is a comma-separated list of the subdomains you've registered at -[duckdns.org](https://www.duckdns.org) (e.g. `myhost` for -`myhost.duckdns.org`), since DuckDNS has no API to list them for you. -DuckDNS only supports one A/AAAA record and one TXT record per domain (no -arbitrary sub-records) — enough for dynamic DNS and DNS-01 wildcard certs. - -**Responses:** -- `200` `{"message": "\"provider-id\" added.", ...}` -- `422` Validation error or invalid API credentials - -### Get DNS Provider - -**GET** `/api/dns/:id` - -Get a specific DNS provider configuration. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns/provider-id -``` - -**Responses:** -- `200` `{"item": "provider-id", "results": {...}}` -- `404` Provider not found - -### Update DNS Provider - -**PUT** `/api/dns/:id` - -Update DNS provider configuration. - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X PUT \ - -d '{"name": "Updated Name"}' \ - https://proxy-host.com/api/dns/provider-id -``` - -**Responses:** -- `200` `{"message": "\"provider-id\" updated.", ...}` -- `404` Provider not found - -### Delete DNS Provider - -**DELETE** `/api/dns/:id` - -Remove a DNS provider and all associated domains. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/dns/provider-id -``` - -**Responses:** -- `200` `{"message": "provider-id deleted", ...}` -- `404` Provider not found - -### List Domains - -**GET** `/api/dns/domain` - -List all domains from all configured providers. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns/domain -``` - -**Query Parameters:** -- `detail` - Include full domain details (optional) - -**Responses:** -- `200` `{"results": ["example.com", "test.com"]}` - -### Get Domain - -**GET** `/api/dns/domain/:domain` - -Get details for a specific domain. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns/domain/example.com -``` - -**Responses:** -- `200` `{"results": [{"domain": "example.com", "zoneId": "...", ...}]}` -- `404` Domain not found - -### Refresh Domains - -**POST** `/api/dns/domain/refresh/:providerId` - -Refresh the domain list from a DNS provider's API. - -```bash -curl -H "auth-token: your-token-here" \ - -X POST \ - https://proxy-host.com/api/dns/domain/refresh/provider-id -``` - -**Responses:** -- `200` `{"results": ...}` Updated domain list -- `404` Provider not found - -### Dynamic DNS - -A-records kept automatically pointed at this box's public (WAN) IP. All -`/api/dns/dynamic*` routes are viewer/manager scoped to the record's domain -(via [Permissions](#permissions)), not admin-only like the rest of `/api/dns`. - -#### Get Current Public IP - -**GET** `/api/dns/dynamic/ip` - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns/dynamic/ip -``` - -**Responses:** -- `200` `{"ip": "203.0.113.5"}` - -#### List Dynamic Records - -**GET** `/api/dns/dynamic` - -Lists records the caller may view (their own/granted domains, or all for admins). - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/dns/dynamic -``` - -**Responses:** -- `200` `{"results": [{"id": "...", "domain": "example.com", "name": "home", "last_status": "ok", ...}, ...]}` - -#### Create Dynamic Record - -**POST** `/api/dns/dynamic` - -Requires `manager` rights on the target domain. Applies the record immediately -against the current public IP (best-effort — failures are recorded in -`last_status` and retried by the scheduler). - -**Parameters:** -- `domain` (required) -- `name` (required) - sub-label, or `@` for the apex - -```bash -curl -H "Content-Type: application/json" \ - -H "auth-token: your-token-here" \ - -X POST \ - -d '{"domain": "example.com", "name": "home"}' \ - https://proxy-host.com/api/dns/dynamic -``` - -**Responses:** -- `200` `{"message": "\"home.example.com\" added.", ...}` -- `403` Missing `manager` rights on the domain -- `422` Validation error - -#### Refresh Dynamic Record - -**POST** `/api/dns/dynamic/:id/refresh` - -Force an immediate refresh of one record against the current public IP. -Requires `manager` rights on the record's domain. - -```bash -curl -H "auth-token: your-token-here" \ - -X POST \ - https://proxy-host.com/api/dns/dynamic//refresh -``` - -**Responses:** -- `200` `{"message": "Refreshed \"home.example.com\".", "result": {...}}` -- `403` Missing `manager` rights on the domain - -#### Delete Dynamic Record - -**DELETE** `/api/dns/dynamic/:id` - -Stop managing a record. Requires `manager` rights on the record's domain. -Leaves the provider's A record in place at its last value. - -```bash -curl -H "auth-token: your-token-here" \ - -X DELETE \ - https://proxy-host.com/api/dns/dynamic/ -``` - -**Responses:** -- `200` `{"message": "home.example.com removed.", ...}` -- `403` Missing `manager` rights on the domain - ---- - -## Certificates - -Retrieve SSL certificate information. - -### Get Certificate - -**GET** `/api/cert/:host` - -Get the SSL certificate for a host. - -```bash -curl -H "auth-token: your-token-here" \ - https://proxy-host.com/api/cert/example.com -``` - -**Responses:** -- `200` Certificate data including `cert_pem`, `fullchain_pem`, `privkey_pem`, expiry information -- `404` Certificate not found - ---- - -## Error Responses - -All endpoints may return the following error responses: - -- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - Authentication required or invalid -- `404` `{"name": "NotFound", "message": "..."}` - Resource not found -- `422` `{"name": "ObjectValidateError", "message": [...], "keys": [...]}` - Validation errors -- `500` Internal server error - -## Notes - -- All timestamps are in milliseconds since epoch -- Authenticated endpoints accept either the `auth-token` header (browser - session / OIDC login) or an `Authorization: Bearer ` API token -- Host names support wildcards: `*` (single level) and `**` (multi-level) -- DNS providers are validated on creation - invalid API credentials will be rejected -- Wildcard certificates are automatically renewed 30 days before expiration diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 29f7a04..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,291 +0,0 @@ ---- -layout: default -title: Architecture -description: How the proxy's OIDC client, LDAP client, and OpenResty routing fit together. ---- - -# Architecture - -[← Back to Home](index.html) - -> Looking for a plainer explanation of hosts, HTTPS, or the local -> permission model instead of internals? See -> [Hosts & HTTPS](concepts-hosts.html) and -> [Users, Groups & Permissions](concepts-access.html). - -## System Overview - -The proxy system consists of three main components working together to provide high-performance reverse proxying with automated SSL management. - -``` -┌──────────────────────────────────────────────────────────────┐ -│ Internet │ -└─────────────────────────┬────────────────────────────────────┘ - │ HTTPS/HTTP - ▼ -┌──────────────────────────────────────────────────────────────┐ -│ OpenResty/Nginx │ -│ ┌────────────────┐ ┌──────────────┐ ┌─────────────────┐ │ -│ │ SSL Termination│ │ Host Routing │ │ Request Proxying│ │ -│ │ (lua-resty- │ │ (targetinfo. │ │ │ │ -│ │ auto-ssl) │ │ lua) │ │ │ │ -│ └────────────────┘ └──────┬───────┘ └─────────────────┘ │ -└────────────┬──────────────────┼───────────────────────────┬──┘ - │ │ │ - Let's Encrypt 1. Check Redis FIRST Backend - HTTP-01 2. Unix Socket (fallback) Services - │ │ │ - ▼ ▼ ▼ -┌──────────────────────┐ ┌──────────────────────────────────┐ -│ Redis │ │ Node.js Application │ -│ (Primary Cache) │ │ ┌──────────────┐ ┌─────────┐ │ -│ - Host configs ◄────┼──┼──┤ Services │ │ Routes │ │ -│ - User accounts │ │ │ - host_lookup│ │ - /api/*│ │ -│ - SSL certs │ │ │ - scheduler │ │ │ │ -│ - Auth tokens │ │ └──────────────┘ └─────────┘ │ -└──────────────────────┘ └─────────┬────────────────────────┘ - │ - ▼ - ┌──────────────────────┐ - │ DNS Providers │ - │ - Cloudflare │ - │ - DigitalOcean │ - │ - PorkBun │ - │ - DuckDNS (free) │ - │ (DNS-01 challenges) │ - └──────────────────────┘ -``` - -## Component Details - -### OpenResty/Nginx (Frontend) - -**Responsibilities:** -- Accept incoming HTTP/HTTPS requests -- SSL termination using lua-resty-auto-ssl -- Host-based routing decisions (Redis-first lookup) -- Proxy requests to backend services - -**Key Features:** -- HTTP-01 ACME challenge handling for automatic SSL -- Redis-first host lookup with Node.js fallback via Unix socket -- High-performance event-driven architecture -- Support for WebSocket connections -- Continues serving cached hosts even if Node.js is down - -**Configuration Files:** -- `/etc/openresty/nginx.conf` - Main configuration -- `/etc/openresty/autossl.conf` - Let's Encrypt integration -- `/etc/openresty/sites-enabled/000-proxy` - Proxy configuration -- `/usr/local/openresty/lualib/targetinfo.lua` - Host lookup module - -### Node.js Application (Backend) - -**Responsibilities:** -- API for host/user/DNS management -- Wildcard SSL certificate orchestration -- Host lookup tree maintenance -- User authentication and authorization - -**Directory Structure:** -``` -nodejs/ -├── bin/www # Application entry point -├── conf/ # Configuration (base.js, environment overlays, secrets.js) -├── controller/ # App-level wiring (pubsub, startup) -├── migrations/ # One-off Redis data migration scripts -├── models/ # Data models -│ ├── host.js # Host configuration and lookup -│ ├── auth.js # Authentication logic -│ ├── user.js # User management -│ └── dns_provider/ # DNS provider implementations -├── routes/ # API endpoints -│ ├── host.js # Host CRUD operations -│ ├── dns.js # DNS provider management -│ ├── user.js # User management -│ ├── auth.js # Authentication (login + OIDC) -│ ├── permission.js # RBAC permission management -│ ├── group.js # Local group management -│ └── api_token.js # Self-service API (PAT) tokens -├── services/ # Background services -│ ├── host_lookup.js # Unix socket server -│ └── host_scheduler.js # Cert renewal scheduler -├── middleware/ # Express middleware -│ └── auth.js # Authentication middleware -└── utils/ # Utility modules - └── unix_socket_json.js # Unix socket server -``` - -### Redis (Data Store) - -**ORM:** [model-redis](https://www.npmjs.com/package/model-redis) - A lightweight Redis ORM for Node.js with schema validation, relationships, and automatic key management. - -**Stored Data:** -- Host configurations (domain, IP, port, SSL settings) -- User accounts and hashed passwords -- Authentication tokens -- SSL certificates (for wildcard domains) -- DNS provider credentials -- Domain-to-provider mappings - -**Key Prefixes:** -``` -proxy_Host_ # Host configuration -proxy_User_ # User account -proxy_AuthToken_ # Auth tokens -proxy_DnsProvider_ # DNS provider -proxy_Domain_ # Domain info -:latest # SSL certificate cache -``` - -## Request Flow - -### Standard HTTP/HTTPS Request - -1. **Client** sends HTTPS request to `app.example.com` -2. **OpenResty** receives request, terminates SSL -3. **Lua script** (`targetinfo.lua`) queries **Redis first** for host config -4. If **found in Redis**, jump to step 7 (Node.js not involved) -5. If **not in Redis**, Lua queries Node.js via Unix socket as fallback -6. **Node.js** performs host lookup (supports wildcards), caches result in Redis -7. **OpenResty** proxies request to backend service using target IP and port -8. **Response** proxied back to client - -**Resilience**: If Node.js goes down, all hosts already cached in Redis continue to work. Only new/uncached hosts will fail until Node.js recovers. - -### Wildcard SSL Certificate Request - -1. **User** creates wildcard host (`*.example.com`) via API -2. **Node.js** validates domain has DNS provider configured -3. **Let's Encrypt** DNS-01 challenge initiated -4. **DNS provider** API creates TXT record (`_acme-challenge.example.com`) -5. **Let's Encrypt** validates TXT record -6. **Certificate** generated and stored in Redis -7. **DNS provider** cleans up TXT record -8. **Background scheduler** monitors expiration, renews 30 days before expiry - -## Host Lookup Algorithm - -The lookup tree enables sophisticated domain matching: - -``` -Input: "api.v1.example.com" - -Tree Structure: -{ - "com": { - "example": { - "*": { // Matches api.example.com - "#record": {...} - }, - "v1": { - "api": { // Matches api.v1.example.com (exact) - "#record": {...} - } - } - } - } -} - -Priority: Exact > Single wildcard (*) > Double wildcard (**) -``` - -**Wildcard Types:** -- `example.com` - Exact match only -- `*.example.com` - Matches `sub.example.com` (single level) -- `**.example.com` - Matches any depth (`sub.deep.example.com`) -- `api.*.example.com` - Matches `api.v1.example.com`, `api.v2.example.com` - -## Security Architecture - -### Authentication Flow - -1. User sends credentials to `/api/auth/login` -2. Credentials validated against stored hash (bcrypt) -3. Token generated and stored in Redis with TTL -4. Token returned to client -5. Subsequent requests include token in `auth-token` header -6. Middleware validates token before processing request - -### SSL Certificate Security - -- **Private keys** stored only in Redis (memory/disk based on config) -- **Fallback certificates** used when SNI unavailable -- **Let's Encrypt** rate limiting respected -- **DNS provider credentials** marked as `isPrivate` (not returned in API) - -### Unix Socket Communication - -- Socket file: `/var/run/proxy_lookup.socket` -- Permissions: `777` (container-safe, single-use deployment) -- Protocol: JSON over Unix stream socket -- Buffer handling: Accumulates partial messages until complete JSON - -## Performance Optimizations - -### Caching Strategy - -The system uses a multi-tier caching approach: - -1. **Redis (L1 Cache)** - OpenResty checks Redis FIRST for every request - - Primary host configuration storage - - Survives Node.js restarts/failures - - Shared across all OpenResty workers - -2. **Node.js Lookup Tree (L2 Cache)** - In-memory host lookup with wildcard matching - - Only queried when Redis has no entry - - Rebuilt automatically when hosts change - - Supports complex wildcard resolution - -3. **Wildcard Parent Caching** - Resolved wildcard matches stored back to Redis - - Subsequent requests to `api.example.com` hit Redis directly - - No repeated wildcard resolution needed - -### Unix Socket vs HTTP API - -Unix socket chosen over HTTP for host lookups: -- **Lower latency** - No TCP overhead -- **Higher throughput** - No HTTP parsing -- **Simpler** - Direct JSON communication -- **Secure** - Filesystem permissions, no network exposure - -## Scalability Considerations - -### Current Architecture - -- **Single instance** - OpenResty + Node.js + Redis on one server -- **Vertical scaling** - Add CPU/RAM as needed -- **Limitations** - Unix socket ties OpenResty to Node.js on same host - -### Future Scaling Options - -- **Redis cluster** - Distribute data storage -- **Multiple OpenResty instances** - Load balance incoming requests -- **Stateless Node.js** - Run multiple API instances -- **Replace Unix socket** - Use TCP/HTTP for cross-host communication -- **Separate cert management** - Dedicated service for wildcard SSL - -## Monitoring and Observability - -### Logs - -- **OpenResty**: `/var/log/nginx/access.log`, `/var/log/nginx/error.log` -- **Node.js**: `journalctl -u proxy.service` -- **Redis**: `redis-cli MONITOR` - -### Health Checks - -- Node.js API: `curl http://localhost:3000/api/host` -- Redis: `redis-cli PING` -- OpenResty: `systemctl status openresty` -- Unix socket: `ls -la /var/run/proxy_lookup.socket` - -### Metrics to Monitor - -- Request rate and response times -- SSL certificate expiration dates -- Redis memory usage -- Host lookup cache hit rate -- Background service execution times - -[← Back to Home](index.html) diff --git a/docs/assets/css/style.css b/docs/assets/css/style.css deleted file mode 100644 index e24a5de..0000000 --- a/docs/assets/css/style.css +++ /dev/null @@ -1,116 +0,0 @@ -/* theta42 docs site — shares the in-app dark navbar/footer + card look - (Bootstrap 5 + Font Awesome, same as the running apps) rather than a - generic Jekyll theme. */ - -body { - background-color: #f4f5f6; -} - -.navbar-brand img { - filter: drop-shadow(0 0 2px rgba(0, 0, 0, .4)); -} - -.navbar-nav .nav-link.active { - color: #fff; - font-weight: 600; -} - -/* Markdown content typography, scoped to the card body so it doesn't leak - into the nav/footer. */ -.site-content h1:first-child { - margin-top: 0; -} - -.site-content h1, -.site-content h2, -.site-content h3 { - font-weight: 700; -} - -.site-content h2 { - margin-top: 2.5rem; - padding-bottom: .4rem; - border-bottom: 1px solid #e9ecef; -} - -.site-content h3 { - margin-top: 1.75rem; -} - -.site-content a { - color: #a3671f; - text-decoration-color: rgba(163, 103, 31, .35); -} - -.site-content a:hover { - color: #8a5a16; -} - -.site-content pre { - background-color: #212529; - color: #f8f9fa; - padding: 1rem 1.25rem; - border-radius: .375rem; - overflow-x: auto; -} - -.site-content code { - color: #a3671f; - background-color: #f4f0e8; - padding: .15em .4em; - border-radius: .25rem; - font-size: .875em; -} - -.site-content pre code { - color: inherit; - background: none; - padding: 0; -} - -.site-content table { - display: block; - overflow-x: auto; - width: 100%; - border-collapse: collapse; - margin: 1.25rem 0; -} - -.site-content table th, -.site-content table td { - border: 1px solid #dee2e6; - padding: .5rem .75rem; - text-align: left; -} - -.site-content table th { - background-color: #f8f9fa; -} - -.site-content blockquote { - border-left: 4px solid #C59341; - padding: .5rem 1rem; - margin: 1.25rem 0; - background-color: #f8f6f1; - color: #495057; -} - -.site-content img { - max-width: 100%; - height: auto; -} - -/* Screenshot grids in the markdown use width="49%" inline attrs for a - two-up desktop layout -- stack them on narrow screens instead of - squeezing to illegibility. */ -@media (max-width: 576px) { - .site-content img[width] { - width: 100% !important; - margin-bottom: .75rem; - } -} - -.site-content hr { - margin: 2rem 0; - border-top: 1px solid #e9ecef; -} diff --git a/docs/assets/img/favicon.svg b/docs/assets/img/favicon.svg deleted file mode 100644 index 35e1881..0000000 --- a/docs/assets/img/favicon.svg +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - - - - - - - - - - - - diff --git a/docs/assets/img/theta42.svg b/docs/assets/img/theta42.svg deleted file mode 100644 index e598305..0000000 --- a/docs/assets/img/theta42.svg +++ /dev/null @@ -1,51 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 42 - - diff --git a/docs/concepts-access.md b/docs/concepts-access.md deleted file mode 100644 index 76c9aa8..0000000 --- a/docs/concepts-access.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -layout: default -title: Users, Groups & Permissions -description: A plain-language guide to local admin accounts, groups, and the domain-scoped permission model in theta42/proxy. ---- - -# Users, Groups & Permissions - -This page explains, in plain language, who can manage what in this app. For -the deeper system-design detail, see [Architecture](architecture.html). - -## Two different ways to log in - -Most people who use apps you've proxied through this app never see this -app's own login at all — they use whatever authentication you set up on -the *individual host* (basic auth, or single sign-on through your SSO -Manager). This page is about a different, smaller group: the people who -manage the proxy itself — adding hosts, registering DNS providers, and so -on. - -There are two ways someone gets into the proxy's own management UI: - -- **A local account**, created on the **Users** page — a username and - password specific to this app. -- **Single sign-on**, if you've connected this proxy to an SSO Manager (or - another OIDC provider) — the same login your other connected apps use. - -Either way, once logged in, what they're actually *allowed to do* here is -controlled by permissions, described below. - -## Groups - -A **group** here is just a named list of local usernames, used to grant -the same permission to several people at once instead of one at a time. -If you're using SSO instead of local accounts, group membership normally -comes from your identity provider instead — local groups exist mainly for -the local-account case. - -## Permissions: scope + role - -Each **permission** entry grants one subject (a user or a group) one -**role**, at one **scope** — the two are independent choices: - -**Scope** — *where* the role applies: - -- **Domain** — only hosts under one specific domain (e.g. someone can - manage everything under `example.com`, but can't see or touch a - completely different domain you also proxy). -- **Global** — everywhere, across every domain this proxy manages. - -**Role** — *what* they can do within that scope: - -- **Viewer** — read-only. Can see hosts and their settings, but not - change anything. -- **Manager** — full control over hosts (create, edit, delete) within - that scope. -- **Admin** — same host control as Manager, **plus**, but *only when - granted at Global scope*, the ability to manage other people's - permissions, DNS providers, and local user accounts. An Admin role - granted at Domain scope instead of Global behaves exactly like Manager - for that one domain — it does not unlock those extra admin-only pages. - -In practice: give someone **Manager** on just the domain(s) they're -responsible for to delegate day-to-day host management without handing -them the keys to everything. Reserve **Global Admin** for people who -should be able to change anything, anywhere, including who else has -access. - -## Want more detail? - -This page doesn't cover the exact permission-checking implementation or -how SSO group membership maps into this system internally — for that, see -[Architecture](architecture.html). - -[← Back to Home](index.html) diff --git a/docs/concepts-api-tokens.md b/docs/concepts-api-tokens.md deleted file mode 100644 index 5efe7d4..0000000 --- a/docs/concepts-api-tokens.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -layout: default -title: API Tokens -description: A plain-language guide to personal access tokens in theta42/proxy. ---- - -# API Tokens - -This page explains what an API token is and when you'd want one. For the -full list of API endpoints a token can call, see the -[API reference](api.html). - -## What's an API token, in plain terms? - -Normally, you interact with this app by logging in through a web browser. -An **API token** (also called a personal access token, or PAT) is an -alternative way in — a long, random string that a script, a scheduled job, -or another program can use instead of a username and password, to act on -your behalf without a human typing a login in each time. - -If you've ever set up a script to talk to GitHub, GitLab, or a similar -service using a "token" instead of your real password, this is the same -idea. - -## When would you actually need one? - -Most people never need to create one of these — you'll only want a token -if you're automating something, for example: - -- A script that registers or updates hosts automatically (say, spinning up - a new service and wanting the proxy entry created for it without a - manual step). -- A monitoring or backup job that checks this app's health via its API. -- A configuration-management tool that keeps your host list in sync with - something else. - -If you're not doing any of that, you don't need an API token — just log in -normally through the web UI. - -## How it works - -Create a token from your Profile page, give it a name so you remember what -it's for later, and optionally an expiry. You'll be shown the token's -value **exactly once** — copy it somewhere safe immediately, because it -can't be viewed again afterward (only revoked or rotated). Whatever script -or tool you're using it with sends it along with each request, the same -way a browser sends your login session. - -A token acts **as you**, with **your** [permissions](concepts-access.html) -— if you're only a Manager on one domain, a token you create can't touch -any other domain either. If you ever suspect a token has leaked (ended up -somewhere it shouldn't have, like a public script or log file), revoke it -immediately from your Profile page; it stops working right away. - -## Want more detail? - -This page doesn't attempt to list every API endpoint or show request/ -response examples — for that, see the full [API reference](api.html). - -[← Back to Home](index.html) diff --git a/docs/concepts-dns.md b/docs/concepts-dns.md deleted file mode 100644 index 23e0d62..0000000 --- a/docs/concepts-dns.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -layout: default -title: DNS Providers -description: A plain-language guide to why theta42/proxy needs a DNS provider, and only for wildcard certificates. ---- - -# DNS Providers - -This page explains, in plain language, what a "DNS provider" is for in this -app and when you actually need one. For setup steps, see -[Installation](installation.html). - -## Do you need this at all? - -**Only if you want a [wildcard host](concepts-hosts.html)** (something like -`*.example.com` covering every subdomain with one certificate). A normal, -single-name host doesn't need a DNS provider configured at all — skip this -page entirely if that's all you're setting up. - -## Why a wildcard cert needs this extra step - -To prove you actually own `example.com` before issuing a certificate that -covers *every* possible subdomain of it, Let's Encrypt needs to see a -specific, temporary DNS record appear on that domain — something only the -real owner of the domain could add. A normal single-host certificate -doesn't need this because it can prove ownership a simpler way (by -responding to a web request instead). - -So: to get a wildcard certificate, this app needs to be able to add (and -later remove) that one temporary DNS record on your domain automatically, -which means it needs your domain registrar or DNS host's API credentials — -that's what registering a **DNS provider** here does. - -## What you're actually giving it access to - -A DNS provider entry only needs enough access to add/remove TXT records — -it's not given your registrar account's full login, and it can't do -anything to your domain besides that one narrow task (and, for some -providers, keeping a dynamic A record updated if you use that feature -separately). Check your specific provider's page in the -[Installation guide](installation.html) for exactly what kind of -credential to generate and how narrowly you can scope it. - -## Want more detail? - -For exact setup steps per provider (Cloudflare, DigitalOcean, Porkbun, -DuckDNS, etc.), see [Installation](installation.html). - -[← Back to Home](index.html) diff --git a/docs/concepts-hosts.md b/docs/concepts-hosts.md deleted file mode 100644 index 504d2e1..0000000 --- a/docs/concepts-hosts.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -layout: default -title: Hosts & HTTPS -description: A plain-language guide to hosts, HTTPS certificates, and wildcards in theta42/proxy. ---- - -# Hosts & HTTPS - -This page explains, in plain language, what a "host" is and how this app -gets you working HTTPS without you having to think about certificates. For -the deeper system-design detail, see [Architecture](architecture.html); for -step-by-step setup, see [Installation](installation.html). - -## What's a "host"? - -A **host** is one entry telling the proxy: "when someone requests *this* -public address, send them to *that* server." For example: requests for -`photos.example.com` get sent to the little box in your closet running your -photo app on port 8080. Each app or service you want to reach from outside -your network — a home automation dashboard, a media server, this proxy's -own management UI — gets its own host entry. - -Two settings on a host are easy to mix up: - -- **Incoming host name** — the public address people type in their - browser (`photos.example.com`). -- **Target IP/port** — where the proxy actually sends the request behind - the scenes (`10.0.0.5:8080`, or a hostname like `photo-server`). - -Everything else on the host form (traffic limits, access rules, -authentication) is optional — a bare host with just those two fields -already works. - -## HTTPS certificates: mostly automatic - -Every public website needs an HTTPS certificate so browsers show the lock -icon instead of a scary warning. This app gets one for you automatically -from [Let's Encrypt](https://letsencrypt.org) the first time a host is -actually requested — you don't manually request, install, or renew -anything for a normal host. This happens behind the scenes using a method -called **HTTP-01**, and it's the default for every new host. - -## Wildcards: one certificate for a whole family of hosts - -Sometimes you want *every* subdomain under one name to work — `app1.`, -`app2.`, `anything.example.com` — without registering each one by hand and -waiting for its own certificate. That's what a **wildcard** host does: a -single host entry named `*.example.com` gets one certificate that covers -the whole family at once. Setting one up needs one extra piece of -information the automatic method above doesn't need — see -[DNS Providers](concepts-dns.html) for why. - -Once a wildcard exists, you have two ways to actually use it: - -- **Register nothing else, and turn on "Match any subdomain"** on the - wildcard host itself — *any* subdomain that doesn't already have its own - entry gets automatically routed to the wildcard's target the first time - it's requested. Convenient, but it means literal typos and random scan - traffic get routed too, not just the subdomains you meant to use. -- **Register each subdomain as its own host, as a "Parent Wildcard" - child** — more setup, but each subdomain can point at a different - target/server while still reusing the one wildcard certificate instead - of getting its own. This is the recommended default and is what - "Match only subdomains defined here" (the host form's default) does. - -You'll see the **"Parent Wildcard"** option light up automatically on the -host form whenever the name you're entering already has a matching -wildcard available to reuse — including the wildcard's own bare base -domain (e.g. `example.com` itself, not just `something.example.com`). - -## Load Balancing - -If you have multiple servers running the same application, you can load balance traffic across them. When editing a host, you can specify **Additional Targets** (one `IP:port` per line). The proxy will automatically distribute incoming requests across your primary target and all additional targets using a round-robin strategy, providing simple high availability and load distribution without extra configuration. - -## Want more detail? - -This page skips the system-internals (Redis, OpenResty, the lookup service) -and the exact install steps. For those, see -[Architecture](architecture.html) and [Installation](installation.html). - -[← Back to Home](index.html) diff --git a/docs/contributing.md b/docs/contributing.md deleted file mode 100644 index 08aa80f..0000000 --- a/docs/contributing.md +++ /dev/null @@ -1,344 +0,0 @@ ---- -layout: default -title: Contributing -description: How to contribute to the proxy — dev setup, tests, and code conventions. ---- - -# Contributing Guide - -[← Back to Home](index.html) - -Thank you for considering contributing to the Proxy project! This guide will help you get started. - -## Development Setup - -### Prerequisites - -- Node.js 18+ (18.x, 20.x, or 22.x recommended) -- Redis server -- Git - -### Local Development - -1. **Clone the repository** - ```bash - git clone https://github.com/theta42/proxy.git - cd proxy/nodejs - ``` - -2. **Install dependencies** - ```bash - npm install - ``` - -3. **Start Redis** (if not already running) - ```bash - redis-server - ``` - -4. **Run in development mode** - ```bash - npm run dev - ``` - - This starts the Node.js API with nodemon for auto-reload on file changes. - -5. **Access the API** - - API: `http://localhost:3000/api` - - Web UI: `http://localhost:3000` - -## Testing - -The project uses Node.js built-in test runner (requires Node 18+). - -### Running Tests - -```bash -# Run all tests -npm test - -# Run only unit tests -npm run test:unit - -# Run only integration tests -npm run test:integration - -# Watch mode for development -npm run test:watch -``` - -### Test Structure - -``` -test/ -├── unit/ # Unit tests for isolated components -│ ├── basicauth.test.js -│ ├── callback_queue.test.js -│ ├── dynamic_record.test.js -│ ├── host_features.test.js -│ ├── host_lookup.test.js -│ ├── hostname_validate.test.js -│ ├── host_sso.test.js -│ ├── oidc.test.js -│ ├── password_policy.test.js -│ ├── roles.test.js -│ ├── safe_redirect.test.js -│ ├── unix_socket.test.js -│ └── wildcard_matchany.test.js -├── integration/ # Integration tests -│ └── dns_provider.test.js -└── helpers/ # Test utilities - └── dns_provider_contract.js -``` - -### Writing Tests - -We test **custom logic**, not third-party libraries: - -**DO test:** -- Host lookup algorithm -- Socket buffering logic -- DNS provider contracts -- Custom utility functions - -**DON'T test:** -- Express.js routing -- Redis ORM -- External DNS APIs (use mocks instead) - -### Adding DNS Provider Tests - -When adding a new DNS provider, you **must** add contract tests: - -```javascript -describe('NewProvider Provider', () => { - const NewProvider = require('../../models/dns_provider/newprovider'); - - test('should meet DNS provider contract', () => { - const mockCredentials = {api_key: 'mock-key'}; - const instance = validateDnsProviderContract(NewProvider, mockCredentials); - assert.ok(instance); - }); - - test('should have valid method signatures', () => { - const instance = new NewProvider({api_key: 'mock'}); - validateMethodSignatures(instance); - }); - - test('should validate key mapping', () => { - const instance = new NewProvider({api_key: 'mock'}); - validateKeyMapping(instance); - }); - - test('should validate type checking', () => { - const instance = new NewProvider({api_key: 'mock'}); - validateTypeChecking(instance); - }); -}); -``` - -See `test/integration/dns_provider.test.js` for examples. - -## Code Style - -### General Guidelines - -- Use strict mode: `'use strict';` -- Use tabs for indentation -- Clear, descriptive variable names -- Comment complex logic -- No trailing whitespace - -### File Organization - -```javascript -'use strict'; - -// 1. Node.js built-ins -const fs = require('fs'); -const path = require('path'); - -// 2. Third-party modules -const express = require('express'); -const redis = require('redis'); - -// 3. Local modules -const {Host} = require('./models'); -const middleware = require('./middleware/auth'); - -// 4. Code... -``` - -### Naming Conventions - -- Classes: `PascalCase` -- Functions: `camelCase` -- Constants: `UPPER_SNAKE_CASE` -- Private methods: `__privateMethod` (double underscore prefix) - -## Project Structure - -Understanding the codebase: - -``` -nodejs/ -├── conf/ # Configuration (base.js, environment overlays, secrets.js) -├── controller/ # App-level wiring (pubsub, startup) -├── migrations/ # One-off Redis data migration scripts -├── models/ # Data models (Host, User, DNS providers) -├── routes/ # API route handlers -├── services/ # Background services (lookup, scheduler) -├── middleware/ # Express middleware -├── utils/ # Utility functions -├── public/ # Static web assets -├── views/ # EJS templates -└── test/ # Test suite -``` - -## Pull Request Process - -### Before Submitting - -1. **Run tests** - Ensure all tests pass - ```bash - npm test - ``` - -2. **Test locally** - Verify your changes work - ```bash - npm run dev - ``` - -3. **Update documentation** - Keep docs in sync with code changes - -4. **Commit messages** - Use clear, descriptive messages - ``` - Add DNS provider for Route53 - - - Implement Route53 DNS API client - - Add contract tests for Route53 - - Update documentation with Route53 setup - ``` - -### Submitting a PR - -1. **Fork the repository** - -2. **Create a feature branch** - ```bash - git checkout -b feature/my-new-feature - ``` - -3. **Make your changes** - -4. **Commit your changes** - ```bash - git add . - git commit -m "Description of changes" - ``` - -5. **Push to your fork** - ```bash - git push origin feature/my-new-feature - ``` - -6. **Open a Pull Request** on GitHub - -### PR Requirements - -- All tests must pass (CI/CD runs automatically) -- Tests run on Node.js 18.x, 20.x, and 22.x -- No merge conflicts with `master` -- Code follows project conventions -- New features include tests -- Documentation updated if needed - -### CI/CD Process - -When you open a PR: -1. GitHub Actions automatically runs tests -2. Tests execute on multiple Node.js versions -3. PR cannot be merged until all checks pass -4. Review from maintainers -5. Merge to master - -## Data Models - -The project uses [model-redis](https://www.npmjs.com/package/model-redis) as the ORM for Redis data storage. All models extend the `Table` class and use a declarative schema via `_keyMap`. - -**Example Model:** -```javascript -const Table = require('../utils/redis_model'); - -class Host extends Table { - static _key = 'host'; // Primary key field - static _keyMap = { - 'host': {isRequired: true, type: 'string', min: 3, max: 500}, - 'ip': {isRequired: true, type: 'string', min: 3, max: 500}, - 'targetPort': {isRequired: true, type: 'number', min: 0, max: 65535}, - 'forcessl': {default: true, type: 'boolean'}, - 'created_on': {default: () => Date.now(), type: 'number'} - }; -} -``` - -**Learn more:** [model-redis documentation](https://www.npmjs.com/package/model-redis) - -## Adding Features - -### Adding a DNS Provider - -1. **Create provider file** in `models/dns_provider/yourprovider.js` - -2. **Extend DnsApi base class** - ```javascript - const {DnsApi} = require('./common'); - - class YourProvider extends DnsApi { - static _keyMap = { - api_key: {isRequired: true, type: 'string', isPrivate: true} - }; - - // Implement required methods - async listDomains() { } - async getRecords(domain, options) { } - async createRecord(domain, options) { } - async deleteRecords(domain, options) { } - } - ``` - -3. **Add to provider list** in `models/dns_provider.js` - -4. **Add contract tests** in `test/integration/dns_provider.test.js` - -5. **Test your provider** - ```bash - npm run test:integration - ``` - -### Adding API Endpoints - -1. **Add route** in appropriate file (`routes/`) -2. **Update API documentation** (`nodejs/api.md` and `docs/api.md` — keep them in sync) -3. **Test the endpoint** manually and add integration tests if needed - -## Getting Help - -- **Questions?** Open a [GitHub Discussion](https://github.com/theta42/proxy/discussions) -- **Bug reports** Use [GitHub Issues](https://github.com/theta42/proxy/issues) -- **Security issues** Email maintainers directly (see package.json) - -## Code of Conduct - -- Be respectful and inclusive -- Focus on constructive feedback -- Help others learn and grow -- Follow the project's technical direction - -## License - -By contributing, you agree that your contributions will be licensed under the MIT License. - ---- - -[← Back to Home](index.html) | [View on GitHub](https://github.com/theta42/proxy) diff --git a/docs/docker.md b/docs/docker.md deleted file mode 100644 index 83b6a3c..0000000 --- a/docs/docker.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -layout: default -title: Docker -description: Running the proxy's all-in-one Docker image — OpenResty, the management app, and Redis in one container. ---- - -# Docker Deployment - -[← Back to Home](index.html) - -The proxy ships as a single all-in-one Docker image bundling **OpenResty + the -Node management app + Redis** in one container, mirroring the bare-metal -[`ops/install.sh`](https://github.com/theta42/proxy/blob/master/ops/install.sh) -layout. This is the easiest way to run the proxy standalone, or as part of the -unified [theta-env](https://github.com/theta42/theta-env) stack. - -## Quick start (standalone) - -```bash -git clone https://github.com/theta42/proxy.git -cd proxy -mkdir -p config && chmod 700 config -cp secrets.js.example config/proxy-secrets.js # set OIDC/LDAP wiring -$EDITOR config/proxy-secrets.js -docker compose up -d --build -``` - -- Proxy (public, auto-SSL): `https:///` -- Management UI / API: `http://127.0.0.1:3000/` (bound to localhost) -- Health: `http://127.0.0.1:3000/health` → `{"status":"ok"}` - -## How configuration works - -The app loads config via [`@simpleworkjs/conf`](https://www.npmjs.com/package/@simpleworkjs/conf), -which deep-merges, in order: - -1. `conf/base.js` (committed defaults) -2. `conf/.js` (optional) -3. `conf/secrets.js` (gitignored) -4. **`app_*` environment variables** — the highest-precedence layer - -The bundled `docker-compose.yml` mounts `./config/proxy-secrets.js` at `/config`, -and `docker-entrypoint.sh` sets `CONF_SECRETS=/config/proxy-secrets.js` so the -app reads the OIDC + LDAP + auth wiring from the file. **No `app_*` env is -passed** — `app_*` env beats `secrets.js`, so the file is authoritative only if -the matching `app_*` env is absent. See `secrets.js.example` for the shape. - -Any env var starting with `app_` overrides the merged config; the rest of the -name splits on **double-underscore** (`__`) into a nested path. Values are -`JSON.parse`-coerced when possible, kept as strings otherwise. `app_*` env is -still supported for advanced/standalone use — add the vars to the compose -`environment:` block yourself (the bundled compose no longer sets them). - -> **Requires `@simpleworkjs/conf` >= 1.1.0.** The `app_*` env layer is not -> honored on 1.0.0. The lock is already on `^1.1.0`. - -### Key `app_*` variables - -| Env var | Sets | -|---------|------| -| `app_oidc__issuer` | `conf.oidc.issuer` (browser-facing SSO URL) | -| `app_oidc__authorizationEndpoint` | `conf.oidc.authorizationEndpoint` | -| `app_oidc__tokenEndpoint` | `conf.oidc.tokenEndpoint` (server-to-server; can be internal) | -| `app_oidc__userinfoEndpoint` | `conf.oidc.userinfoEndpoint` (server-to-server) | -| `app_oidc__endSessionEndpoint` | `conf.oidc.endSessionEndpoint` | -| `app_oidc__clientId` / `app_oidc__clientSecret` | OIDC client creds | -| `app_oidc__redirectUri` | `conf.oidc.redirectUri` (must match the SSO client) | -| `app_oidc__enabled` | `conf.oidc.enabled` (boolean) | -| `app_ldap__url` | `conf.ldap.url` (`ldaps://…:636` or `ldap://…:389`) | -| `app_ldap__bindDN` / `app_ldap__bindPassword` | LDAP service account | -| `app_ldap__searchBase` / `app_ldap__userFilter` | user search | -| `app_ldap__tlsOptions__rejectUnauthorized` | `false` for self-signed LDAPS | -| `app_ldap__tlsOptions__ca` | path to a CA cert for strict trust | -| `app_auth__adminUsers` | local anti-lockout admin (uid) | -| `app_auth__adminGroups` | SSO/LDAP groups that are global admin (JSON array) | -| `app_redis__prefix` | `conf.redis.prefix` (default `proxy_`) | - -See [`DEPLOYMENT.md`](https://github.com/theta42/proxy/blob/master/DEPLOYMENT.md) -for the complete reference. - -## OpenResty runtime env - -| Variable | Default | Description | -|----------|---------|-------------| -| `RESOLVER` | `127.0.0.11` | DNS for upstream names in Host records (Docker's embedded DNS) | -| `REAL_IP_FROM` | _empty_ | Trusted CIDR for `X-Real-IP`. Empty = the proxy is the front (removes the real_ip block). Set to an upstream proxy's CIDR if one sits in front. | - -## Auto-SSL / Let's Encrypt - -`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB -persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs -survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is -required for HTTP-01 challenges (mapped in the compose). Back up + restore Redis -to back up + restore cert state (see *Backups and restore* in `DEPLOYMENT.md`). - -## Fronting an SSO Manager - -The proxy is a natural front for -[`theta42/sso-manager-node`](https://github.com/theta42/sso-manager-node): it -terminates TLS for the SSO's UI and protects it with OIDC login, while also -binding to the SSO's LDAP directly for user lookups. To run both together: - -1. **One Docker network** so the proxy reaches the SSO internally at - `http://sso-manager:3001` (token/userinfo) and `ldaps://sso-manager:636`. -2. **Set the SSO's `app_oauth__issuer`** to the browser-facing HTTPS URL the - proxy serves the SSO at (e.g. `https://sso.example.com`). -3. **Register the proxy as an OIDC client** in the SSO, with `redirectUri` - matching `https://proxy.example.com/api/auth/oidc/callback`. -4. **LDAP**: point `app_ldap__url` at `ldaps://sso-manager:636`, create a - dedicated service account (`cn=ldapclient,ou=people,…`), and for the SSO's - self-signed LDAPS cert set `app_ldap__tlsOptions__rejectUnauthorized=false` - (or mount the cert and use `app_ldap__tlsOptions__ca=`). - -The [`theta42/theta-env`](https://github.com/theta42/theta-env) unified repo -automates all four steps with `./setup.sh` — see -[theta-env docs](https://theta42.github.io/theta-env/). - -## API tokens (personal access tokens) - -Any logged-in user can mint a long-lived bearer token to call the management API -from scripts/CI without an OIDC browser session. Self-service; authenticates as -the creator (groups snapshotted at mint; authz layer unchanged). - -Create one under **API Tokens** in the UI (shown once), then: - -```bash -curl -H "Authorization: Bearer prx__" https://proxy.example.com/api/host -``` - -Rotate/revoke from the same page (immediate effect). Optional expiry at -creation. The token carries the creator's rights (admin → full mgmt API; -domain manager → those domains; `requireAdmin` routes 403). To tighten after -group changes, revoke + re-mint. Tokens persist in Redis (AOF) and survive -rebuilds. - -## Bare metal - -Prefer a systemd install? See the [Installation Guide](installation.html) for -the `ops/install.sh` automated installer on Debian/Ubuntu. - -[← Back to Home](index.html) \ No newline at end of file diff --git a/docs/images/host-auth-basic.png b/docs/images/host-auth-basic.png deleted file mode 100644 index f4d8026..0000000 Binary files a/docs/images/host-auth-basic.png and /dev/null differ diff --git a/docs/images/host-auth-sso.png b/docs/images/host-auth-sso.png deleted file mode 100644 index cccb8ec..0000000 Binary files a/docs/images/host-auth-sso.png and /dev/null differ diff --git a/docs/images/hosts.png b/docs/images/hosts.png deleted file mode 100644 index cb6cf08..0000000 Binary files a/docs/images/hosts.png and /dev/null differ diff --git a/docs/images/load-balancing.png b/docs/images/load-balancing.png deleted file mode 100644 index d21f2b9..0000000 Binary files a/docs/images/load-balancing.png and /dev/null differ diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 1178b61..0000000 --- a/docs/index.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -layout: default -title: Home -description: A reverse proxy and HTTPS termination service built on OpenResty/nginx, with automatic Let's Encrypt certs, OIDC login, and direct LDAP access control per host. ---- - -# Proxy - -A reverse proxy and HTTPS termination service built on OpenResty/nginx, with a -management API and web GUI. It puts any of your apps behind single sign-on -(OIDC) and can also look users up directly in LDAP — so the same people who -log in to your SSO are the people allowed to reach your proxied apps. - -Automatic HTTPS from Let's Encrypt (including wildcards), routing by hostname, -and per-host access control tied to your identity provider — managed from a -web UI or a REST API, with no downtime on config changes. - -Part of the theta42 self-hosted identity stack, alongside -[SSO Manager](https://theta42.github.io/sso-manager-node/) and -[theta-env](https://theta42.github.io/theta-env/) (the two composed with one -command). - -## Screenshots - -Host list -Per-host SSO auth - -Basic auth and SSO are mutually exclusive per host, with per-user password -management once basic auth is enabled: - -Per-host basic auth - -*(click any screenshot to view full size)* - -## Why this over the alternatives - -Nginx Proxy Manager, Traefik, and Caddy are all good reverse proxies with -auto-HTTPS. This one is built around identity: it is both an **OIDC client** -of an SSO provider (for browser login) **and** a direct **LDAP client** (for -user lookups and per-host access control), so access decisions come from your -real user directory, not a static allow-list or a separate auth proxy bolted -on top. The trade-off is that it expects an OIDC/LDAP identity source to point -at — it is not a standalone auth server. Pair it with -[SSO Manager](https://theta42.github.io/sso-manager-node/) (bundled OpenLDAP + -OIDC) for a self-hosted SSO + proxy stack, or point it at any OIDC provider + -LDAP directory you already run. - -## Features - -- Automated HTTPS via Let's Encrypt — HTTP-01 and DNS-01 (wildcard) challenges -- Multiple DNS providers (Cloudflare, DigitalOcean, PorkBun, DuckDNS — free) -- Dynamic host routing with wildcard domain matching (`*`, `**`) -- **Multi-target load balancing** — configure multiple backend targets per host with built-in round-robin load balancing -- **OIDC login** and **direct LDAP lookups**, independently of each other -- Per-host **basic auth** as an alternative to SSO (mutually exclusive, so - it's never ambiguous which one gated a request) -- **Role-based access control** — global admins, local groups, and - per-domain permissions (viewer/manager) -- Self-service API tokens for scripting/CI without a browser session -- Web UI and a full REST API - -## Get it - -```bash -git clone https://github.com/theta42/proxy.git -cd proxy && docker compose up -d --build -``` - -That's the standalone quick start. For the full set of install options (Docker, -bare-metal, or as part of the combined SSO + proxy stack), configuration -reference, and API docs, see the -**[GitHub repository](https://github.com/theta42/proxy)**. - -## Related projects - -- **[SSO Manager](https://theta42.github.io/sso-manager-node/)** — the OIDC - provider + LDAP directory this proxy is designed to sit in front of. -- **[theta-env](https://theta42.github.io/theta-env/)** — runs this proxy and - SSO Manager together with one command. diff --git a/docs/installation.md b/docs/installation.md deleted file mode 100644 index 7fbacdf..0000000 --- a/docs/installation.md +++ /dev/null @@ -1,283 +0,0 @@ ---- -layout: default -title: Installation -description: Installing the proxy — Docker, bare metal, or as part of the unified theta-env stack. ---- - -# Installation Guide - -[← Back to Home](index.html) - -> Looking for a plainer explanation of hosts, HTTPS, and DNS providers -> instead of install steps? See [Hosts & HTTPS](concepts-hosts.html) and -> [DNS Providers](concepts-dns.html). - -## Quick Install (Recommended) - -For modern Debian-based systems (Ubuntu 20.04+, Debian 11+): - -```bash -wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash -``` - -This automated installer will: -- Install Node.js 22.x -- Install OpenResty and required dependencies -- Install and configure Redis -- Set up SSL fallback certificates -- Install Lua dependencies -- Clone and install the proxy application -- Configure systemd service -- Start the proxy service - -## Manual Installation - -> **Recommended path:** `ops/install.sh` (above) is idempotent and safe to -> re-run — it symlinks the OpenResty/systemd config from the repo checkout, -> so future updates stay in sync automatically (`git pull` + re-run). The -> manual steps below *copy* those same files instead of symlinking them, so -> they will **not** auto-track later changes to `ops/nginx_conf/` or -> `ops/proxy.service` — you'd need to re-copy them by hand after every -> update. Prefer the manual path only if `install.sh` doesn't fit your -> distribution. - -### System Requirements - -- Modern Linux distribution (Ubuntu 20.04+, Debian 11+, or equivalent) -- Root access -- Inbound internet access for Let's Encrypt validation -- Minimum 1GB RAM, 10GB disk space - -### Step 1: Install Dependencies - -**Ubuntu/Debian:** -```bash -apt install libpam0g-dev build-essential redis-server luarocks -y -``` - -### Step 2: Install Node.js 22.x - -```bash -curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \ - sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg - -NODE_MAJOR=22 -echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | \ - sudo tee /etc/apt/sources.list.d/nodesource.list - -apt update && apt install nodejs -y -``` - -Verify installation: -```bash -node --version # Should show v22.x.x -npm --version -``` - -### Step 3: Install OpenResty - -openresty.org ships distinct trees (and components) for Debian and Ubuntu. The -Debian tree is published only up to **bookworm** (there is no trixie block) and -uses the **`openresty`** component; Ubuntu uses the host codename and **`main`**. -So on a Debian 13 (trixie) host, point at the `bookworm` distribution (binary- -compatible, same OpenSSL 3 era). - -```bash -. /etc/os-release -CODENAME="$(lsb_release -sc)" -case "$ID" in - debian) - OR_PATH=package/debian - OR_COMPONENT=openresty - # Debian tree only publishes up to bookworm; fall back to it for trixie+. - case "$CODENAME" in jessie|stretch|buster|bullseye|bookworm) OR_DISTRO="$CODENAME";; *) OR_DISTRO=bookworm;; esac - ;; - *) - OR_PATH=package/ubuntu - OR_DISTRO="$CODENAME" - OR_COMPONENT=main - ;; -esac - -wget -O - https://openresty.org/package/pubkey.gpg | \ - sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg - -echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/${OR_PATH} ${OR_DISTRO} ${OR_COMPONENT}" | \ - sudo tee /etc/apt/sources.list.d/openresty.list -``` - -> **Debian 13 (trixie):** apt's sequoia GPG backend rejects SHA-1 signatures by -> default, and the OpenResty signing key is still SHA-1, so `apt update` will -> refuse the repo. Extend the SHA-1 acceptance window before updating: -> ```bash -> sudo mkdir -p /etc/crypto-policies/back-ends -> sudo cp /usr/share/apt/default-sequoia.config /etc/crypto-policies/back-ends/apt-sequoia.config -> sudo sed -i 's/2026-02-01/2028-02-01/' /etc/crypto-policies/back-ends/apt-sequoia.config -> ``` -> (The `default-sequoia.config` file only ships on Debian 13+, so this is a no-op -> on older releases. `ops/install.sh` applies this automatically.) - -```bash -apt update && apt install openresty -y -``` - -### Step 4: Install Lua Dependencies - -```bash -luarocks install lua-resty-auto-ssl -luarocks install luasocket -``` - -### Step 5: SSL Configuration - -Create fallback SSL certificates: - -```bash -mkdir -p /etc/ssl/ - -openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \ - -subj '/CN=sni-support-required-for-valid-ssl' \ - -keyout /etc/ssl/resty-auto-ssl-fallback.key \ - -out /etc/ssl/resty-auto-ssl-fallback.crt -``` - -### Step 6: Configure OpenResty - -Clone the repository and copy configuration files: - -```bash -mkdir -p /opt/theta42 -cd /opt/theta42 -git clone https://github.com/theta42/proxy.git -cd proxy - -# Copy nginx configs -mkdir -p /etc/openresty/sites-enabled/ -cp ops/nginx_conf/nginx.conf /etc/openresty/nginx.conf -cp ops/nginx_conf/autossl.conf /etc/openresty/autossl.conf -cp ops/nginx_conf/proxy.conf /etc/openresty/sites-enabled/000-proxy -cp ops/nginx_conf/targetinfo.lua /usr/local/openresty/lualib/targetinfo.lua -``` - -### Step 7: Install Application - -```bash -cd /opt/theta42/proxy/nodejs -npm install -``` - -### Step 7b: Configure Secrets - -```bash -mkdir -p /etc/proxy -cp /opt/theta42/proxy/secrets.js.example /etc/proxy/secrets.js -chmod 600 /etc/proxy/secrets.js -$EDITOR /etc/proxy/secrets.js # set oidc.clientId/clientSecret, ldap.bindPassword, ... -``` - -`@simpleworkjs/conf` reads this file via the `CONF_SECRETS` env var, which the -systemd unit below sets to `/etc/proxy/secrets.js`. - -### Step 8: Configure Systemd Service - -```bash -cp /opt/theta42/proxy/ops/proxy.service /etc/systemd/system/proxy.service -systemctl daemon-reload -systemctl enable proxy.service -systemctl start proxy.service -``` - -Verify service is running: -```bash -systemctl status proxy.service -``` - -### Step 9: Initial Setup - -The proxy API will be available on port 3000 by default. You'll need to: - -1. Create your first user account -2. Configure DNS providers (for wildcard SSL) -3. Add your first host - -See the [API Reference](api.html) for details. - -## Configuration - -### Environment Variables - -- `NODE_ENV` - Set to `production` for production deployments -- `NODE_PORT` - Override default port (default: 3000) - -### Redis Configuration - -The proxy uses Redis with the prefix `proxy_`. To change this, edit `nodejs/conf/base.js`: - -```javascript -redis: { - prefix: 'proxy_' -} -``` - -### OpenResty Configuration - -Key configuration files in `/etc/openresty/`: -- `nginx.conf` - Main nginx configuration -- `autossl.conf` - Let's Encrypt HTTP-01 challenge handler -- `sites-enabled/000-proxy` - Proxy server configuration - -### Unix Socket - -The proxy communicates with OpenResty via Unix socket at: -``` -/var/run/proxy_lookup.socket -``` - -This path is configurable in `nodejs/conf/base.js`. - -## Troubleshooting - -### Service won't start - -Check logs: -```bash -journalctl -u proxy.service -f -``` - -Common issues: -- Port 3000 already in use -- Redis not running: `systemctl status redis-server` -- Permission issues: Service must run as root for user management - -### SSL certificates not working - -Check OpenResty logs: -```bash -tail -f /var/log/nginx/error.log -``` - -Common issues: -- Firewall blocking ports 80/443 -- DNS not pointing to server -- Let's Encrypt rate limits exceeded - -### Host lookup not working - -Check Unix socket: -```bash -ls -la /var/run/proxy_lookup.socket -# Should show srwxrwxrwx (socket permissions) -``` - -Test lookup: -```bash -echo '{"domain":"example.com"}' | nc -U /var/run/proxy_lookup.socket -``` - -## Next Steps - -- [Configure DNS Providers](api.html#dns-providers) for wildcard SSL -- [Add your first host](api.html#hosts) -- [Set up the web interface](index.html) - -[← Back to Home](index.html) diff --git a/docs/robots.txt b/docs/robots.txt deleted file mode 100644 index bd503ab..0000000 --- a/docs/robots.txt +++ /dev/null @@ -1,4 +0,0 @@ -User-agent: * -Allow: / - -Sitemap: https://theta42.github.io/proxy/sitemap.xml diff --git a/nodejs/package-lock.json b/nodejs/package-lock.json index af08a07..c983787 100644 --- a/nodejs/package-lock.json +++ b/nodejs/package-lock.json @@ -1,12 +1,12 @@ { "name": "proxy-api", - "version": "1.13.2", + "version": "1.13.3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "proxy-api", - "version": "1.13.2", + "version": "1.13.3", "license": "MIT", "dependencies": { "@fortawesome/fontawesome-free": "^7.3.0", diff --git a/nodejs/package.json b/nodejs/package.json index 5cee527..5e90cc3 100644 --- a/nodejs/package.json +++ b/nodejs/package.json @@ -1,6 +1,6 @@ { "name": "proxy-api", - "version": "1.13.2", + "version": "1.13.3", "author": [ { "name": "William Mantly",