diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md index 7a15589..1ed85bc 100644 --- a/DEPLOYMENT.md +++ b/DEPLOYMENT.md @@ -59,14 +59,26 @@ in the foreground under `dumb-init`. ### Setup +The bundled `docker-compose.yml` reads the OIDC + LDAP + auth wiring from a +bind-mounted `./config/proxy-secrets.js` (not from `app_*` env). Copy the +example, fill in your secrets, then build + start: + ```bash -# Minimal: set the OIDC + LDAP wiring, then build + start. -OIDC_CLIENT_ID=... OIDC_CLIENT_SECRET=... LDAP_BIND_PASSWORD=... \ +mkdir -p config && chmod 700 config +cp secrets.js.example config/proxy-secrets.js +$EDITOR config/proxy-secrets.js # set oidc.clientId/clientSecret, ldap.bindPassword, ... docker compose up -d --build ``` -For a real deployment, put the overrides in a `.env` (or `environment:` in the -compose). See `docker-compose.yml` for the full set. +`docker-entrypoint.sh` symlinks `/config/proxy-secrets.js` → `/app/conf/secrets.js` +so `@simpleworkjs/conf` reads it. No `app_*` env is passed — `app_*` env would +override the file (env beats secrets.js in `@simpleworkjs/conf`), so the file is +kept authoritative. `RESOLVER` / `REAL_IP_FROM` / `NODE_ENV` / `NODE_PORT` are +OpenResty-runtime / process env, not `app_*` config, so they stay in the compose. + +> Running the unified `theta-env` stack? Its `setup.sh` generates +> `./config/proxy-secrets.js` (+ `./config/sso-secrets.js`) for you and +> registers the OAuth client with the SSO — see the theta-env README. ### Access @@ -84,10 +96,72 @@ compose). See `docker-compose.yml` for the full set. ### Auto-SSL / Let's Encrypt -`lua-resty-auto-ssl` stores certs in the bundled Redis (in-memory by default — -lost on container recreation). For cert persistence, enable Redis persistence -in `docker-entrypoint.sh` or mount a redis AOF/RDB volume. Port 80 is required -for HTTP-01 challenges (mapped in the compose). +`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB +persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs +survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is +required for HTTP-01 challenges (mapped in the compose). + +### Backups and restore + +**What lives where** + +| State | Location | Persisted? | +|-------|----------|------------| +| Host records, permissions, DNS creds, local users | `proxy-data` volume (`/data`, Redis) | yes (AOF + RDB) | +| Let's Encrypt certs (auto-ssl) | `proxy-data` volume (`/data`, Redis) | yes — same Redis | +| nginx response cache / logs | `proxy-cache` / `proxy-logs` volumes | yes (volume) | +| Secrets (OIDC client secret, LDAP bind password) | `./config/proxy-secrets.js` (bind mount) | your responsibility — back up off-host | + +**Automatic snapshots** — when run as part of the unified `theta-env` stack, +`setup.sh` snapshots Redis + `./config/` to `./backups//` before every +rebuild and keeps the last `BACKUP_KEEP` (default 5). Standalone deployments use +the manual steps below. + +**Manual backup** + +```bash +# Redis — hot snapshot: trigger a save, then copy the RDB out +docker compose exec proxy redis-cli BGSAVE +docker compose cp proxy:/data/dump.rdb proxy-redis-$(date +%F).rdb + +# Secrets — copy the config dir (holds OIDC client secret, LDAP bind password) +cp -a ./config config-backup-$(date +%F) && chmod 700 config-backup-$(date +%F) +``` +Store the `.rdb` and config copy **off the host** — they contain secrets and +the whole Host/permission/user dataset. + +**Restore — Redis (full proxy state + certs)** + +```bash +cp -a config-backup- ./config && chmod 700 ./config +docker compose up -d +docker compose stop proxy +# AOF wins on startup — delete it so the RDB is loaded instead (see note). +docker compose run --rm --no-deps --entrypoint sh proxy -c \ + 'rm -f /data/appendonly.aof /data/appendonly.aof.*' +docker compose cp proxy-redis-.rdb proxy:/data/dump.rdb +docker compose start proxy +``` + +> **AOF vs RDB (important):** with `--appendonly yes`, Redis loads +> `appendonly.aof` on startup and **ignores** `dump.rdb` if the AOF exists. To +> restore from an RDB snapshot you **must delete the AOF first** (the runbook +> does this); Redis then loads the RDB and writes a fresh AOF. Verify: +> `docker compose exec proxy redis-cli DBSIZE`. +> +> Restoring Redis restores cert state **at the snapshot time** — certs issued +> after the snapshot are lost and will be re-issued on next request. + +**Upgrades** + +```bash +./setup.sh # backs up, then rebuilds — proxy-data keeps Redis state +# (standalone) docker compose pull && docker compose up -d +``` +Host records, permissions, DNS creds, local users, and Let's Encrypt certs all +survive the rebuild because they live on the `proxy-data` volume, not in the +image. **Migrations note:** if a release ships a `nodejs/migrations/` script, +run it after upgrading (it transforms in-Redis records); see the release notes. ### Logs diff --git a/docker-compose.yml b/docker-compose.yml index 8618cc8..704aaf7 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,15 +2,19 @@ # (OpenResty + Node management app + Redis in one container). # # The proxy is an OIDC client of an SSO Manager (or any OIDC provider) AND a -# direct LDAP client for user lookups. Supply that wiring via `app_*` -# environment variables — the highest-precedence config layer in -# @simpleworkjs/conf (>= 1.1.0, pinned in nodejs/package-lock.json). No -# secrets.js is baked in; set the values here, in a .env file, or via an -# env_file (the theta42/theta-env unified repo generates one with setup.sh). +# direct LDAP client for user lookups. That wiring (oidc/ldap/auth) is read from +# a bind-mounted ./config/proxy-secrets.js — docker-entrypoint.sh symlinks it +# into /app/conf/secrets.js so @simpleworkjs/conf reads it. No app_* env is +# passed here: app_* env beats secrets.js in @simpleworkjs/conf (precedence: +# base.js < .js < secrets.js < app_* env), so the file must be the only +# source. See secrets.js.example for the shape. # -# Requires @simpleworkjs/conf >= 1.1.0 in the image (env overrides). The lock -# is already on ^1.1.0; rebuild with `docker compose up -d --build` after any -# package change. +# Compose only interpolates the port + OpenResty-runtime defaults below — there +# is no .env file. Override on the command line if needed: +# HTTP_PORT=8080 HTTPS_PORT=8443 docker compose up -d +# +# Requires @simpleworkjs/conf >= 1.1.0 in the image. The lock is already on +# ^1.1.0; rebuild with `docker compose up -d --build` after any package change. services: proxy: @@ -29,51 +33,26 @@ services: # LAN — the OpenResty front proxies the UI/api under its own TLS. - "127.0.0.1:${MGMT_PORT:-3000}:3000" environment: - # ── OpenResty runtime (see docker-entrypoint.sh) ── + # OpenResty runtime (see docker-entrypoint.sh). These are NOT app_* config + # keys, so they don't conflict with secrets.js. oidc/ldap/auth config comes + # from ./config/proxy-secrets.js, not from env. # Resolver for upstream names in Host records (default = Docker DNS). - RESOLVER=${RESOLVER:-127.0.0.11} # Trusted range for X-Real-IP. Empty = proxy is the front (default, # removes the real_ip block). Set to an upstream proxy's CIDR if one # sits in front and sets X-Real-IP. - REAL_IP_FROM=${REAL_IP_FROM:-} - - # ── OIDC client config (app_oidc__*) ── - # Point at your SSO Manager. Issuer + authorization/endSession are the - # browser-facing URLs; token/userinfo can be the internal URL if the SSO - # is on the same docker network (avoids a TLS hairpin through the proxy). - - app_oidc__enabled=${OIDC_ENABLED:-true} - - app_oidc__issuer=${OIDC_ISSUER:-https://sso.example.com} - - app_oidc__authorizationEndpoint=${OIDC_AUTHORIZATION_ENDPOINT:-https://sso.example.com/oauth/authorize} - - app_oidc__tokenEndpoint=${OIDC_TOKEN_ENDPOINT:-http://sso-manager:3001/oauth/token} - - app_oidc__userinfoEndpoint=${OIDC_USERINFO_ENDPOINT:-http://sso-manager:3001/oauth/userinfo} - - app_oidc__endSessionEndpoint=${OIDC_ENDSESSION_ENDPOINT:-https://sso.example.com/oauth/logout} - - app_oidc__clientId=${OIDC_CLIENT_ID:-} - - app_oidc__clientSecret=${OIDC_CLIENT_SECRET:-} - - app_oidc__redirectUri=${OIDC_REDIRECT_URI:-https://proxy.example.com/api/auth/oidc/callback} - - # ── LDAP client config (app_ldap__*) ── - # Direct user lookups. ldaps:// + rejectUnauthorized=false for a - # self-signed cert, or set app_ldap__tlsOptions__ca= for strict. - - app_ldap__url=${LDAP_URL:-ldaps://sso-manager:636} - - app_ldap__bindDN=${LDAP_BIND_DN:-cn=ldapclient,ou=people,dc=example,dc=com} - - app_ldap__bindPassword=${LDAP_BIND_PASSWORD:-} - - app_ldap__searchBase=${LDAP_SEARCH_BASE:-ou=people,dc=example,dc=com} - - app_ldap__userFilter=${LDAP_USER_FILTER:-(objectClass=inetOrgPerson)} - - app_ldap__tlsOptions__rejectUnauthorized=${LDAP_REJECT_UNAUTHORIZED:-false} - - # ── Authorization ── - # Local anti-lockout admin (matches auth.adminUsers in conf/base.js). - - app_auth__adminUsers=${AUTH_ADMIN_USERS:-proxyadmin} - - NODE_ENV=production - NODE_PORT=3000 volumes: - # Let's Encrypt cert store + auto-ssl redis data live with the bundled - # redis (in-container, in-memory). Mount these to persist across recreation: - # proxy-cache -> /var/cache/nginx/proxy (response cache) - # proxy-logs -> /var/log/nginx (access/error logs) - # Auto-ssl certs are in the bundled redis (in-memory, lost on recreation) - # unless you enable redis persistence in docker-entrypoint.sh. + # Operator-edited secrets (proxy-secrets.js). The entrypoint symlinks + # /config/proxy-secrets.js -> /app/conf/secrets.js so @simpleworkjs/conf + # reads the oidc/ldap/auth config. See secrets.js.example for the shape. + - ./config:/config:ro + # Persist Redis (AOF + RDB) so Host records, permissions, DNS creds, local + # users, AND the lua-resty-auto-ssl Let's Encrypt certs survive container + # recreation. Restoring Redis also restores cert state at snapshot time. + - proxy-data:/data - proxy-cache:/var/cache/nginx/proxy - proxy-logs:/var/log/nginx healthcheck: @@ -85,4 +64,5 @@ services: volumes: proxy-cache: - proxy-logs: \ No newline at end of file + proxy-logs: + proxy-data: \ No newline at end of file diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh index f27c341..a12c72f 100755 --- a/docker-entrypoint.sh +++ b/docker-entrypoint.sh @@ -9,10 +9,13 @@ # 3. OpenResty (80/443/4443) — exec'd in the foreground as PID 2 (under # dumb-init, PID 1) so it receives SIGTERM from `docker stop`. # -# The app reads its config from conf/base.js deep-merged with `app_*` env vars -# (requires @simpleworkjs/conf >= 1.1.0, pinned in nodejs/package-lock.json). No -# secrets.js is baked into the image — supply oidc/ldap/auth config via `app_*` -# env (compose `environment:` / `env_file:`), or mount a conf/secrets.js. +# The app reads its config from conf/base.js deep-merged with conf/secrets.js +# and `app_*` env vars (requires @simpleworkjs/conf >= 1.1.0, pinned in +# nodejs/package-lock.json). No secrets.js is baked into the image. The unified +# theta-env stack mounts ./config/proxy-secrets.js at /config; this entrypoint +# symlinks it into /app/conf/secrets.js so the app reads oidc/ldap/auth config +# from the file (no app_* env needed). Without the mount, supply the same config +# via `app_*` env (compose `environment:` / `env_file:`). # # OpenResty config: the committed ops/nginx_conf/*.conf carry the bare-metal # home-LAN values (set_real_ip_from 192.168.1.0/24; resolver 192.168.1.1). They @@ -25,6 +28,19 @@ set -e info() { echo "[INFO] $*"; } error() { echo "[ERROR] $*" >&2; } +# ── Optional: mount proxy secrets.js ───────────────────────────────────────── +# When /config/proxy-secrets.js is present (unified theta-env stack, or any +# deployment that bind-mounts ./config), symlink it into /app/conf/secrets.js so +# @simpleworkjs/conf reads the oidc/ldap/auth config from the file. No app_* env +# should then be passed — app_* env beats secrets.js in @simpleworkjs/conf +# (precedence: base.js < .js < secrets.js < app_* env), so the file is +# authoritative only if the matching app_* env is absent. When the file is +# absent the app falls back to app_* env (compose environment / env_file). +if [[ -f /config/proxy-secrets.js ]]; then + ln -sf /config/proxy-secrets.js /app/conf/secrets.js + info "Loaded config from /config/proxy-secrets.js (secrets.js authoritative)" +fi + # ── Fallback SSL cert for lua-resty-auto-ssl ───────────────────────────────── # autossl.conf references /etc/ssl/resty-auto-ssl-fallback.{crt,key} — auto-ssl # serves this for unknown SNI before a real Let's Encrypt cert is issued. @@ -77,14 +93,21 @@ if ! openresty -t >/dev/null 2>&1; then fi # ── Redis ─────────────────────────────────────────────────────────────────── -# In-memory, no persistence (cache/session/cert data only). The app, -# auto-ssl, and targetinfo.lua all reach it at 127.0.0.1:6379 (the redis client -# default + the one literal in targetinfo.lua). Run in the background -# (--daemonize no, backgrounded by the shell); the container's lifecycle is -# owned by OpenResty (exec'd below), and `restart: unless-stopped` in compose -# handles full restarts. -info "Starting Redis..." -redis-server --save "" --appendonly no --daemonize no & +# Persisted to /data (AOF + RDB) so Host records, permissions, DNS creds, local +# users, AND the lua-resty-auto-ssl Let's Encrypt certs all survive container +# recreation (persisting Redis persists the cert store — avoids LE re-issue / +# rate limits on rebuild). The app, auto-ssl, and targetinfo.lua all reach it at +# 127.0.0.1:6379 (the redis client default + the one literal in targetinfo.lua). +# Run in the background (--daemonize no, backgrounded by the shell); the +# container's lifecycle is owned by OpenResty (exec'd below), and +# `restart: unless-stopped` in compose handles full restarts. +REDIS_DATA_DIR="${REDIS_DATA_DIR:-/data}" +mkdir -p "$REDIS_DATA_DIR" +chmod 700 "$REDIS_DATA_DIR" +info "Starting Redis (AOF persisted to $REDIS_DATA_DIR)..." +redis-server --daemonize no --dir "$REDIS_DATA_DIR" --appendonly yes \ + --appendfilename appendonly.aof --save 900 1 --save 300 10 --save 60 10000 \ + --dbfilename dump.rdb & REDIS_PID=$! for i in $(seq 1 15); do if redis-cli ping >/dev/null 2>&1; then info "Redis is ready"; break; fi diff --git a/docs/docker.md b/docs/docker.md index f1f562d..7ecfa6f 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -18,7 +18,9 @@ unified [theta-env](https://github.com/theta42/theta-env) stack. ```bash git clone https://github.com/theta42/proxy.git cd proxy -cp .env.example .env # optional: set OIDC/LDAP wiring + ports +mkdir -p config && chmod 700 config +cp secrets.js.example config/proxy-secrets.js # set OIDC/LDAP wiring +$EDITOR config/proxy-secrets.js docker compose up -d --build ``` @@ -36,9 +38,17 @@ which deep-merges, in order: 3. `conf/secrets.js` (gitignored) 4. **`app_*` environment variables** — the highest-precedence layer +The bundled `docker-compose.yml` mount `./config/proxy-secrets.js` at `/config`, +and `docker-entrypoint.sh` symlinks it into `/app/conf/secrets.js` so the app +reads the OIDC + LDAP + auth wiring from the file. **No `app_*` env is passed** — +`app_*` env beats `secrets.js`, so the file is authoritative only if the matching +`app_*` env is absent. See `secrets.js.example` for the shape. + Any env var starting with `app_` overrides the merged config; the rest of the name splits on **double-underscore** (`__`) into a nested path. Values are -`JSON.parse`-coerced when possible, kept as strings otherwise. +`JSON.parse`-coerced when possible, kept as strings otherwise. `app_*` env is +still supported for advanced/standalone use — add the vars to the compose +`environment:` block yourself (the bundled compose no longer sets them). > **Requires `@simpleworkjs/conf` >= 1.1.0.** The `app_*` env layer is not > honored on 1.0.0. The lock is already on `^1.1.0`. @@ -76,9 +86,11 @@ for the complete reference. ## Auto-SSL / Let's Encrypt -`lua-resty-auto-ssl` stores certs in the bundled Redis (in-memory by default — -lost on container recreation). For cert persistence, mount a Redis AOF/RDB -volume. Port 80 is required for HTTP-01 challenges (mapped in the compose). +`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB +persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs +survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is +required for HTTP-01 challenges (mapped in the compose). Back up + restore Redis +to back up + restore cert state (see *Backups and restore* in `DEPLOYMENT.md`). ## Fronting an SSO Manager diff --git a/secrets.js.example b/secrets.js.example new file mode 100644 index 0000000..fcbff7e --- /dev/null +++ b/secrets.js.example @@ -0,0 +1,72 @@ +'use strict'; + +// Example secrets configuration for the theta42/proxy. +// +// The proxy is an OIDC client of an SSO Manager (or any OIDC provider) AND a +// direct LDAP client for user lookups. This file supplies that wiring. +// +// Docker / unified stack: place at ./config/proxy-secrets.js and bind-mount +// ./config at /config (see docker-compose.yml); docker-entrypoint.sh symlinks +// it into /app/conf/secrets.js so @simpleworkjs/conf reads it. No app_* env +// should be passed — app_* env beats this file in @simpleworkjs/conf, so the +// file is authoritative only if the matching app_* env is absent. +// +// Bare-metal: copy to nodejs/conf/secrets.js and fill in your values. Values +// here override conf/base.js and win over .js. +// +// Only the keys the app reads are listed below. The `stack` key is read by the +// theta-env orchestrator (setup.sh) and ignored by the app. + +module.exports = { + // OpenID Connect — point at your SSO Manager. Issuer + authorization/ + // endSession are browser-facing URLs; token/userinfo can be the internal + // URL if the SSO is on the same docker network (avoids a TLS hairpin). + oidc: { + enabled: true, + issuer: 'https://sso.example.com', + authorizationEndpoint: 'https://sso.example.com/oauth/authorize', + tokenEndpoint: 'http://sso-manager:3001/oauth/token', + userinfoEndpoint: 'http://sso-manager:3001/oauth/userinfo', + endSessionEndpoint: 'https://sso.example.com/oauth/logout', + clientId: 'set-me', // registered on the SSO + clientSecret: 'set-me', // from the SSO client record + redirectUri: 'https://proxy.example.com/api/auth/oidc/callback', + scopes: ['openid', 'profile', 'email', 'groups'], + groupsClaim: 'groups', + usernameClaim: 'preferred_username', + }, + + // Direct LDAP user lookups. ldaps:// + rejectUnauthorized:false for a + // self-signed cert (the SSO's default), or set tlsOptions.ca to a CA path + // for strict verification. bindPassword MUST match the + // serviceAccountPass in the SSO's sso-secrets.js (the proxy binds as that + // service account). + ldap: { + url: 'ldaps://sso-manager:636', + bindDN: 'cn=ldapclient,ou=people,dc=example,dc=com', + bindPassword: 'set-me', + searchBase: 'ou=people,dc=example,dc=com', + userFilter: '(objectClass=inetOrgPerson)', + userNameAttribute: 'uid', + tlsOptions: { + rejectUnauthorized: false, // true + ca for a CA-signed cert + }, + }, + + // Authorization. adminUsers is the local anti-lockout admin (matches + // auth.adminUsers in conf/base.js). adminGroups: SSO/LDAP groups whose + // members are always global admins. + auth: { + adminGroups: [], + adminUsers: ['proxyadmin'], + groupRoleMap: {}, + }, + + // ── Orchestrator-only (ignored by the app) ─────────────────────────────── + // Read by the theta-env setup.sh (e.g. to seed the OAuth client). Omit for + // bare-metal use. + stack: { + ssoHost: 'sso.example.com', // public SSO hostname + proxyHost: 'proxy.example.com', // public proxy hostname + }, +}; \ No newline at end of file