Persist Redis (AOF+vol), read config from mounted secrets.js, add backup/restore docs (#118)

Lossless upgrades + config story for the all-in-one proxy image.

Redis persistence (Part A):
- Replace in-memory `--save "" --appendonly no` with AOF + RDB persisted to
  /data. Host records, permissions, DNS creds, local users, AND the
  lua-resty-auto-ssl Let's Encrypt certs now all survive container recreation
  (persisting Redis persists the cert store — no LE re-issue / rate-limit on
  rebuild).
- Add the `proxy-data` named volume -> /data in docker-compose.yml; fix the
  stale "in-memory, lost on recreation" comment.

Config from ./config/proxy-secrets.js (Part B):
- docker-entrypoint.sh: when /config/proxy-secrets.js is mounted, symlink it to
  /app/conf/secrets.js so @simpleworkjs/conf reads the oidc/ldap/auth config
  from the file. No app_* env should then be passed (app_* beats secrets.js).
  Falls back to app_* env when the file is absent (standalone still works).
- docker-compose.yml: drop all app_oidc__* / app_ldap__* / app_auth__* env and
  add `./config:/config:ro`. Keep RESOLVER/REAL_IP_FROM/NODE_ENV/NODE_PORT
  (OpenResty-runtime / process env, not app_* config). No env_file.
- New secrets.js.example (the proxy had none): oidc (enabled, endpoints,
  clientId/clientSecret, redirectUri, scopes, claims), ldap (url, bindDN,
  bindPassword, searchBase, userFilter, tlsOptions), auth (adminGroups,
  adminUsers, groupRoleMap), plus an orchestrator-only `stack` key.

Backup/restore docs:
- Full "Backups and restore" runbook in DEPLOYMENT.md (what lives where, manual
  backup, Redis restore with the AOF-vs-RDB note — AOF wins on startup so the
  AOF must be deleted before an RDB load; restoring Redis restores cert state
  at snapshot time; migrations note). Update the Setup + Auto-SSL sections.
- docs/docker.md: update Quick start + How configuration works + Auto-SSL for
  the new ./config/ approach (app_* env now advanced/optional).

Co-authored-by: Claude <noreply@anthropic.com>

DEPLOYMENT.md

@@ -59,14 +59,26 @@ in the foreground under `dumb-init`.
 
 ### Setup
 
+The bundled `docker-compose.yml` reads the OIDC + LDAP + auth wiring from a
+bind-mounted `./config/proxy-secrets.js` (not from `app_*` env). Copy the
+example, fill in your secrets, then build + start:
+
 ```bash
-# Minimal: set the OIDC + LDAP wiring, then build + start.
+mkdir -p config && chmod 700 config
-OIDC_CLIENT_ID=... OIDC_CLIENT_SECRET=... LDAP_BIND_PASSWORD=... \
+cp secrets.js.example config/proxy-secrets.js
+$EDITOR config/proxy-secrets.js     # set oidc.clientId/clientSecret, ldap.bindPassword, ...
 docker compose up -d --build
 ```
 
-For a real deployment, put the overrides in a `.env` (or `environment:` in the
+`docker-entrypoint.sh` symlinks `/config/proxy-secrets.js` → `/app/conf/secrets.js`
-compose). See `docker-compose.yml` for the full set.
+so `@simpleworkjs/conf` reads it. No `app_*` env is passed — `app_*` env would
+override the file (env beats secrets.js in `@simpleworkjs/conf`), so the file is
+kept authoritative. `RESOLVER` / `REAL_IP_FROM` / `NODE_ENV` / `NODE_PORT` are
+OpenResty-runtime / process env, not `app_*` config, so they stay in the compose.
+
+> Running the unified `theta-env` stack? Its `setup.sh` generates
+> `./config/proxy-secrets.js` (+ `./config/sso-secrets.js`) for you and
+> registers the OAuth client with the SSO — see the theta-env README.
 
 ### Access
 
@@ -84,10 +96,72 @@ compose). See `docker-compose.yml` for the full set.
 
 ### Auto-SSL / Let's Encrypt
 
-`lua-resty-auto-ssl` stores certs in the bundled Redis (in-memory by default —
+`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB
-lost on container recreation). For cert persistence, enable Redis persistence
+persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs
-in `docker-entrypoint.sh` or mount a redis AOF/RDB volume. Port 80 is required
+survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is
-for HTTP-01 challenges (mapped in the compose).
+required for HTTP-01 challenges (mapped in the compose).
+
+### Backups and restore
+
+**What lives where**
+
+| State | Location | Persisted? |
+|-------|----------|------------|
+| Host records, permissions, DNS creds, local users | `proxy-data` volume (`/data`, Redis) | yes (AOF + RDB) |
+| Let's Encrypt certs (auto-ssl) | `proxy-data` volume (`/data`, Redis) | yes — same Redis |
+| nginx response cache / logs | `proxy-cache` / `proxy-logs` volumes | yes (volume) |
+| Secrets (OIDC client secret, LDAP bind password) | `./config/proxy-secrets.js` (bind mount) | your responsibility — back up off-host |
+
+**Automatic snapshots** — when run as part of the unified `theta-env` stack,
+`setup.sh` snapshots Redis + `./config/` to `./backups/<timestamp>/` before every
+rebuild and keeps the last `BACKUP_KEEP` (default 5). Standalone deployments use
+the manual steps below.
+
+**Manual backup**
+
+```bash
+# Redis — hot snapshot: trigger a save, then copy the RDB out
+docker compose exec proxy redis-cli BGSAVE
+docker compose cp proxy:/data/dump.rdb proxy-redis-$(date +%F).rdb
+
+# Secrets — copy the config dir (holds OIDC client secret, LDAP bind password)
+cp -a ./config config-backup-$(date +%F) && chmod 700 config-backup-$(date +%F)
+```
+Store the `.rdb` and config copy **off the host** — they contain secrets and
+the whole Host/permission/user dataset.
+
+**Restore — Redis (full proxy state + certs)**
+
+```bash
+cp -a config-backup-<date> ./config && chmod 700 ./config
+docker compose up -d
+docker compose stop proxy
+# AOF wins on startup — delete it so the RDB is loaded instead (see note).
+docker compose run --rm --no-deps --entrypoint sh proxy -c \
+  'rm -f /data/appendonly.aof /data/appendonly.aof.*'
+docker compose cp proxy-redis-<date>.rdb proxy:/data/dump.rdb
+docker compose start proxy
+```
+
+> **AOF vs RDB (important):** with `--appendonly yes`, Redis loads
+> `appendonly.aof` on startup and **ignores** `dump.rdb` if the AOF exists. To
+> restore from an RDB snapshot you **must delete the AOF first** (the runbook
+> does this); Redis then loads the RDB and writes a fresh AOF. Verify:
+> `docker compose exec proxy redis-cli DBSIZE`.
+>
+> Restoring Redis restores cert state **at the snapshot time** — certs issued
+> after the snapshot are lost and will be re-issued on next request.
+
+**Upgrades**
+
+```bash
+./setup.sh          # backs up, then rebuilds — proxy-data keeps Redis state
+# (standalone) docker compose pull && docker compose up -d
+```
+Host records, permissions, DNS creds, local users, and Let's Encrypt certs all
+survive the rebuild because they live on the `proxy-data` volume, not in the
+image. **Migrations note:** if a release ships a `nodejs/migrations/` script,
+run it after upgrading (it transforms in-Redis records); see the release notes.
 
 ### Logs
 

docker-compose.yml

@@ -2,15 +2,19 @@
 # (OpenResty + Node management app + Redis in one container).
 #
 # The proxy is an OIDC client of an SSO Manager (or any OIDC provider) AND a
-# direct LDAP client for user lookups. Supply that wiring via `app_*`
+# direct LDAP client for user lookups. That wiring (oidc/ldap/auth) is read from
-# environment variables — the highest-precedence config layer in
+# a bind-mounted ./config/proxy-secrets.js — docker-entrypoint.sh symlinks it
-# @simpleworkjs/conf (>= 1.1.0, pinned in nodejs/package-lock.json). No
+# into /app/conf/secrets.js so @simpleworkjs/conf reads it. No app_* env is
-# secrets.js is baked in; set the values here, in a .env file, or via an
+# passed here: app_* env beats secrets.js in @simpleworkjs/conf (precedence:
-# env_file (the theta42/theta-env unified repo generates one with setup.sh).
+# base.js < <env>.js < secrets.js < app_* env), so the file must be the only
+# source. See secrets.js.example for the shape.
 #
-# Requires @simpleworkjs/conf >= 1.1.0 in the image (env overrides). The lock
+# Compose only interpolates the port + OpenResty-runtime defaults below — there
-# is already on ^1.1.0; rebuild with `docker compose up -d --build` after any
+# is no .env file. Override on the command line if needed:
-# package change.
+#   HTTP_PORT=8080 HTTPS_PORT=8443 docker compose up -d
+#
+# Requires @simpleworkjs/conf >= 1.1.0 in the image. The lock is already on
+# ^1.1.0; rebuild with `docker compose up -d --build` after any package change.
 
 services:
   proxy:
@@ -29,51 +33,26 @@ services:
       # LAN — the OpenResty front proxies the UI/api under its own TLS.
       - "127.0.0.1:${MGMT_PORT:-3000}:3000"
     environment:
-      # ── OpenResty runtime (see docker-entrypoint.sh) ──
+      # OpenResty runtime (see docker-entrypoint.sh). These are NOT app_* config
+      # keys, so they don't conflict with secrets.js. oidc/ldap/auth config comes
+      # from ./config/proxy-secrets.js, not from env.
       # Resolver for upstream names in Host records (default = Docker DNS).
       - RESOLVER=${RESOLVER:-127.0.0.11}
       # Trusted range for X-Real-IP. Empty = proxy is the front (default,
       # removes the real_ip block). Set to an upstream proxy's CIDR if one
       # sits in front and sets X-Real-IP.
       - REAL_IP_FROM=${REAL_IP_FROM:-}
-
-      # ── OIDC client config (app_oidc__*) ──
-      # Point at your SSO Manager. Issuer + authorization/endSession are the
-      # browser-facing URLs; token/userinfo can be the internal URL if the SSO
-      # is on the same docker network (avoids a TLS hairpin through the proxy).
-      - app_oidc__enabled=${OIDC_ENABLED:-true}
-      - app_oidc__issuer=${OIDC_ISSUER:-https://sso.example.com}
-      - app_oidc__authorizationEndpoint=${OIDC_AUTHORIZATION_ENDPOINT:-https://sso.example.com/oauth/authorize}
-      - app_oidc__tokenEndpoint=${OIDC_TOKEN_ENDPOINT:-http://sso-manager:3001/oauth/token}
-      - app_oidc__userinfoEndpoint=${OIDC_USERINFO_ENDPOINT:-http://sso-manager:3001/oauth/userinfo}
-      - app_oidc__endSessionEndpoint=${OIDC_ENDSESSION_ENDPOINT:-https://sso.example.com/oauth/logout}
-      - app_oidc__clientId=${OIDC_CLIENT_ID:-}
-      - app_oidc__clientSecret=${OIDC_CLIENT_SECRET:-}
-      - app_oidc__redirectUri=${OIDC_REDIRECT_URI:-https://proxy.example.com/api/auth/oidc/callback}
-
-      # ── LDAP client config (app_ldap__*) ──
-      # Direct user lookups. ldaps:// + rejectUnauthorized=false for a
-      # self-signed cert, or set app_ldap__tlsOptions__ca=<path> for strict.
-      - app_ldap__url=${LDAP_URL:-ldaps://sso-manager:636}
-      - app_ldap__bindDN=${LDAP_BIND_DN:-cn=ldapclient,ou=people,dc=example,dc=com}
-      - app_ldap__bindPassword=${LDAP_BIND_PASSWORD:-}
-      - app_ldap__searchBase=${LDAP_SEARCH_BASE:-ou=people,dc=example,dc=com}
-      - app_ldap__userFilter=${LDAP_USER_FILTER:-(objectClass=inetOrgPerson)}
-      - app_ldap__tlsOptions__rejectUnauthorized=${LDAP_REJECT_UNAUTHORIZED:-false}
-
-      # ── Authorization ──
-      # Local anti-lockout admin (matches auth.adminUsers in conf/base.js).
-      - app_auth__adminUsers=${AUTH_ADMIN_USERS:-proxyadmin}
-
       - NODE_ENV=production
       - NODE_PORT=3000
     volumes:
-      # Let's Encrypt cert store + auto-ssl redis data live with the bundled
+      # Operator-edited secrets (proxy-secrets.js). The entrypoint symlinks
-      # redis (in-container, in-memory). Mount these to persist across recreation:
+      # /config/proxy-secrets.js -> /app/conf/secrets.js so @simpleworkjs/conf
-      #   proxy-cache  -> /var/cache/nginx/proxy  (response cache)
+      # reads the oidc/ldap/auth config. See secrets.js.example for the shape.
-      #   proxy-logs   -> /var/log/nginx          (access/error logs)
+      - ./config:/config:ro
-      # Auto-ssl certs are in the bundled redis (in-memory, lost on recreation)
+      # Persist Redis (AOF + RDB) so Host records, permissions, DNS creds, local
-      # unless you enable redis persistence in docker-entrypoint.sh.
+      # users, AND the lua-resty-auto-ssl Let's Encrypt certs survive container
+      # recreation. Restoring Redis also restores cert state at snapshot time.
+      - proxy-data:/data
       - proxy-cache:/var/cache/nginx/proxy
       - proxy-logs:/var/log/nginx
     healthcheck:
@@ -85,4 +64,5 @@ services:
 
 volumes:
   proxy-cache:
-  proxy-logs:
+  proxy-logs:
+  proxy-data:

docker-entrypoint.sh

@@ -9,10 +9,13 @@
 #   3. OpenResty        (80/443/4443) — exec'd in the foreground as PID 2 (under
 #                      dumb-init, PID 1) so it receives SIGTERM from `docker stop`.
 #
-# The app reads its config from conf/base.js deep-merged with `app_*` env vars
+# The app reads its config from conf/base.js deep-merged with conf/secrets.js
-# (requires @simpleworkjs/conf >= 1.1.0, pinned in nodejs/package-lock.json). No
+# and `app_*` env vars (requires @simpleworkjs/conf >= 1.1.0, pinned in
-# secrets.js is baked into the image — supply oidc/ldap/auth config via `app_*`
+# nodejs/package-lock.json). No secrets.js is baked into the image. The unified
-# env (compose `environment:` / `env_file:`), or mount a conf/secrets.js.
+# theta-env stack mounts ./config/proxy-secrets.js at /config; this entrypoint
+# symlinks it into /app/conf/secrets.js so the app reads oidc/ldap/auth config
+# from the file (no app_* env needed). Without the mount, supply the same config
+# via `app_*` env (compose `environment:` / `env_file:`).
 #
 # OpenResty config: the committed ops/nginx_conf/*.conf carry the bare-metal
 # home-LAN values (set_real_ip_from 192.168.1.0/24; resolver 192.168.1.1). They
@@ -25,6 +28,19 @@ set -e
 info()  { echo "[INFO] $*"; }
 error() { echo "[ERROR] $*" >&2; }
 
+# ── Optional: mount proxy secrets.js ─────────────────────────────────────────
+# When /config/proxy-secrets.js is present (unified theta-env stack, or any
+# deployment that bind-mounts ./config), symlink it into /app/conf/secrets.js so
+# @simpleworkjs/conf reads the oidc/ldap/auth config from the file. No app_* env
+# should then be passed — app_* env beats secrets.js in @simpleworkjs/conf
+# (precedence: base.js < <env>.js < secrets.js < app_* env), so the file is
+# authoritative only if the matching app_* env is absent. When the file is
+# absent the app falls back to app_* env (compose environment / env_file).
+if [[ -f /config/proxy-secrets.js ]]; then
+    ln -sf /config/proxy-secrets.js /app/conf/secrets.js
+    info "Loaded config from /config/proxy-secrets.js (secrets.js authoritative)"
+fi
+
 # ── Fallback SSL cert for lua-resty-auto-ssl ─────────────────────────────────
 # autossl.conf references /etc/ssl/resty-auto-ssl-fallback.{crt,key} — auto-ssl
 # serves this for unknown SNI before a real Let's Encrypt cert is issued.
@@ -77,14 +93,21 @@ if ! openresty -t >/dev/null 2>&1; then
 fi
 
 # ── Redis ───────────────────────────────────────────────────────────────────
-# In-memory, no persistence (cache/session/cert data only). The app,
+# Persisted to /data (AOF + RDB) so Host records, permissions, DNS creds, local
-# auto-ssl, and targetinfo.lua all reach it at 127.0.0.1:6379 (the redis client
+# users, AND the lua-resty-auto-ssl Let's Encrypt certs all survive container
-# default + the one literal in targetinfo.lua). Run in the background
+# recreation (persisting Redis persists the cert store — avoids LE re-issue /
-# (--daemonize no, backgrounded by the shell); the container's lifecycle is
+# rate limits on rebuild). The app, auto-ssl, and targetinfo.lua all reach it at
-# owned by OpenResty (exec'd below), and `restart: unless-stopped` in compose
+# 127.0.0.1:6379 (the redis client default + the one literal in targetinfo.lua).
-# handles full restarts.
+# Run in the background (--daemonize no, backgrounded by the shell); the
-info "Starting Redis..."
+# container's lifecycle is owned by OpenResty (exec'd below), and
-redis-server --save "" --appendonly no --daemonize no &
+# `restart: unless-stopped` in compose handles full restarts.
+REDIS_DATA_DIR="${REDIS_DATA_DIR:-/data}"
+mkdir -p "$REDIS_DATA_DIR"
+chmod 700 "$REDIS_DATA_DIR"
+info "Starting Redis (AOF persisted to $REDIS_DATA_DIR)..."
+redis-server --daemonize no --dir "$REDIS_DATA_DIR" --appendonly yes \
+    --appendfilename appendonly.aof --save 900 1 --save 300 10 --save 60 10000 \
+    --dbfilename dump.rdb &
 REDIS_PID=$!
 for i in $(seq 1 15); do
     if redis-cli ping >/dev/null 2>&1; then info "Redis is ready"; break; fi

docs/docker.md

@@ -18,7 +18,9 @@ unified [theta-env](https://github.com/theta42/theta-env) stack.
 ```bash
 git clone https://github.com/theta42/proxy.git
 cd proxy
-cp .env.example .env   # optional: set OIDC/LDAP wiring + ports
+mkdir -p config && chmod 700 config
+cp secrets.js.example config/proxy-secrets.js   # set OIDC/LDAP wiring
+$EDITOR config/proxy-secrets.js
 docker compose up -d --build
 ```
 
@@ -36,9 +38,17 @@ which deep-merges, in order:
 3. `conf/secrets.js` (gitignored)
 4. **`app_*` environment variables** — the highest-precedence layer
 
+The bundled `docker-compose.yml` mount `./config/proxy-secrets.js` at `/config`,
+and `docker-entrypoint.sh` symlinks it into `/app/conf/secrets.js` so the app
+reads the OIDC + LDAP + auth wiring from the file. **No `app_*` env is passed** —
+`app_*` env beats `secrets.js`, so the file is authoritative only if the matching
+`app_*` env is absent. See `secrets.js.example` for the shape.
+
 Any env var starting with `app_` overrides the merged config; the rest of the
 name splits on **double-underscore** (`__`) into a nested path. Values are
-`JSON.parse`-coerced when possible, kept as strings otherwise.
+`JSON.parse`-coerced when possible, kept as strings otherwise. `app_*` env is
+still supported for advanced/standalone use — add the vars to the compose
+`environment:` block yourself (the bundled compose no longer sets them).
 
 > **Requires `@simpleworkjs/conf` >= 1.1.0.** The `app_*` env layer is not
 > honored on 1.0.0. The lock is already on `^1.1.0`.
@@ -76,9 +86,11 @@ for the complete reference.
 
 ## Auto-SSL / Let's Encrypt
 
-`lua-resty-auto-ssl` stores certs in the bundled Redis (in-memory by default —
+`lua-resty-auto-ssl` stores certs in the bundled Redis. Redis is now AOF+RDB
-lost on container recreation). For cert persistence, mount a Redis AOF/RDB
+persisted to the `proxy-data` volume (not in-memory), so **Let's Encrypt certs
-volume. Port 80 is required for HTTP-01 challenges (mapped in the compose).
+survive container recreation** — no re-issue / rate-limit on rebuild. Port 80 is
+required for HTTP-01 challenges (mapped in the compose). Back up + restore Redis
+to back up + restore cert state (see *Backups and restore* in `DEPLOYMENT.md`).
 
 ## Fronting an SSO Manager
 

secrets.js.example

@@ -0,0 +1,72 @@
+'use strict';
+
+// Example secrets configuration for the theta42/proxy.
+//
+// The proxy is an OIDC client of an SSO Manager (or any OIDC provider) AND a
+// direct LDAP client for user lookups. This file supplies that wiring.
+//
+// Docker / unified stack: place at ./config/proxy-secrets.js and bind-mount
+// ./config at /config (see docker-compose.yml); docker-entrypoint.sh symlinks
+// it into /app/conf/secrets.js so @simpleworkjs/conf reads it. No app_* env
+// should be passed — app_* env beats this file in @simpleworkjs/conf, so the
+// file is authoritative only if the matching app_* env is absent.
+//
+// Bare-metal: copy to nodejs/conf/secrets.js and fill in your values. Values
+// here override conf/base.js and win over <environment>.js.
+//
+// Only the keys the app reads are listed below. The `stack` key is read by the
+// theta-env orchestrator (setup.sh) and ignored by the app.
+
+module.exports = {
+    // OpenID Connect — point at your SSO Manager. Issuer + authorization/
+    // endSession are browser-facing URLs; token/userinfo can be the internal
+    // URL if the SSO is on the same docker network (avoids a TLS hairpin).
+    oidc: {
+        enabled: true,
+        issuer: 'https://sso.example.com',
+        authorizationEndpoint: 'https://sso.example.com/oauth/authorize',
+        tokenEndpoint: 'http://sso-manager:3001/oauth/token',
+        userinfoEndpoint: 'http://sso-manager:3001/oauth/userinfo',
+        endSessionEndpoint: 'https://sso.example.com/oauth/logout',
+        clientId: 'set-me',                       // registered on the SSO
+        clientSecret: 'set-me',                   // from the SSO client record
+        redirectUri: 'https://proxy.example.com/api/auth/oidc/callback',
+        scopes: ['openid', 'profile', 'email', 'groups'],
+        groupsClaim: 'groups',
+        usernameClaim: 'preferred_username',
+    },
+
+    // Direct LDAP user lookups. ldaps:// + rejectUnauthorized:false for a
+    // self-signed cert (the SSO's default), or set tlsOptions.ca to a CA path
+    // for strict verification. bindPassword MUST match the
+    // serviceAccountPass in the SSO's sso-secrets.js (the proxy binds as that
+    // service account).
+    ldap: {
+        url: 'ldaps://sso-manager:636',
+        bindDN: 'cn=ldapclient,ou=people,dc=example,dc=com',
+        bindPassword: 'set-me',
+        searchBase: 'ou=people,dc=example,dc=com',
+        userFilter: '(objectClass=inetOrgPerson)',
+        userNameAttribute: 'uid',
+        tlsOptions: {
+            rejectUnauthorized: false,            // true + ca for a CA-signed cert
+        },
+    },
+
+    // Authorization. adminUsers is the local anti-lockout admin (matches
+    // auth.adminUsers in conf/base.js). adminGroups: SSO/LDAP groups whose
+    // members are always global admins.
+    auth: {
+        adminGroups: [],
+        adminUsers: ['proxyadmin'],
+        groupRoleMap: {},
+    },
+
+    // ── Orchestrator-only (ignored by the app) ───────────────────────────────
+    // Read by the theta-env setup.sh (e.g. to seed the OAuth client). Omit for
+    // bare-metal use.
+    stack: {
+        ssoHost: 'sso.example.com',               // public SSO hostname
+        proxyHost: 'proxy.example.com',            // public proxy hostname
+    },
+};