Step 1 (submodule update) only refreshes proxy/sso-manager-node — it
never pulls setup.sh or this repo's own files. So on an existing
deployment, running ./setup.sh alone would build fresh submodule code
but execute a stale copy of the orchestration script itself (missing
whatever fixes landed in it, e.g. the CFG_DOMAIN rename or the
Host-registration step), unless the operator remembered to `git pull`
theta-env manually first.
Add a step 0 that fast-forwards the current branch to its upstream
before anything else runs, then re-execs the script so the rest of
the run uses the freshly-pulled version rather than the copy already
read into memory. Never blocks the run: skips silently if this isn't
a git checkout, is on a detached HEAD, or has no upstream configured
(all normal for e.g. a tarball download); warns and continues on the
current checkout for any other pull failure (offline, local changes
that prevent a fast-forward). Skip entirely with SKIP_SELF_UPDATE=1,
matching the existing SKIP_SUBMODULE_UPDATE convention.
Verified in an isolated scratch clone (not the working repo): pulling
a real commit forward triggers the re-exec and the second invocation
picks up the new HEAD; already-up-to-date and detached-HEAD cases are
both silent no-ops.
The proxy routes every hostname it serves purely off a Host record
(ops/nginx_conf/proxy.conf has no default/self route — targetinfo.lua
does a Redis lookup per request, full stop). Nothing created these for
the SSO's own UI or the proxy's own management UI, so on a fresh
install https://<SSO_HOST> and https://<PROXY_HOST> both 404 despite
setup.sh's summary claiming they're "fronted by the proxy under TLS".
Add a step after the proxy is healthy that runs a short script inside
the proxy container calling its Host model directly (no HTTP API call,
since no authenticated session exists yet at this point in the run):
- <SSO_HOST> -> sso-manager:3001 (the Docker service)
- <PROXY_HOST> -> 127.0.0.1:3000 (the proxy's own management app)
Both created with sso_enabled: false — each app already gates its own
login, and SSO-gating the SSO's own login page would be circular.
Idempotent: skips a host that already exists.
Entering the base DN directly (CFG_BASE_DN=dc=foo,dc=bar) is fragile —
a missing comma between labels silently produces a malformed domain
(e.g. "theta42dc=duckdns.org" instead of "theta42.duckdns.org") with
no validation to catch it. Flip the direction: operators now set
CFG_DOMAIN to a plain domain (any number of labels — a DuckDNS domain
like foo.duckdns.org works the same as a normal one), and setup.sh
derives the base DN from it via the new dn_from_domain().
CFG_BASE_DN is still supported as an explicit override (e.g. to
namespace under an OU-style prefix) and is how migrated .env/proxy.env
deployments keep working, since domain_from_dn() still reads the
domain back out of an existing DN either way.
The first-run flow generated ./config/*.js with example.com/dc=example,dc=com
defaults and then exit 0'd, telling the operator to hand-edit. The domain/base
DN was repeated across ~9 fields in each secrets file; on the deploy host only
the stack block was updated (to dc=718it,dc=biz) while ldap.* DNs stayed at
dc=example,dc=com, so slapd's root DN didn't match the app's bindDN and
bootstrap failed with 401 Invalid Credentials.
Enter the domain once: a new setup.env (gitignored; setup.env.example is the
committed template) holds the essential, non-repeating first-run info — the
domain as an LDAP base DN (CFG_BASE_DN). setup.sh reads it ONLY on first run
(when ./config/*.js don't exist), derives hostnames (sso.<domain> /
proxy.<domain>) and all LDAP DNs from it, generates both secrets files with
the real domain filled in everywhere + random secrets, and proceeds to build
in the same run (no edit-and-re-run step). After first run the secrets files
are operator-owned and setup.env is ignored — the apps and secrets files are
unchanged.
- setup.sh ensure_config: source setup.env -> bind CFG_* to empty (set -u
safe) -> unchanged .env/proxy.env legacy migration -> derive from base DN
(no example.com defaults; die with a helpful msg if CFG_BASE_DN blank) ->
generate + proceed. Header comments updated.
- setup.env.example: committed template; secrets stay out (generated into
./config/*.js).
- .gitignore: ignore setup.env (per-deployment).
- README.md + docs/quickstart.md: Quickstart now cp setup.env.example ->
set CFG_BASE_DN -> ./setup.sh; domain-asked-once note in "Before you begin".
- Bump sso-manager-node gitlink to 11fb2c0 (docs PR #37: base-DN-is-the-one-
domain-value note in sso-manager README/DEPLOYMENT/secrets.js.example).
Co-authored-by: Claude <noreply@anthropic.com>
After #14 the deploy-host snapshot still stopped right after
'config -> config/' with no further output — neither the LDAP/Redis
'snapshotting...' lines nor the post-snapshot 'Building + starting
sso-manager' line, so the stall was somewhere in the no-container path
(no containers are up yet when ./setup.sh is first run) but invisible
because every skip was silent and there was no marker between the config
copy and the function return.
Instrument + harden backup_before_rebuild() so the next run localizes it:
- ERR trap (scoped to the function): if a command trips set -e and aborts
the snapshot, print 'snapshot aborted by command: <cmd>' instead of
dying mute after 'Snapshotting state to ...'.
- Explicit 'skipped' branches: 'LDAP: sso-manager not running — skipped'
and 'Redis (<svc>): not running — skipped' so a no-container run shows
which path was taken instead of going quiet.
- 'pruning old backups (keep=N)...' before the retention loop and
'snapshot complete.' at the end, so a stall is pinned to the retention
loop (or ruled out of the snapshot entirely).
- Retention: clamp with [[ ]] not (( )) — (( keep < 1 )) returns exit 1
when false, a classic set -e landmine; guard rm -rf with '|| true';
skip symlinks and non-dir entries so a stray symlink in ./backups can't
point rm at an arbitrary tree.
Co-authored-by: Claude <noreply@anthropic.com>
backup_before_rebuild() stalled on the deploy host after printing
"Snapshotting state to ..." with no LDAP/Redis output, and silently
no-oped on any stack brought up under a different compose project.
Root causes:
- The LDAP slapcat + in-container base-DN read used
`$COMPOSE exec -T sso-manager`, which exits 1 *silently* (no stderr)
when the running container belongs to a different compose project than
the one the superproject resolves (the standalone SSO from the
sso-manager-node submodule is project "sso-manager-node", not
"theta-env"). The snapshot then no-ops with no breadcrumb.
- The host-side `node -e 'require(./config/sso-secrets.js)'` base-DN read
had no timeout, so a malformed secrets.js could block at require() time
and hang the whole rebuild at that line.
- The Redis RDB copy hardcoded `/data/dump.rdb` and used
`$COMPOSE cp` — wrong path on the standalone layout (Redis dir is
/app, not /data) and the same compose-project mismatch as above.
Fix:
- Switch every in-container call to `docker exec <name>` / `docker cp`
by container name, which works regardless of the owning compose project
(matches the already-working Redis BGSAVE/SAVE path).
- Wrap the host-side node read in `timeout 5` and slapcat in `timeout 20`
so no single command can hang the rebuild.
- Ask Redis for its own `CONFIG GET dir` / `dbfilename` and copy from the
real path, so the RDB is found on both the unified (/data) and standalone
(/app) layouts.
- Add per-step progress lines (config/LDAP/each Redis service) so any future
stall is localized to the exact step instead of hanging silently.
Co-authored-by: Claude <noreply@anthropic.com>
Two bugs in backup_before_rebuild() that made every pre-rebuild snapshot
fail once a stack was actually running:
1. Redis "BGSAVE did not finish in 30s" — a race. The code issued BGSAVE and
only THEN captured `before = LASTSAVE`. On a small dataset BGSAVE finishes
in well under a second, so `before` was already the post-save value and the
poll waited 30s for a second advance that never came (both services, every
run). Fix: capture LASTSAVE before BGSAVE. Also add a synchronous SAVE
fallback — BGSAVE can fork-fail when the host has vm.overcommit_memory=0
(this host does: 0), and SAVE can't fork-fail. The brief block is fine
pre-rebuild. Poll shortened to 10s since a small dataset completes in <1s.
2. "could not read ldapBaseDn from sso-secrets.js" — the base DN was read via
`docker compose exec sso-manager node -e 'require("/config/sso-secrets.js")'`,
which fails when the running container predates the ./config bind-mount
(no /config in the container). Fix: read from the host-side
./config/sso-secrets.js first (same require pattern the SSO entrypoint
uses; works regardless of the running container's mounts), falling back to
the in-container read.
Verified live: SAVE advances LASTSAVE (reply "OK"); host-side require returns
stack.ldapBaseDn on the example config.
Co-authored-by: Claude <noreply@anthropic.com>
Part A — lossless upgrades:
- Persist both bundled Redis stores via AOF+RDB on named volumes (sso-data,
proxy-data) so OAuth clients, Host records, perms, DNS creds, and auto-ssl
Let's Encrypt certs survive rebuilds.
- setup.sh: backup_before_rebuild() snapshots ./config/ + LDAP (slapcat) +
both Redis (BGSAVE + compose cp) to ./backups/<ts>/ before each rebuild,
keeps last BACKUP_KEEP (default 5). First run is a no-op.
- Restore runbook (README + docs): full / Redis-only / LDAP-only, with the
AOF-vs-RDB note (delete the AOF before restoring an RDB).
Part B — eliminate .env / proxy.env:
- All config + secrets live in bind-mounted ./config/ (gitignored), read by each
app's @simpleworkjs/conf from a symlinked secrets.js. Compose passes only
NODE_ENV + NODE_PORT (no app_* env, which would override secrets.js).
- ./config/sso-secrets.js: app secrets + orchestrator-only stack/bootstrap/
serviceAccountPass keys (app ignores the ones it doesn't use).
- ./config/proxy-secrets.js: oidc (clientId/clientSecret filled in by the
bootstrap), ldap (bind creds), auth (admin groups/users).
- setup.sh ensure_config(): generates ./config/ with random secrets on first
run (then exits for editing); one-time migration from .env/proxy.env
preserving existing secrets (LDAP admin pass, JWT, OAuth client, service
pass) so a running deployment keeps its directory + tokens + OAuth client.
- bootstrap/bootstrap.js: reads /config/*.js (not process.env), registers the
proxy as an OIDC client, and writes the SSO-generated client id+secret back
into ./config/proxy-secrets.js (sso mounts ./config RW, proxy RO).
- config.example/ holds committed annotated templates for manual reference.
- .gitignore: add config/, backups/, *.rdb, *.ldif.
Bump both gitlinks to the merged submodule tips:
- sso-manager-node -> 6920a9f (PR #34)
- proxy -> 8e78604 (PR #118)
Co-authored-by: Claude <noreply@anthropic.com>
Add a first step to setup.sh that runs `git submodule update --init --remote
--recursive`, so each ./setup.sh builds from the newest sso-manager-node + proxy
upstream rather than whatever was pinned at clone time. --init also populates
the submodules if the repo was cloned without --recursive.
Behavior:
- If the fetch is unreachable (offline), warn and continue building the
currently checked-out code instead of hard-failing.
- SKIP_SUBMODULE_UPDATE=1 locks to the pinned commits (offline rebuild /
deliberate pin).
- Verifies the build contexts (Dockerfile.openldap, Dockerfile) exist and
dies with a clear message if a submodule was never initialized.
- Requires git (added to the Requires line); git absent is fatal unless
SKIP_SUBMODULE_UPDATE=1.
Renumbered the subsequent step headers (env -> 2, sso-manager -> 3, ...).
README repo-layout note updated to say setup.sh auto-updates submodules.
Co-Authored-By: Claude <noreply@anthropic.com>
wmantly