theta42/theta-suite
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Persist Redis + config in bind-mounted ./config/ (no .env); add backup/restore (#8)

Browse Source 
Part A — lossless upgrades:
- Persist both bundled Redis stores via AOF+RDB on named volumes (sso-data,
  proxy-data) so OAuth clients, Host records, perms, DNS creds, and auto-ssl
  Let's Encrypt certs survive rebuilds.
- setup.sh: backup_before_rebuild() snapshots ./config/ + LDAP (slapcat) +
  both Redis (BGSAVE + compose cp) to ./backups/<ts>/ before each rebuild,
  keeps last BACKUP_KEEP (default 5). First run is a no-op.
- Restore runbook (README + docs): full / Redis-only / LDAP-only, with the
  AOF-vs-RDB note (delete the AOF before restoring an RDB).

Part B — eliminate .env / proxy.env:
- All config + secrets live in bind-mounted ./config/ (gitignored), read by each
  app's @simpleworkjs/conf from a symlinked secrets.js. Compose passes only
  NODE_ENV + NODE_PORT (no app_* env, which would override secrets.js).
- ./config/sso-secrets.js: app secrets + orchestrator-only stack/bootstrap/
  serviceAccountPass keys (app ignores the ones it doesn't use).
- ./config/proxy-secrets.js: oidc (clientId/clientSecret filled in by the
  bootstrap), ldap (bind creds), auth (admin groups/users).
- setup.sh ensure_config(): generates ./config/ with random secrets on first
  run (then exits for editing); one-time migration from .env/proxy.env
  preserving existing secrets (LDAP admin pass, JWT, OAuth client, service
  pass) so a running deployment keeps its directory + tokens + OAuth client.
- bootstrap/bootstrap.js: reads /config/*.js (not process.env), registers the
  proxy as an OIDC client, and writes the SSO-generated client id+secret back
  into ./config/proxy-secrets.js (sso mounts ./config RW, proxy RO).
- config.example/ holds committed annotated templates for manual reference.
- .gitignore: add config/, backups/, *.rdb, *.ldif.

Bump both gitlinks to the merged submodule tips:
- sso-manager-node -> 6920a9f (PR #34)
- proxy -> 8e78604 (PR #118)

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
William Mantly
2026-07-12 13:16:17 -04:00
committed by GitHub
parent 2d2941e394
commit b5f24d40fc
13 changed files with 929 additions and 445 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+52 -24
View File
@@ -73,7 +73,9 @@ redis instances is the no-source-patch path and is fine at this scale.
`./setup.sh` orchestrates first-run wiring; `bootstrap/bootstrap.js` does the
actual work, running **inside the sso-manager container** (bind-mounted
read-only from this repo). It's deliberately self-contained — only Node
built-ins (`child_process`, `crypto`) + global `fetch`:
built-ins (`child_process`, `crypto`, `fs`) + global `fetch`, and it reads its
inputs from the bind-mounted `./config/sso-secrets.js` + `./config/proxy-secrets.js`
(not from env):
1. **Build + start sso-manager**, wait for `/health`.
2. **LDAP service account** — `ldapadd` `cn=ldapclient,ou=people,<base>` (an
@@ -86,44 +88,70 @@ built-ins (`child_process`, `crypto`) + global `fetch`:
4. **Log in** as that admin via `POST /api/auth/login {uid,password}` — this
also validates the password end-to-end.
5. **Register the proxy as an OIDC client** via `POST /api/oauth/client` (gated
by `app_sso_oauth_admin`, satisfied by step 3), capturing the raw
`client_secret` (shown once). If the client already exists and `proxy.env` is
present, leave it; if `proxy.env` was lost, rotate the secret so a restored
proxy gets one it can read.
6. **Write `./proxy.env`** (the proxy's `env_file`) from `.env` + the bootstrap
output — all `app_*` env overrides so the proxy reads them via
`@simpleworkjs/conf` (≥1.1.0).
7. **Build + start the proxy**, wait for `/health`.
by `app_sso_oauth_admin`, satisfied by step 3). The SSO **generates** the
`client_id`/`client_secret` (UUIDs) — supplied creds are ignored — so the
bootstrap writes the generated creds **back into `./config/proxy-secrets.js`**
(the sso-manager mounts `./config` read-write for this; the proxy mounts it
read-only). If `proxy-secrets.js` already holds a `clientId`+`clientSecret`
matching an existing client, they are kept; if the client exists but the file
has no usable secret, the secret is rotated and written back.
6. **Build + start the proxy**, wait for `/health`. The proxy entrypoint symlinks
`./config/proxy-secrets.js` to `/app/conf/secrets.js`, so `@simpleworkjs/conf`
(≥1.1.0) reads the OAuth creds + LDAP bind creds from the file.
`setup.sh` then prints the first-admin login + the public URLs.
### How config reaches the apps (no `.env`)
All config and secrets live in `./config/` (gitignored, bind-mounted). Each
entrypoint symlinks its file to `/app/conf/secrets.js` early, before the app
starts:
```
./config/sso-secrets.js -> sso-manager:/app/conf/secrets.js (./config RW)
./config/proxy-secrets.js -> proxy:/app/conf/secrets.js (./config RO)
```
`@simpleworkjs/conf` loads `conf/base.js → <env>.js → conf/secrets.js → app_*
env`, where **env beats `secrets.js`**. So compose passes **no `app_*` env vars**
(only `NODE_ENV`, `NODE_PORT`) — that makes `secrets.js` authoritative. The SSO
entrypoint reads the few values it needs at startup (LDAP base DN, admin
password, JWT secret, cert CN) from `secrets.js` via an in-container `node` call.
### Why not `require` the SSO's internal models?
A `docker compose exec` process reads `conf/base.js` defaults (the docker-exec
env doesn't carry the entrypoint's exported `app_*` vars), so the SSO's models
would bind the wrong LDAP DN. Using the `openldap-clients` binaries with explicit
admin creds sidesteps that entirely, and going through the HTTP API for the
OAuth client validates the whole admin login path end-to-end.
env doesn't carry the entrypoint's exported vars), so the SSO's models would
bind the wrong LDAP DN. Using the `openldap-clients` binaries with explicit
admin creds from `./config/sso-secrets.js` sidesteps that entirely, and going
through the HTTP API for the OAuth client validates the whole admin login path
end-to-end.
## Idempotency
Re-running `./setup.sh` converges to `.env`:
Re-running `./setup.sh` converges to `./config/`:
- The LDAP service account + admin passwords are **reset to `.env`**.
- The LDAP service account + admin passwords are **reset to `./config/`**.
- Group membership is ensured (add is a no-op if already a member).
- The OAuth client is left alone if `proxy.env` exists, rotated if not.
- The OAuth client is kept if `proxy-secrets.js` already holds its creds;
created or rotated otherwise, and the new creds written back.
So `setup.sh` is safe to re-run after editing `.env`, after a `docker compose
down`, or after restoring from backup.
So `setup.sh` is safe to re-run after editing `./config/`, after a `docker
compose down`, or after restoring from backup.
## Backups
## Backups and restore
`./setup.sh` auto-snapshots `./config/` + LDAP + both Redis to
`./backups/<timestamp>/` before each rebuild (keeps the last `BACKUP_KEEP`,
default 5). State lives on named volumes (`ldap-data`, `sso-data`, `proxy-data`)
and survives recreation; `down -v` wipes them. Redis is persisted with AOF +
RDB on those volumes. For the full manual-backup + restore runbook (full /
Redis-only / LDAP-only, with the AOF-vs-RDB note), see the *Backups and
restore* section of the [README](https://github.com/theta42/theta-env#backups-and-restore).
Quick LDAP backup:
```bash
docker compose exec sso-manager slapcat -f /etc/openldap/slapd.conf -b "$LDAP_BASE_DN" > backup.ldif
docker compose exec sso-manager slapcat -f /etc/openldap/slapd.conf -b "<base>" > backup.ldif
```
Keep your `.env` (holds `LDAP_ADMIN_PASS` + `JWT_SECRET`) and `proxy.env` too.
Restore is `ldapadd`/`ldapmodify` from the LDIF into a fresh directory, then
re-run `./setup.sh`.
[← Back to Home](index.html)
docs/index.md
+21 -17
View File
@@ -19,17 +19,16 @@ wires them together and automates the first-run glue.
```bash
git clone --recursive https://github.com/theta42/theta-env.git
cd theta-env
cp .env.example .env # edit the REQUIRED values (below)
./setup.sh
./setup.sh # generates ./config/ the first time — edit it, then re-run
./setup.sh # builds + bootstraps + starts the stack
```
You need **Docker** + **Docker Compose**. `./setup.sh` is idempotent — re-run any
time to converge the stack to your `.env`.
time to converge the stack to `./config/`.
See the [Quickstart Guide](quickstart.html) for a walkthrough of every `.env`
value and what `setup.sh` does, [Architecture](architecture.html) for how the
pieces fit together, and [Standalone](standalone.html) for running each project
on its own.
See the [Quickstart Guide](quickstart.html) for a walkthrough of `./config/` and
what `setup.sh` does, [Architecture](architecture.html) for how the pieces fit
together, and [Standalone](standalone.html) for running each project on its own.
## What you get
@@ -40,18 +39,23 @@ on its own.
- **LDAPS** at `ldaps://<host>:636` — legacy apps can bind directly (admin or
the read-only `cn=ldapclient` service account the bootstrap creates).
## The `.env` values you must set
## The `./config/` values you must set
| Key | What it is |
All config and secrets live in `./config/sso-secrets.js` +
`./config/proxy-secrets.js` (gitignored), generated by the first `./setup.sh`.
There is **no `.env`**. Set at least these in `./config/sso-secrets.js`:
| Key (in `sso-secrets.js`) | What it is |
|-----|------------|
| `LDAP_BASE_DN` | Directory base, e.g. `dc=lab,dc=local`. |
| `LDAP_ADMIN_PASS` | LDAP root password. **Save it.** |
| `JWT_SECRET` | Signs the SSO's tokens. Leave blank to auto-generate + persist. **Save it.** |
| `SSO_HOST` | Public hostname the proxy serves the SSO UI at. |
| `PROXY_HOST` | Public hostname the proxy serves its own mgmt UI at. |
| `BOOTSTRAP_ADMIN_UID` / `BOOTSTRAP_ADMIN_PASS` | Your first admin login. |
| `stack.ldapBaseDn` | Directory base, e.g. `dc=lab,dc=local`. |
| `ldap.bindPassword` | LDAP root password (generated). **Back it up.** |
| `oauth.jwtSecret` | Signs the SSO's tokens (generated). **Back it up.** |
| `stack.ssoHost` | Public hostname the proxy serves the SSO UI at. |
| `stack.proxyHost` | Public hostname the proxy serves its own mgmt UI at. |
| `bootstrap.adminUid` / `bootstrap.adminPass` | Your first admin login. |
See `.env.example` for the full list (SMTP, port overrides, LDAP cert CN, …).
See `config.example/` for the full annotated shape (SMTP, LDAP cert CN, proxy
OIDC/LDAP/auth, …).
## Architecture
@@ -81,7 +85,7 @@ diagram + the first-run bootstrap flow.
## Documentation
- [Quickstart Guide](quickstart.html) — full walkthrough of `.env` + `setup.sh`.
- [Quickstart Guide](quickstart.html) — full walkthrough of `./config/` + `setup.sh`.
- [Architecture](architecture.html) — the 3-repo + submodule + 2-container
design, and how the bootstrap wires the proxy into a fresh SSO.
- [Standalone](standalone.html) — running SSO Manager or the proxy on its own.
docs/quickstart.md
+45 -38
View File
@@ -11,9 +11,9 @@ title: Quickstart
- A Linux host with **Docker** + **Docker Compose** (the v2 plugin `docker
compose` or the v1 standalone `docker-compose` both work).
- Two hostnames that resolve to the host: one for the SSO UI (`SSO_HOST`), one
for the proxy mgmt UI (`PROXY_HOST`). On a real network add DNS records; for a
local try, add them to `/etc/hosts`.
- Two hostnames that resolve to the host: one for the SSO UI (your `stack.ssoHost`),
one for the proxy mgmt UI (your `stack.proxyHost`). On a real network add DNS
records; for a local try, add them to `/etc/hosts`.
- Port **80 + 443** reachable from the internet if you want Let's Encrypt
certs; otherwise the proxy serves a self-signed fallback (browsers warn —
expected for LAN use).
@@ -32,28 +32,32 @@ step. If you forgot it:
git submodule update --init --recursive
```
## 2. Configure `.env`
## 2. Configure `./config/`
```bash
cp .env.example .env
./setup.sh # generates ./config/ with random secrets, then exits
```
Edit `.env`. The **required** values:
The first `./setup.sh` generates `./config/sso-secrets.js` +
`./config/proxy-secrets.js` and **exits**, telling you to edit. Edit
`./config/sso-secrets.js` and at minimum set:
| Key | Example | Notes |
| Key (in `sso-secrets.js`) | Example | Notes |
|-----|---------|-------|
| `LDAP_BASE_DN` | `dc=lab,dc=local` | your directory base |
| `LDAP_ADMIN_PASS` | `...` | LDAP root password — **save it** |
| `JWT_SECRET` | _(blank)_ | leave blank to auto-generate + persist — **save it** |
| `SSO_HOST` | `sso.lab.local` | hostname the proxy serves the SSO UI at |
| `PROXY_HOST` | `proxy.lab.local` | hostname the proxy serves its own UI at |
| `BOOTSTRAP_ADMIN_UID` | `admin` | your first admin login |
| `BOOTSTRAP_ADMIN_PASS` | `...` | first admin password |
| `stack.ldapBaseDn` | `dc=lab,dc=local` | your directory base |
| `stack.ssoHost` | `sso.lab.local` | hostname the proxy serves the SSO UI at |
| `stack.proxyHost` | `proxy.lab.local` | hostname the proxy serves its own UI at |
| `bootstrap.adminUid` | `admin` | your first admin login |
| `bootstrap.adminPass` | `...` | first admin password |
Optional: `BOOTSTRAP_ADMIN_EMAIL`, `LDAP_SERVICE_PASS` (auto-generated if blank),
`SMTP_*` (for SSO password-reset/invite emails), `LDAP_CERT_CN`, and host port
overrides (`SSO_PORT`, `LDAPS_PORT`, `HTTP_PORT`, `HTTPS_PORT`,
`HTTPS_ALT_PORT`, `MGMT_PORT`). See `.env.example` for the full commented list.
Random secrets (`ldap.bindPassword`, `oauth.jwtSecret`, `serviceAccountPass`)
are generated for you — change them in the file if you like. Optional:
`bootstrap.adminEmail`, `smtp.*`, `stack.ldapCertCn`. See `config.example/` for
the full annotated shape, and each submodule's `secrets.js.example`.
> **Migrating from an older `.env`-based deployment?** If `.env`/`proxy.env`
> exist, `./setup.sh` migrates them into `./config/` preserving your existing
> secrets — no need to reconfigure.
## 3. Run
@@ -63,24 +67,22 @@ overrides (`SSO_PORT`, `LDAPS_PORT`, `HTTP_PORT`, `HTTPS_PORT`,
What happens:
1. Validates `.env` (copies from `.env.example` if missing, then exits so you
can edit it).
1. Snapshots state to `./backups/<timestamp>/` before rebuilding (a no-op on the
very first run).
2. Builds + starts **sso-manager**, waits for `/health`.
3. Runs the **bootstrap** inside the sso-manager container — creates the LDAP
service account, your first admin, and the proxy's OAuth client, and prints
the client id + secret.
4. Writes **`./proxy.env`** (the proxy's `app_*` config) from `.env` + the
bootstrap output.
5. Builds + starts **proxy**, waits for `/health`.
6. Prints your first-admin login + the public URLs.
service account, your first admin, and the proxy's OAuth client, and writes
the generated client id + secret into `./config/proxy-secrets.js`.
4. Builds + starts **proxy**, waits for `/health`.
5. Prints your first-admin login + the public URLs.
The first run builds two Docker images (a few minutes). Subsequent runs are
fast.
## 4. Point DNS at the host
`SSO_HOST` and `PROXY_HOST` must resolve to the host running the stack. Add DNS
records, or for a local try:
`stack.ssoHost` and `stack.proxyHost` (from `./config/sso-secrets.js`) must
resolve to the host running the stack. Add DNS records, or for a local try:
```bash
echo "127.0.0.1 sso.lab.local proxy.lab.local" | sudo tee -a /etc/hosts
@@ -92,7 +94,7 @@ serves a self-signed cert — browsers will warn, which is fine for home-lab use
## 5. Log in
Open `https://<SSO_HOST>` and log in as your bootstrap admin
(`BOOTSTRAP_ADMIN_UID` / `BOOTSTRAP_ADMIN_PASS`). From there you can add users,
(`bootstrap.adminUid` / `bootstrap.adminPass`). From there you can add users,
groups, and OAuth clients.
The proxy mgmt UI is at `https://<PROXY_HOST>` (same admin SSO login protects
@@ -103,10 +105,11 @@ First-run fallbacks (if DNS/TLS isn't ready yet): SSO UI at
## Re-running
`./setup.sh` is **idempotent** — safe to re-run after editing `.env`, after a
`docker compose down`, or after restoring from backup. It converges the stack to
your `.env` values (LDAP service account + admin passwords are reset to `.env`;
the OAuth client is left alone if `proxy.env` exists).
`./setup.sh` is **idempotent** — safe to re-run after editing `./config/`, after
a `docker compose down`, or after restoring from backup. It snapshots state,
then converges the stack to your `./config/` values (LDAP service account + admin
passwords are reset to the config; the OAuth client is kept if `proxy-secrets.js`
already holds its creds).
## Direct LDAP for legacy apps
@@ -121,16 +124,20 @@ ldapsearch -x -H ldaps://<host>:636 \
Use the `cn=ldapclient` service account (read-only, the bootstrap created it)
or the admin DN. Use LDAPS (636), not plain LDAP.
## Backups
## Backups and restore
`./setup.sh` auto-snapshots `./config/` + LDAP + both Redis to `./backups/<ts>/`
before each rebuild (keeps the last `BACKUP_KEEP`, default 5). For manual
backups and the full restore runbook (full / Redis-only / LDAP-only, with the
AOF-vs-RDB note), see the *Backups and restore* section of the
[README](https://github.com/theta42/theta-env#backups-and-restore). Quick LDAP
backup:
```bash
docker compose exec sso-manager slapcat -f /etc/openldap/slapd.conf \
-b "$LDAP_BASE_DN" > backup-$(date +%F).ldif
-b "<base>" > backup-$(date +%F).ldif
```
Keep `.env` + `proxy.env` alongside it. Restore is `ldapadd`/`ldapmodify` into a
fresh directory, then re-run `./setup.sh`.
## Next steps
- Add users / groups in the SSO UI.
docs/standalone.md
+20 -29
View File
@@ -18,22 +18,22 @@ The all-in-one image (`Dockerfile.openldap`) bundles the app + OpenLDAP + Redis:
```bash
git clone https://github.com/theta42/sso-manager-node.git
cd sso-manager-node
# Option A: configure via app_* env (preferred for Docker):
LDAP_ADMIN_PASS='choose-a-strong-password' \
JWT_SECRET="$(openssl rand -hex 32)" \
docker compose up -d --build
# Option B: configure via a file:
cp secrets.js.example nodejs/conf/secrets.js # edit it
mkdir -p config && cp secrets.js.example config/sso-secrets.js # edit it
docker compose up -d --build
```
The entrypoint symlinks `config/sso-secrets.js` to `nodejs/conf/secrets.js` so
`@simpleworkjs/conf` reads it. Set `ldap.bindPassword`, `oauth.jwtSecret`, and
the `stack`/`bootstrap` keys (the app ignores the ones it doesn't use). Pass
**no `app_*` env** — env beats `secrets.js`, so `app_*` would silently override
your file.
- Web UI: `http://localhost:3001`
- Health: `http://localhost:3001/health`
- OIDC discovery: `http://localhost:3001/.well-known/openid-configuration`
- LDAPS: `ldaps://<host>:636`
Requires `@simpleworkjs/conf` >= 1.1.0 for `app_*` env overrides. Full reference:
Requires `@simpleworkjs/conf` >= 1.1.0. Full reference:
[SSO Manager deployment docs](https://theta42.github.io/sso-manager-node/deployment.html).
### Bare metal
@@ -53,26 +53,16 @@ The all-in-one image (`Dockerfile`) bundles OpenResty + the Node app + Redis:
```bash
git clone https://github.com/theta42/proxy.git
cd proxy
# Wire it to an external SSO + LDAP via app_* env (or nodejs/conf/secrets.js):
cat > .env <<EOF
app_oidc__issuer=https://sso.example.com
app_oidc__authorizationEndpoint=https://sso.example.com/oauth/authorize
app_oidc__endSessionEndpoint=https://sso.example.com/oauth/logout
app_oidc__tokenEndpoint=https://sso.example.com/oauth/token
app_oidc__userinfoEndpoint=https://sso.example.com/oauth/userinfo
app_oidc__clientId=...
app_oidc__clientSecret=...
app_oidc__redirectUri=https://proxy.example.com/api/auth/oidc/callback
app_ldap__url=ldaps://sso.example.com:636
app_ldap__bindDN=cn=ldapclient,ou=people,dc=example,dc=com
app_ldap__bindPassword=...
app_ldap__searchBase=ou=people,dc=example,dc=com
app_ldap__userFilter=(objectClass=posixAccount)
app_ldap__tlsOptions__rejectUnauthorized=false
EOF
mkdir -p config && cp secrets.js.example config/proxy-secrets.js # edit it
docker compose up -d --build
```
The entrypoint symlinks `config/proxy-secrets.js` to `nodejs/conf/secrets.js` so
`@simpleworkjs/conf` reads it. Fill in `oidc` (your SSO's endpoints +
`clientId`/`clientSecret`/`redirectUri`), `ldap` (bind creds + search base), and
`auth` (admin groups/users). Pass **no `app_*` env** — env beats `secrets.js`,
so `app_*` would silently override your file.
- Proxy (public, auto-SSL): `https://<host>/`
- Mgmt UI / API: `http://127.0.0.1:3000/`
- Health: `http://127.0.0.1:3000/health`
@@ -97,12 +87,13 @@ documented in both projects' deployment guides:
1. One Docker network (or reachable hostnames) so the proxy can reach the SSO
internally for token/userinfo + LDAPS.
2. Set the SSO's `OAUTH_ISSUER` / `app_oauth__issuer` to the browser-facing HTTPS
2. Set the SSO's `oauth.issuer` (in its `secrets.js`) to the browser-facing HTTPS
URL the proxy serves the SSO at.
3. Register the proxy as an OIDC client in the SSO, with `redirectUri` matching
the proxy's callback.
4. Point the proxy's `app_ldap__url` at the SSO's LDAPS + create a dedicated
`cn=ldapclient` service account.
the proxy's callback; put the resulting `clientId`/`clientSecret` in the
proxy's `secrets.js`.
4. Point the proxy's `ldap.url` at the SSO's LDAPS + create a dedicated
`cn=ldapclient` service account; set the same password as `bindPassword`.
theta-env just automates those four steps with `./setup.sh`. If you prefer to
do them by hand (or want the two on separate hosts), follow the standalone