b5f24d40fc
Part A — lossless upgrades: - Persist both bundled Redis stores via AOF+RDB on named volumes (sso-data, proxy-data) so OAuth clients, Host records, perms, DNS creds, and auto-ssl Let's Encrypt certs survive rebuilds. - setup.sh: backup_before_rebuild() snapshots ./config/ + LDAP (slapcat) + both Redis (BGSAVE + compose cp) to ./backups/<ts>/ before each rebuild, keeps last BACKUP_KEEP (default 5). First run is a no-op. - Restore runbook (README + docs): full / Redis-only / LDAP-only, with the AOF-vs-RDB note (delete the AOF before restoring an RDB). Part B — eliminate .env / proxy.env: - All config + secrets live in bind-mounted ./config/ (gitignored), read by each app's @simpleworkjs/conf from a symlinked secrets.js. Compose passes only NODE_ENV + NODE_PORT (no app_* env, which would override secrets.js). - ./config/sso-secrets.js: app secrets + orchestrator-only stack/bootstrap/ serviceAccountPass keys (app ignores the ones it doesn't use). - ./config/proxy-secrets.js: oidc (clientId/clientSecret filled in by the bootstrap), ldap (bind creds), auth (admin groups/users). - setup.sh ensure_config(): generates ./config/ with random secrets on first run (then exits for editing); one-time migration from .env/proxy.env preserving existing secrets (LDAP admin pass, JWT, OAuth client, service pass) so a running deployment keeps its directory + tokens + OAuth client. - bootstrap/bootstrap.js: reads /config/*.js (not process.env), registers the proxy as an OIDC client, and writes the SSO-generated client id+secret back into ./config/proxy-secrets.js (sso mounts ./config RW, proxy RO). - config.example/ holds committed annotated templates for manual reference. - .gitignore: add config/, backups/, *.rdb, *.ldif. Bump both gitlinks to the merged submodule tips: - sso-manager-node -> 6920a9f (PR #34) - proxy -> 8e78604 (PR #118) Co-authored-by: Claude <noreply@anthropic.com>
157 lines
8.2 KiB
Markdown
157 lines
8.2 KiB
Markdown
---
|
|
layout: default
|
|
title: Architecture
|
|
---
|
|
|
|
# Architecture
|
|
|
|
[← Back to Home](index.html)
|
|
|
|
theta-env is a **composition** repo: it builds the two existing projects from
|
|
their git submodules and adds the glue that wires them together. It does not
|
|
fork or patch them — both projects work unchanged on their own.
|
|
|
|
## The three repos
|
|
|
|
| Repo | Role |
|
|
|------|------|
|
|
| [`theta42/sso-manager-node`](https://github.com/theta42/sso-manager-node) | OIDC provider + OpenLDAP directory + web UI. All-in-one image (`Dockerfile.openldap`). |
|
|
| [`theta42/proxy`](https://github.com/theta42/proxy) | OIDC-protected reverse proxy (OpenResty + Node mgmt app + Redis). All-in-one image (`Dockerfile`). |
|
|
| `theta42/theta-env` (this repo) | Composes the two on one Docker network + automates first-run wiring. |
|
|
|
|
The two projects are pinned as **git submodules**. `git clone --recursive`
|
|
fetches all three in one step; `git submodule update --remote` bumps them.
|
|
|
|
## The two containers
|
|
|
|
```
|
|
┌──────────────────────────────────────────────┐
|
|
│ your browser / apps / legacy LDAP clients │
|
|
└───────────────┬──────────────────────────────┘
|
|
│ https (:443) ldaps (:636)
|
|
┌─────────▼─────────┐
|
|
│ proxy container │ OpenResty :80/:443/:4443
|
|
│ (OIDC + LDAP) │ Node mgmt app :3000 (localhost only)
|
|
│ │ bundled Redis (127.0.0.1:6379)
|
|
└─────────┬─────────┘
|
|
┌─────────────┼────────────────────────────┐
|
|
│ ldaps:636 │ http:3001 (internal) │ OIDC token + userinfo
|
|
│ (docker net)│ (docker net, not published)│ (server-to-server)
|
|
▼ ▼ │
|
|
┌──────────────────────────────┐ │
|
|
│ sso-manager container │◄──────────────────┘
|
|
│ OIDC provider (Express) │ bundled Redis (127.0.0.1:6379)
|
|
│ OpenLDAP (slapd) │ web UI :3001 (localhost only)
|
|
│ ldaps :636 (published) │
|
|
└───────────────────────────────┘
|
|
▲
|
|
│ ldaps :636 (published to host) — legacy apps bind directly
|
|
│
|
|
┌──────────────────────────────┐
|
|
│ legacy apps (Gitea, Emby, …)│
|
|
└──────────────────────────────┘
|
|
```
|
|
|
|
Both containers bundle their **own Redis** (the proxy hardcodes `127.0.0.1:6379`
|
|
in three places that ignore config; the SSO's models default to the same). Two
|
|
redis instances is the no-source-patch path and is fine at this scale.
|
|
|
|
### What's exposed, what's not
|
|
|
|
| Port | On host? | Purpose |
|
|
|------|----------|---------|
|
|
| `443` (proxy) | **yes** | the public entry point — OIDC login + proxied apps + the SSO/proxy UIs |
|
|
| `80` (proxy) | **yes** | HTTP-01 for Let's Encrypt (and redirect to 443) |
|
|
| `4443` (proxy) | yes (optional) | alt HTTPS listener |
|
|
| `3000` (proxy) | localhost only | proxy mgmt UI/API (first-run convenience; fronted by 443 normally) |
|
|
| `636` (sso) | yes | LDAPS for legacy direct-LDAP clients |
|
|
| `3001` (sso) | localhost only | SSO web UI (first-run convenience; fronted by the proxy normally) |
|
|
| `389` (sso) | **no** | plain LDAP — internal only (app↔slapd over localhost) |
|
|
|
|
## The first-run bootstrap
|
|
|
|
`./setup.sh` orchestrates first-run wiring; `bootstrap/bootstrap.js` does the
|
|
actual work, running **inside the sso-manager container** (bind-mounted
|
|
read-only from this repo). It's deliberately self-contained — only Node
|
|
built-ins (`child_process`, `crypto`, `fs`) + global `fetch`, and it reads its
|
|
inputs from the bind-mounted `./config/sso-secrets.js` + `./config/proxy-secrets.js`
|
|
(not from env):
|
|
|
|
1. **Build + start sso-manager**, wait for `/health`.
|
|
2. **LDAP service account** — `ldapadd` `cn=ldapclient,ou=people,<base>` (an
|
|
`organizationalRole` with a `{SSHA512}` password). The proxy binds as this
|
|
DN — not the admin DN.
|
|
3. **First admin user** — `ldapadd` `cn=<uid>,ou=people,<base>` (inetOrgPerson +
|
|
posixAccount, `{SSHA512}` password) and add them as `member` of
|
|
`app_sso_admin` + `app_sso_oauth_admin` (the SSO's permission check reads the
|
|
group's `member` list).
|
|
4. **Log in** as that admin via `POST /api/auth/login {uid,password}` — this
|
|
also validates the password end-to-end.
|
|
5. **Register the proxy as an OIDC client** via `POST /api/oauth/client` (gated
|
|
by `app_sso_oauth_admin`, satisfied by step 3). The SSO **generates** the
|
|
`client_id`/`client_secret` (UUIDs) — supplied creds are ignored — so the
|
|
bootstrap writes the generated creds **back into `./config/proxy-secrets.js`**
|
|
(the sso-manager mounts `./config` read-write for this; the proxy mounts it
|
|
read-only). If `proxy-secrets.js` already holds a `clientId`+`clientSecret`
|
|
matching an existing client, they are kept; if the client exists but the file
|
|
has no usable secret, the secret is rotated and written back.
|
|
6. **Build + start the proxy**, wait for `/health`. The proxy entrypoint symlinks
|
|
`./config/proxy-secrets.js` to `/app/conf/secrets.js`, so `@simpleworkjs/conf`
|
|
(≥1.1.0) reads the OAuth creds + LDAP bind creds from the file.
|
|
|
|
`setup.sh` then prints the first-admin login + the public URLs.
|
|
|
|
### How config reaches the apps (no `.env`)
|
|
|
|
All config and secrets live in `./config/` (gitignored, bind-mounted). Each
|
|
entrypoint symlinks its file to `/app/conf/secrets.js` early, before the app
|
|
starts:
|
|
|
|
```
|
|
./config/sso-secrets.js -> sso-manager:/app/conf/secrets.js (./config RW)
|
|
./config/proxy-secrets.js -> proxy:/app/conf/secrets.js (./config RO)
|
|
```
|
|
|
|
`@simpleworkjs/conf` loads `conf/base.js → <env>.js → conf/secrets.js → app_*
|
|
env`, where **env beats `secrets.js`**. So compose passes **no `app_*` env vars**
|
|
(only `NODE_ENV`, `NODE_PORT`) — that makes `secrets.js` authoritative. The SSO
|
|
entrypoint reads the few values it needs at startup (LDAP base DN, admin
|
|
password, JWT secret, cert CN) from `secrets.js` via an in-container `node` call.
|
|
|
|
### Why not `require` the SSO's internal models?
|
|
|
|
A `docker compose exec` process reads `conf/base.js` defaults (the docker-exec
|
|
env doesn't carry the entrypoint's exported vars), so the SSO's models would
|
|
bind the wrong LDAP DN. Using the `openldap-clients` binaries with explicit
|
|
admin creds from `./config/sso-secrets.js` sidesteps that entirely, and going
|
|
through the HTTP API for the OAuth client validates the whole admin login path
|
|
end-to-end.
|
|
|
|
## Idempotency
|
|
|
|
Re-running `./setup.sh` converges to `./config/`:
|
|
|
|
- The LDAP service account + admin passwords are **reset to `./config/`**.
|
|
- Group membership is ensured (add is a no-op if already a member).
|
|
- The OAuth client is kept if `proxy-secrets.js` already holds its creds;
|
|
created or rotated otherwise, and the new creds written back.
|
|
|
|
So `setup.sh` is safe to re-run after editing `./config/`, after a `docker
|
|
compose down`, or after restoring from backup.
|
|
|
|
## Backups and restore
|
|
|
|
`./setup.sh` auto-snapshots `./config/` + LDAP + both Redis to
|
|
`./backups/<timestamp>/` before each rebuild (keeps the last `BACKUP_KEEP`,
|
|
default 5). State lives on named volumes (`ldap-data`, `sso-data`, `proxy-data`)
|
|
and survives recreation; `down -v` wipes them. Redis is persisted with AOF +
|
|
RDB on those volumes. For the full manual-backup + restore runbook (full /
|
|
Redis-only / LDAP-only, with the AOF-vs-RDB note), see the *Backups and
|
|
restore* section of the [README](https://github.com/theta42/theta-env#backups-and-restore).
|
|
Quick LDAP backup:
|
|
|
|
```bash
|
|
docker compose exec sso-manager slapcat -f /etc/openldap/slapd.conf -b "<base>" > backup.ldif
|
|
```
|
|
|
|
[← Back to Home](index.html) |