Compare commits

..

8 Commits

Author SHA1 Message Date
wmantly 3450ef1a8a Merge pull request #218 from theta42/release-v2.8.0
CI/CD / docker-push (push) Has been skipped
CI/CD / build-theta-agent (push) Successful in 40s
release(v2.8.0): promotion/LDAP-orphan fix, site-slug unification, LDAP status UI
2026-08-10 21:03:27 -07:00
wmantly 1c9d9fe405 release(v2.8.0): promotion/LDAP-orphan fix, site-slug unification, LDAP status UI 2026-08-11 00:02:23 -04:00
wmantly 56e76e1a40 Merge pull request #217 from theta42/docs-multi-site-staleness
docs: fix stale multi-site claims, document LDAP MMR auto-config
2026-08-10 20:33:25 -07:00
wmantly 5abd5296f3 docs: fix stale multi-site claims, document LDAP MMR auto-config + CFG_PUBLIC_DOMAIN
The published docs site had drifted from what actually shipped:

- docs/sso/multi-site.md said the no-inbound relay was "designed but
  not automated" -- it's been automated since earlier this session.
  Also said "don't combine" multi-site join and LDAP replication --
  they're integrated now (join auto-configures LDAP MMR).
- docs/sso/replication.md (the actually-published/linked replication
  page -- distinct from theta-directory's own docs/replication.md,
  which isn't linked from this site's nav at all) still only described
  the old fully-manual LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS setup,
  with zero mention of the new auto-config or CFG_LDAP_MMR_MANUAL.
- docs/index.md's feature bullet described multi-site purely as "N-Way
  Multi-Master LDAP replication" -- the actual master/spoke join
  feature (the more commonly-used, higher-level mechanism) wasn't
  mentioned on the homepage at all.
- CFG_PUBLIC_DOMAIN (shipped, never documented anywhere an operator
  would read it) now explained in multi-site.md.
- spoke.env now discoverable from quickstart.md, not just multi-site.md.

Also documents the real, load-bearing limitation from this session's
promotion/LDAP-orphan fix: the master's own replication peer list only
updates on its own next setup.sh run, not live the instant a spoke
joins or a promotion happens.
2026-08-10 23:32:19 -04:00
wmantly 5166859f29 Merge pull request #216 from theta42/release-v2.7.0
CI/CD / docker-push (push) Has been skipped
CI/CD / build-theta-agent (push) Successful in 42s
release(v2.7.0): auto-configured OpenLDAP multi-master replication
2026-08-10 20:10:37 -07:00
wmantly f05fb27e5e release(v2.7.0): auto-configured OpenLDAP multi-master replication 2026-08-10 23:08:49 -04:00
wmantly 33dfb682f4 Merge pull request #215 from theta42/feat-ldap-mmr-auto-config
feat(multi-site): auto-configure OpenLDAP replication on join/every run
2026-08-10 20:03:04 -07:00
wmantly da60310834 feat(multi-site): auto-configure OpenLDAP replication on join/every run
Removes LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS as vars an operator has
to hand-set and keep in sync across every site. bootstrap/
site-ldap-register.js (new) asks sso-manager-node's new
GET /api/site/ldap-peers (spoke) or GET /directory-admin/
ldap-replication-config (master) for this node's assigned ServerID +
current peer list, persists it to /config/ldap-replication.env, and
restarts sso-manager only when the computed config actually changed
(OpenLDAP's static slapd.conf is only read at process start). Runs on
every setup.sh invocation -- both master (peer list grows as spokes
join) and spoke.

CFG_LDAP_MMR_MANUAL=true skips the automatic step entirely, for a
topology outside this theta-suite cluster the script can't derive on
its own -- without this escape hatch, an operator's hand-set
LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS would get silently overwritten
on the next run, since every fresh install starts as a master (the
automatic step always runs by default).

Bumps sso-manager-node to pick up the new endpoints + SiteSpoke.ldapServerId.
2026-08-10 23:02:01 -04:00
10 changed files with 331 additions and 38 deletions
+47
View File
@@ -9,6 +9,53 @@ orchestration code; see each submodule's own `CHANGELOG.md`
[jump-host](https://github.com/theta42/jump-host/blob/master/CHANGELOG.md)) [jump-host](https://github.com/theta42/jump-host/blob/master/CHANGELOG.md))
for what changed inside the apps it composes. for what changed inside the apps it composes.
## [v2.8.0] - 2026-08-11
Rolls up **sso-manager-node v2.8.0**. Fixes found while auditing the v2.7.0
multi-site work for gaps, plus the published docs staleness that same audit
turned up.
### sso-manager-node v2.8.0
- Promotion no longer orphans the demoted old master's LDAP replication --
`/demote` now registers itself with the new master immediately instead of
being left with no way to ever get a real `ldapServerId` again.
- The Directory's site slug and the multi-site replication identity are
unified -- previously unrelated values that happened to share a name.
- LDAP replication status (real vs. advertised ServerID, a `stale` flag) and
per-spoke detail (not just an aggregate count) on the Multi-Site modal.
### theta-suite docs
- Fixed multiple stale/contradictory claims on the published site: the
no-inbound relay described as "designed but not automated" (shipped
earlier this session), `docs/sso/replication.md` never mentioning the new
LDAP MMR auto-config at all, and the homepage feature list only
describing the old manual N-way replication.
## [v2.7.0] - 2026-08-11
Rolls up **sso-manager-node v2.7.0**. Closes the last "operator hand-sets
this" item on the multi-site TODO: OpenLDAP N-way multi-master replication
now configures itself.
### theta-suite orchestration
- **`bootstrap/site-ldap-register.js`**: runs on every `setup.sh` invocation
(master and spoke), fetches this node's auto-assigned `LDAP_SERVER_ID` +
current peer list from sso-manager-node's new endpoints, and restarts
`sso-manager` only when the computed config actually changed.
- `CFG_LDAP_MMR_MANUAL=true` skips the automatic step for a topology outside
this cluster the script can't derive on its own -- otherwise it always
runs (every fresh install starts as a master) and would overwrite
hand-set values.
- `setup.env.example`'s old `#LDAP_SERVER_ID=1`/`#LDAP_REPLICATION_HOSTS=`
manual-config prompt is gone for the common case.
### sso-manager-node v2.7.0
- `SiteSpoke.ldapServerId` auto-assigned at registration (same pattern as
jump-host's mesh index); each site's LDAP URL derived from its
already-known HTTP(S) endpoint. New `GET /api/site/ldap-peers`
(spoke-facing) and `GET /directory-admin/ldap-replication-config`
(master-local). Verified against real running containers.
## [v2.6.0] - 2026-08-11 ## [v2.6.0] - 2026-08-11
Rolls up **sso-manager-node v2.6.0** and **jump-host v2.1.1** (already Rolls up **sso-manager-node v2.6.0** and **jump-host v2.1.1** (already
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env node
/*
* theta-suite site-ldap-register — runs inside the sso-manager container on
* every setup.sh run (both master and spoke) to keep OpenLDAP N-way
* multi-master replication config (docs/replication.md) in sync without an
* operator hand-maintaining LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS.
*
* The master assigns each spoke a unique LDAP_SERVER_ID at join time (same
* mechanism as jump-host's WireGuard mesh index) and derives every site's
* LDAP URL from its already-known HTTPS endpoint -- see sso-manager-node's
* GET /api/site/ldap-peers (spoke-facing) and
* GET /directory-admin/ldap-replication-config (master-local).
*
* This script fetches whichever of those two applies to this node's role,
* and writes the result to /config/ldap-replication.env (KEY=VALUE, the
* same shape setup.env/spoke.env use) if it changed since last run. setup.sh
* sources that file before starting sso-manager on every invocation, and
* restarts the container when this script reports a change -- OpenLDAP's
* static slapd.conf is only read at process start, so a config change needs
* a restart to take effect; there's no live push, which is why this has to
* be re-run periodically (every setup.sh invocation) rather than working
* once at join time and never again, especially on the MASTER, whose peer
* list changes every time a new spoke joins.
*
* docker compose exec sso-manager node /bootstrap/site-ldap-register.js <selfUrl>
*
* Self-contained (Node built-ins + global fetch), same rule as
* bootstrap.js/site-join.js -- does NOT require the SSO's internal models.
*
* Output (stdout, KEY=VALUE for setup.sh): LDAP_CONFIG_CHANGED=<yes|no>,
* LDAP_SERVER_ID=<n>, LDAP_REPLICATION_HOSTS=<space-separated, may be empty>.
* Progress logs go to stderr.
*/
'use strict';
const fs = require('fs');
const SITE_CONFIG = '/config/site.json';
const LDAP_CONFIG_FILE = '/config/ldap-replication.env';
const SSO_INTERNAL = 'http://localhost:3001';
const selfUrl = process.argv[2];
function log(msg) { console.error('[site-ldap-register] ' + msg); }
function readPersisted() {
if (!fs.existsSync(LDAP_CONFIG_FILE)) return { LDAP_SERVER_ID: '', LDAP_REPLICATION_HOSTS: '' };
const out = { LDAP_SERVER_ID: '', LDAP_REPLICATION_HOSTS: '' };
for (const line of fs.readFileSync(LDAP_CONFIG_FILE, 'utf8').split('\n')) {
const m = line.match(/^([A-Z_]+)=(.*)$/);
if (m && m[1] in out) out[m[1]] = m[2];
}
return out;
}
async function fetchMasterConfig() {
const sso = require('/config/sso-secrets.js');
const adminUid = (sso.bootstrap && sso.bootstrap.adminUid) || 'admin';
const adminPass = (sso.bootstrap && sso.bootstrap.adminPass) || '';
const loginRes = await fetch(`${SSO_INTERNAL}/api/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ uid: adminUid, password: adminPass }),
});
if (!loginRes.ok) throw new Error(`local admin login failed (${loginRes.status}): ${await loginRes.text().catch(() => '')}`);
const { token } = await loginRes.json();
if (!token) throw new Error('local admin login returned no token');
const cfgRes = await fetch(`${SSO_INTERNAL}/api/directory-admin/ldap-replication-config`, {
headers: { 'auth-token': token },
});
if (!cfgRes.ok) throw new Error(`ldap-replication-config failed (${cfgRes.status}): ${await cfgRes.text().catch(() => '')}`);
return cfgRes.json();
}
async function fetchSpokeConfig(site, selfUrl) {
const url = `${site.masterUrl.replace(/\/+$/, '')}/api/site/ldap-peers?endpoint=${encodeURIComponent(selfUrl)}`;
const res = await fetch(url, { headers: { Authorization: 'Bearer ' + site.masterJoinKey } });
const text = await res.text().catch(() => '');
let data = null;
try { data = JSON.parse(text); } catch (e) { /* not JSON */ }
if (!res.ok) {
if (res.status === 404) {
log('This site is not registered as a spoke on the master yet (join with selfUrl, or re-run site-relay-register.js). Skipping.');
return null;
}
throw new Error(`ldap-peers failed (${res.status}): ${(data && data.message) || text}`);
}
return data;
}
async function main() {
if (!fs.existsSync(SITE_CONFIG)) {
log('No /config/site.json yet. Skipping.');
console.log('LDAP_CONFIG_CHANGED=no');
return;
}
const site = JSON.parse(fs.readFileSync(SITE_CONFIG, 'utf8'));
let result;
if (site.isMaster) {
result = await fetchMasterConfig();
} else {
if (!site.masterUrl || !site.masterJoinKey) {
log('Spoke role but missing masterUrl/masterJoinKey. Skipping.');
console.log('LDAP_CONFIG_CHANGED=no');
return;
}
if (!selfUrl) throw new Error('usage: node /bootstrap/site-ldap-register.js <selfUrl> (required for a spoke)');
result = await fetchSpokeConfig(site, selfUrl);
if (!result) {
console.log('LDAP_CONFIG_CHANGED=no');
return;
}
}
const serverId = String(result.ldapServerId || '');
const hosts = (result.peers || []).map((p) => p.ldapHost).filter(Boolean).join(' ');
const before = readPersisted();
const changed = before.LDAP_SERVER_ID !== serverId || before.LDAP_REPLICATION_HOSTS !== hosts;
if (changed) {
fs.writeFileSync(LDAP_CONFIG_FILE, `LDAP_SERVER_ID=${serverId}\nLDAP_REPLICATION_HOSTS=${hosts}\n`);
log(`Replication config changed -- ServerID ${serverId}, ${(result.peers || []).length} peer(s). Wrote ${LDAP_CONFIG_FILE}.`);
} else {
log(`Replication config unchanged -- ServerID ${serverId}, ${(result.peers || []).length} peer(s).`);
}
console.log(`LDAP_CONFIG_CHANGED=${changed ? 'yes' : 'no'}`);
console.log(`LDAP_SERVER_ID=${serverId}`);
console.log(`LDAP_REPLICATION_HOSTS=${hosts}`);
}
main().catch((e) => {
console.error('[site-ldap-register] FAILED: ' + e.message);
process.exit(1);
});
+1
View File
@@ -259,6 +259,7 @@ See [`AGENT_LOCAL_DISCOVERY_SPEC.md`](./AGENT_LOCAL_DISCOVERY_SPEC.md) — split
| `setup.env` / `setup.sh` join wiring | **Shipped**`CFG_MASTER_DIRECTORY_URL` / `CFG_MASTER_DIRECTORY_JOIN_KEY`, `bootstrap/site-join.js` (theta-suite v2.2.0). Also readable from a dedicated `spoke.env` (`spoke.env.example`, layered on top of `setup.env`) for operators who want join-a-cluster config kept separate from the rest of first-run setup. | | `setup.env` / `setup.sh` join wiring | **Shipped**`CFG_MASTER_DIRECTORY_URL` / `CFG_MASTER_DIRECTORY_JOIN_KEY`, `bootstrap/site-join.js` (theta-suite v2.2.0). Also readable from a dedicated `spoke.env` (`spoke.env.example`, layered on top of `setup.env`) for operators who want join-a-cluster config kept separate from the rest of first-run setup. |
| Continuous/live replication (vs. one-time export-on-join) | **Shipped** (`sso-manager-node`) — a spoke registers its own endpoint at join time (`POST /api/site/spokes`), and every successful master catalog write fires a fire-and-forget push (`utils/site_replicate.js`) at every registered spoke, which re-pulls a fresh export. Verified end-to-end in `docker-compose.multisite-e2e.yml`. | | Continuous/live replication (vs. one-time export-on-join) | **Shipped** (`sso-manager-node`) — a spoke registers its own endpoint at join time (`POST /api/site/spokes`), and every successful master catalog write fires a fire-and-forget push (`utils/site_replicate.js`) at every registered spoke, which re-pulls a fresh export. Verified end-to-end in `docker-compose.multisite-e2e.yml`. |
| Identical-directory signing key | **Shipped**`POST /api/site/export` includes the master's agent-signing key; a spoke adopts it via `agent_keys.adopt()` on join and every resync. OpenBao secret replication *beyond* this one key is still not built. | | Identical-directory signing key | **Shipped**`POST /api/site/export` includes the master's agent-signing key; a spoke adopts it via `agent_keys.adopt()` on join and every resync. OpenBao secret replication *beyond* this one key is still not built. |
| OpenLDAP N-way multi-master replication auto-config | **Shipped** — the master auto-assigns each spoke a unique `LDAP_SERVER_ID` at registration (`SiteSpoke.ldapServerId`, same pattern as jump-host's mesh index) and derives every site's LDAP URL from its already-known HTTPS endpoint; `theta-suite`'s `bootstrap/site-ldap-register.js` applies it, re-checked on every `setup.sh` run since the peer list grows as spokes join. Verified against real running containers. Known gap: the master's own config only updates when ITS `setup.sh` is re-run, not live the moment a new spoke joins (see `docs/replication.md`). |
| Coordinated master promotion (demote the old master as one action) | **Shipped**`POST /api/site/demote` + `site-promote`'s handoff logic. Fixed two real pre-existing bugs while wiring this in: `site-promote`'s god_admin check read a `req.user.groups` field nothing ever populated (permanently 403'd for everyone), and the read-only write-gate 403'd `site-promote` itself before the handler could run. | | Coordinated master promotion (demote the old master as one action) | **Shipped**`POST /api/site/demote` + `site-promote`'s handoff logic. Fixed two real pre-existing bugs while wiring this in: `site-promote`'s god_admin check read a `req.user.groups` field nothing ever populated (permanently 403'd for everyone), and the read-only write-gate 403'd `site-promote` itself before the handler could run. |
| WireGuard gateway-to-gateway mesh (`theta-gateway`) | **Shipped**`POST /api/mesh/register`/`/join` (join-token bootstrap), `utils/wg_iface.js` (kernel WireGuard, falls back to userspace `wireguard-go`). Verified with a real two-container test: actual encrypted tunnel, real ICMP traffic across it, 0% loss. `wg_iface.removePeer()` also cleans up the kernel routes `setPeer()` added (verified live: routes present after `setPeer`, gone after `removePeer`, own local route untouched), and `DELETE /api/mesh/gateways/:id` exposes it from the mesh UI. | | WireGuard gateway-to-gateway mesh (`theta-gateway`) | **Shipped**`POST /api/mesh/register`/`/join` (join-token bootstrap), `utils/wg_iface.js` (kernel WireGuard, falls back to userspace `wireguard-go`). Verified with a real two-container test: actual encrypted tunnel, real ICMP traffic across it, 0% loss. `wg_iface.removePeer()` also cleans up the kernel routes `setPeer()` added (verified live: routes present after `setPeer`, gone after `removePeer`, own local route untouched), and `DELETE /api/mesh/gateways/:id` exposes it from the mesh UI. |
| Cross-component routing (replication over the mesh) | **Shipped**`utils/site_replicate.js` tries a registered spoke's `meshIp` first (falling back to its public `endpoint` on failure) when pushing resync pings; a spoke with no `meshIp` on file behaves exactly as before. | | Cross-component routing (replication over the mesh) | **Shipped**`utils/site_replicate.js` tries a registered spoke's `meshIp` first (falling back to its public `endpoint` on failure) when pushing resync pings; a spoke with no `meshIp` on file behaves exactly as before. |
+6 -4
View File
@@ -74,10 +74,12 @@ The SSO Manager and the proxy it fronts, both stood up by one `./setup.sh` run:
management built directly into the web interface, allowing operators to drive management built directly into the web interface, allowing operators to drive
system administration and automation pipelines programmatically without an system administration and automation pipelines programmatically without an
active browser session. active browser session.
- **Multi-Site Geo-Replication**: Built-in support for N-Way Multi-Master LDAP - **Multi-Site Geo-Replication**: A master site and any number of spokes join
replication, allowing directory states to sync across geographically separated with one key and one URL, staying in sync automatically (no manual LDAP
physical hardware or remote data centers for high availability and low-latency config) — see [Multi-Site (Master/Spoke Join)](sso/multi-site.html). Raw
local reads. N-Way Multi-Master LDAP replication is also available directly for
deployments that want every site independently writable with no
master/spoke concept — see [Geo-Location Scaling](sso/replication.html).
- **Multi-Target Load Balancing**: Native reverse-proxy load balancing that - **Multi-Target Load Balancing**: Native reverse-proxy load balancing that
distributes traffic across multiple application backends using customizable distributes traffic across multiple application backends using customizable
health checks and round-robin strategies. health checks and round-robin strategies.
+5
View File
@@ -74,6 +74,11 @@ file shape.
> exist, `./setup.sh` migrates them into `./config/` preserving your existing > exist, `./setup.sh` migrates them into `./config/` preserving your existing
> secrets — no need to write a `setup.env`. > secrets — no need to write a `setup.env`.
> **Joining an existing Theta Directory cluster instead of seeding a fresh
> one?** See [Multi-Site (Master/Spoke Join)](sso/multi-site.html) —
> `spoke.env.example` has the join-a-cluster vars split out into their own
> file, or set them directly in `setup.env` (which has every option).
--- ---
## 3. Run ## 3. Run
+42 -17
View File
@@ -10,10 +10,10 @@ the **master** (single write authority for the shared catalog) and any
number of **spokes** — read-only replicas that stay in sync automatically and number of **spokes** — read-only replicas that stay in sync automatically and
run local authentication with zero WAN dependency. run local authentication with zero WAN dependency.
This is a different, higher-level mechanism than [raw LDAP N-way This is a higher-level mechanism than [raw LDAP N-way
replication](replication.html) — see [How this relates to LDAP replication](replication.html) — and it now drives that lower-level
replication](#how-this-relates-to-ldap-replication) below if you're deciding replication for you automatically. See [How this relates to LDAP
between the two. replication](#how-this-relates-to-ldap-replication) below.
## Why and when to use this ## Why and when to use this
@@ -45,6 +45,13 @@ between the two.
stack's setup. Both are read; `spoke.env`'s values win on a conflict. stack's setup. Both are read; `spoke.env`'s values win on a conflict.
No public IP on this site at all? `spoke.env.example` also covers the No public IP on this site at all? `spoke.env.example` also covers the
no-inbound relay vars (`CFG_SPOKE_NO_INBOUND`/`CFG_SPOKE_PUBLIC_HOST`). no-inbound relay vars (`CFG_SPOKE_NO_INBOUND`/`CFG_SPOKE_PUBLIC_HOST`).
Want this spoke reachable at its own public domain rather than sharing
the master's? `CFG_DOMAIN` (the LDAP identity namespace) must stay
identical across every site in a cluster — MMR replicas can't diverge
on base DN — but `CFG_PUBLIC_DOMAIN` overrides just this site's own web
hostnames (`sso.*`/`proxy.*`) independently of it. Only meaningful for
an inbound spoke serving its own traffic directly.
3. The spoke pulls the master's full export (LDAP tree, resource catalog, 3. The spoke pulls the master's full export (LDAP tree, resource catalog,
agent-signing key) and adopts it, then registers its own reachable URL agent-signing key) and adopts it, then registers its own reachable URL
with the master so it can receive live updates going forward. with the master so it can receive live updates going forward.
@@ -82,26 +89,44 @@ so on) are **not** currently synced — each site still generates its own.
- Both sites need a network path to each other's HTTP(S) API — the master to - Both sites need a network path to each other's HTTP(S) API — the master to
pull an export from, the spoke to push replication updates back to. A site pull an export from, the spoke to push replication updates back to. A site
with no inbound path at all (e.g. behind CGNAT) can't join yet on its own; with **no inbound path at all** (e.g. behind CGNAT) can still join: set
a relay mechanism for that case is designed but not automated (see the `CFG_SPOKE_NO_INBOUND=true` + `CFG_SPOKE_PUBLIC_HOST` (`spoke.env.example`)
[architecture spec](https://github.com/theta42/theta-suite/blob/master/docs/MULTI_SITE_SPEC.md) once its jump-host is meshed to the master's over WireGuard (mesh peering
for the current status). itself is a manual, one-time step on both jump-hosts — see [Theta Gateway
→ Mesh](../jump-host/mesh.html)) — the master then relays traffic to it
and auto-creates the matching route on its own `theta-proxy`. A spoke with
**zero inbound and zero outbound** path still can't join at all (the join
itself needs to reach the master's API directly).
- Joining only ever happens on a **fresh install**. There's no way to merge - Joining only ever happens on a **fresh install**. There's no way to merge
an already-populated directory into a master's — re-provision the host an already-populated directory into a master's — re-provision the host
first. first.
- Promoting a spoke to master doesn't instantly finish reconciling OpenLDAP
replication (see below) — re-run `setup.sh` on the newly-promoted node
promptly afterward.
## How this relates to LDAP replication ## How this relates to LDAP replication
[N-way LDAP replication](replication.html) is a *lower-level*, different [N-way LDAP replication](replication.html) is the *lower-level* mechanism
mechanism: every site runs a fully independent, fully writable `slapd`, wired underneath this: `slapd`'s own `syncrepl`, wired via `LDAP_SERVER_ID` +
together with raw `syncrepl` environment variables, and there's no concept of `LDAP_REPLICATION_HOSTS`. Originally this was hand-configured by the
a master or a managed join. It predates this feature and is still there for operator, separately from the join flow above, for deployments that wanted
deployments that specifically want every site independently writable. every site independently writable with no concept of a master.
Multi-site join (this page) is the opposite design: one write authority, a **When you join via this page's flow, that lower-level config is now handled
managed onboarding flow, and automatic ongoing sync — closer to what most for you.** The master auto-assigns each spoke a unique `LDAP_SERVER_ID` at
"add a second office" or "add a home-lab spoke" setups actually want. **Don't join time and derives every site's LDAP URL from its already-known HTTPS
combine the two** — pick one per deployment. endpoint — `theta-suite`'s `bootstrap/site-ldap-register.js` applies it,
re-checked on every `setup.sh` run since the peer list grows as spokes join.
You don't hand-set `LDAP_SERVER_ID`/`LDAP_REPLICATION_HOSTS` for a cluster
built this way. See [Geo-Location Scaling](replication.html#automatic-config-via-multi-site-join)
for the mechanics, and its documented limitation: the *master's* own
replication list only updates on ITS next `setup.sh` run, not live the
instant a new spoke joins.
Still want fully independent, always-writable sites with no master/spoke
concept at all? `CFG_LDAP_MMR_MANUAL=true` opts out of the automatic path so
you can hand-set `LDAP_SERVER_ID`/`LDAP_REPLICATION_HOSTS` directly, same as
before this integration existed.
## See also ## See also
+34 -9
View File
@@ -23,33 +23,58 @@ In an N-Way Multi-Master setup, every site runs a fully active OpenLDAP server (
## Configuration ## Configuration
To enable replication, you must pass two environment variables to the `sso-manager` container: The container's entrypoint reads two environment variables to configure this
-- `LDAP_SERVER_ID` (a unique integer for this node) and
`LDAP_REPLICATION_HOSTS` (a space-separated list of every **other** node's
LDAP URL) -- and, when both are set, automatically loads the `syncprov`
module, enables `mirrormode`, and generates the necessary `syncrepl` blocks
in `/etc/openldap/slapd.conf`.
1. `LDAP_SERVER_ID`: A unique integer for this node (e.g., `1`, `2`, `3`). This MUST be unique across the cluster. ### Automatic config via Multi-Site join
2. `LDAP_REPLICATION_HOSTS`: A space-separated list of the LDAP URLs of all **other** nodes in the cluster.
### Example using `theta-env` / Docker Compose **If you're using [Multi-Site join](multi-site.html) (`CFG_MASTER_DIRECTORY_URL`/`spoke.env`),
you don't set these by hand.** The master assigns each spoke a unique
`LDAP_SERVER_ID` at join time (the same way it assigns a WireGuard mesh
index) and derives `LDAP_REPLICATION_HOSTS` from every site's already-known
HTTPS endpoint (`ldaps://<same-host>:636`) -- `theta-suite`'s `bootstrap/
site-ldap-register.js` applies it and re-checks on every `setup.sh` run,
since the peer list grows as new spokes join, restarting `sso-manager` only
when the computed config actually changed.
**Site 1 (`setup.env` or `docker-compose.yml`)** **Known limitation**: the *master's* own `LDAP_REPLICATION_HOSTS` only gets
recomputed when its `setup.sh` is re-run — there's no live push telling an
already-running master about a spoke that joined five minutes ago. Re-run
`setup.sh` on the master after bringing up a new spoke (or after promoting
one to master) to pick up the current peer list. A spoke's own config, by
contrast, is re-checked and applied on every `setup.sh` run there, which is
the common/recurring event.
### Manual configuration
Have a topology outside a `theta-suite`-managed cluster (fully independent,
always-writable sites, no master/spoke concept)? Set
`CFG_LDAP_MMR_MANUAL=true` to skip the automatic path entirely and set the
two variables directly -- without this, the automatic step runs on every
deployment (every fresh install starts as a master) and will overwrite them.
**Site 1**
```env ```env
LDAP_SERVER_ID=1 LDAP_SERVER_ID=1
LDAP_REPLICATION_HOSTS="ldaps://sso.site2.com:636 ldaps://sso.site3.com:636" LDAP_REPLICATION_HOSTS="ldaps://sso.site2.com:636 ldaps://sso.site3.com:636"
``` ```
**Site 2 (`setup.env` or `docker-compose.yml`)** **Site 2**
```env ```env
LDAP_SERVER_ID=2 LDAP_SERVER_ID=2
LDAP_REPLICATION_HOSTS="ldaps://sso.site1.com:636 ldaps://sso.site3.com:636" LDAP_REPLICATION_HOSTS="ldaps://sso.site1.com:636 ldaps://sso.site3.com:636"
``` ```
**Site 3 (`setup.env` or `docker-compose.yml`)** **Site 3**
```env ```env
LDAP_SERVER_ID=3 LDAP_SERVER_ID=3
LDAP_REPLICATION_HOSTS="ldaps://sso.site1.com:636 ldaps://sso.site2.com:636" LDAP_REPLICATION_HOSTS="ldaps://sso.site1.com:636 ldaps://sso.site2.com:636"
``` ```
Once configured, the container's entrypoint will automatically load the `syncprov` module, enable `mirrormode`, and generate the necessary `syncrepl` blocks in `/etc/openldap/slapd.conf`.
## User Locations ## User Locations
When creating or editing a user, you can specify their **Location (Site)**. This maps directly to the standard LDAP `l` (localityName) attribute, allowing you to track which physical site a user belongs to natively within the directory. When creating or editing a user, you can specify their **Location (Site)**. This maps directly to the standard LDAP `l` (localityName) attribute, allowing you to track which physical site a user belongs to natively within the directory.
+13 -7
View File
@@ -152,14 +152,20 @@ CFG_DOMAIN=example.com
#CFG_THETA_AGENT_FULL_CONTROL=1 #CFG_THETA_AGENT_FULL_CONTROL=1
# ── Geo-Location Scaling (N-Way Multi-Master LDAP) ─────────────────────────── # ── Geo-Location Scaling (N-Way Multi-Master LDAP) ───────────────────────────
# If deploying this stack across multiple physical sites to provide local HA # If you're joining a directory cluster (CFG_MASTER_DIRECTORY_URL/spoke.env
# for directory services, you can enable N-Way Multi-Master OpenLDAP replication. # above), N-Way Multi-Master OpenLDAP replication is configured for you
# This requires assigning a unique ID to each site and listing the LDAPS URLs # automatically -- setup.sh's bootstrap/site-ldap-register.js asks the master
# of all OTHER sites in the cluster. # for a unique LDAP_SERVER_ID and the current list of every other site's LDAP
# URL on every run (see docs/replication.md), restarting sso-manager only
# when that config actually changed. Nothing to set here for the common case.
# #
# Each site MUST have a unique LDAP_SERVER_ID (e.g. 1, 2, 3). # Have a manually-coordinated LDAP MMR topology this script can't derive on
# LDAP_REPLICATION_HOSTS is a space-separated list of the other sites' LDAP URLs. # its own (e.g. peers outside this theta-suite cluster)? Set
# Example for Site 1: # CFG_LDAP_MMR_MANUAL=true to skip the automatic step entirely and set
# LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS directly -- without this, the
# automatic step runs on every deployment (every fresh install starts as a
# master) and will overwrite them.
#CFG_LDAP_MMR_MANUAL=true
#LDAP_SERVER_ID=1 #LDAP_SERVER_ID=1
#LDAP_REPLICATION_HOSTS="ldaps://sso.site2.com:636 ldaps://sso.site3.com:636" #LDAP_REPLICATION_HOSTS="ldaps://sso.site2.com:636 ldaps://sso.site3.com:636"
# ── Proxy HTTP/HTTPS Defaults ──────────────────────────────────────────────── # ── Proxy HTTP/HTTPS Defaults ────────────────────────────────────────────────
+43
View File
@@ -1060,6 +1060,15 @@ info "Starting bao-renewer (service-token renewal sidecar)..."
SSO_GIT_COMMIT="$(git -C sso-manager-node rev-parse --short HEAD 2>/dev/null || echo unknown)" SSO_GIT_COMMIT="$(git -C sso-manager-node rev-parse --short HEAD 2>/dev/null || echo unknown)"
export SSO_GIT_COMMIT export SSO_GIT_COMMIT
env_upsert SSO_GIT_COMMIT "$SSO_GIT_COMMIT" env_upsert SSO_GIT_COMMIT "$SSO_GIT_COMMIT"
# OpenLDAP multi-master replication (docs/replication.md, auto-configured --
# see step 7e below and bootstrap/site-ldap-register.js): pick up whatever
# LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS a PRIOR run already computed, so a
# restart doesn't silently drop back to standalone (no LDAP_SERVER_ID env at
# all). A truly fresh install has no file yet -- that's fine, it just starts
# standalone until step 7e computes and applies real values. Skipped under
# CFG_LDAP_MMR_MANUAL=true so a manually hand-set LDAP_SERVER_ID/
# LDAP_REPLICATION_HOSTS in setup.env isn't clobbered by a stale auto file.
[[ "${CFG_LDAP_MMR_MANUAL:-false}" != "true" && -f "$CONFIG_DIR/ldap-replication.env" ]] && parse_kv_file "$CONFIG_DIR/ldap-replication.env"
info "Building + starting sso-manager (first run builds the image; this takes a while)..." info "Building + starting sso-manager (first run builds the image; this takes a while)..."
"${COMPOSE[@]}" up -d --build sso-manager "${COMPOSE[@]}" up -d --build sso-manager
@@ -1322,6 +1331,40 @@ if [[ "${CFG_SPOKE_NO_INBOUND:-false}" == "true" ]]; then
fi fi
fi fi
# ── 7e. OpenLDAP multi-master replication auto-config (every run) ────────────
# docs/replication.md: the master assigns each spoke a unique LDAP_SERVER_ID
# and derives every site's LDAP URL automatically (bootstrap/
# site-ldap-register.js) instead of an operator hand-maintaining
# LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS. Runs on every invocation -- both
# master (its peer list grows as spokes join) and spoke -- and restarts
# sso-manager only when the computed config actually changed, since
# OpenLDAP's static slapd.conf is only read at process start.
#
# CFG_LDAP_MMR_MANUAL=true skips this entirely -- every fresh install starts
# as a master, so without this escape hatch an operator's own hand-set
# LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS (a topology outside this theta-suite
# cluster this script can't derive) would get silently overwritten.
if [[ "${CFG_LDAP_MMR_MANUAL:-false}" == "true" ]]; then
info "CFG_LDAP_MMR_MANUAL=true — skipping automatic LDAP replication config."
LDAP_REG_OUT=""
else
LDAP_REG_OUT=$("${COMPOSE[@]}" exec -T sso-manager node /bootstrap/site-ldap-register.js "https://$CFG_SSO_HOST" 2>&1) || warn "LDAP replication config check failed — check: ${COMPOSE[*]} exec sso-manager node /bootstrap/site-ldap-register.js https://$CFG_SSO_HOST"
echo "$LDAP_REG_OUT" | sed 's/^/[setup] /'
fi
if echo "$LDAP_REG_OUT" | grep -q '^LDAP_CONFIG_CHANGED=yes'; then
info "LDAP replication config changed — restarting sso-manager to apply it..."
[[ -f "$CONFIG_DIR/ldap-replication.env" ]] && parse_kv_file "$CONFIG_DIR/ldap-replication.env"
"${COMPOSE[@]}" up -d --force-recreate sso-manager
info "Waiting for sso-manager to be healthy again..."
for i in $(seq 1 60); do
if docker exec sso-manager wget -q -O- http://localhost:3001/health >/dev/null 2>&1; then
info "sso-manager is healthy."; break
fi
if (( i == 60 )); then warn "sso-manager did not become healthy in 120s after the LDAP config restart. Check: ${COMPOSE[*]} logs sso-manager"; break; fi
sleep 2
done
fi
# ── 7c. Install theta-agent on the host ────────────────────────────────────── # ── 7c. Install theta-agent on the host ──────────────────────────────────────
# Controlled by CFG_THETA_AGENT_ENABLE (default: 1 = enabled) # Controlled by CFG_THETA_AGENT_ENABLE (default: 1 = enabled)
CFG_THETA_AGENT_ENABLE="${CFG_THETA_AGENT_ENABLE:-1}" CFG_THETA_AGENT_ENABLE="${CFG_THETA_AGENT_ENABLE:-1}"