Two real bugs, both only surfaced by the live two-container e2e test
(docker-compose.multisite-e2e.yml), not by inspection:
1. POST /site-promote's god_admin check read req.user.groups -- a field
nothing in the codebase ever populates (Auth.checkToken returns
User.get(), which has no .groups; every other admin gate resolves
membership live via permission.byGroup()/Group.list(user.dn), which
also handles nested-group membership). The check silently evaluated to
an empty array on every request, so site-promote returned 403 for
every user, including a real god_admin -- unusable since it shipped in
v2.0.0. Fixed to use permission.byGroup(), the same pattern used
elsewhere in this file and in api_site.js.
2. The read-only write-gate middleware (api_directory_admin.js) is
registered before router.post('/site-promote', ...) later in the same
file, so on a spoke it 403'd every promotion attempt before the
handler ever ran -- the one mutating request a spoke must be able to
make to itself. Exempted /site-promote from the gate.
Added coordinated demotion (MULTI_SITE_SPEC.md §3.2 -- promotion as ONE
action, never a two-step gap with two masters): site-promote now calls
the previous master's new POST /api/site/demote (Bearer the join key it
already holds, handing over a freshly-minted key for the demoted node's
own future use) before flipping itself to master. Best-effort: an
unreachable old master never blocks a god_admin's local promotion (the
WAN-outage scenario is the entire reason this control exists), it's
just reported in the response for manual reconciliation.
e2e test extended to promote the spoke, verify the old master was
actually demoted (isMaster:false, masterUrl pointing at the new master),
and verify writes now succeed on the new master and 403 on the old one.
Full chain verified passing: join -> live replication -> promotion ->
demotion -> write authority follows the promotion.
The shipped join flow (v2.2.0-v2.3.0) was a one-time snapshot: a spoke's
catalog never updated after joining. This adds the two pieces that were
explicitly designed but missing:
- Live replication: a spoke registers its own endpoint with the master
right after joining (POST /api/site/spokes, Bearer join-key), receiving
a pushToken. Every successful catalog write on the master now fires a
fire-and-forget resync ping (utils/site_replicate.js) at every known
spoke, concurrently -- one unreachable spoke never blocks or delays
another (wired into the existing write-gate middleware in
api_directory_admin.js). The spoke's POST /api/site/resync handler
reuses the already-tested export+import path rather than applying a
partial diff.
- Identical directories: POST /api/site/export now best-effort includes
the master's agent-signing key; a spoke adopts it via agent_keys.adopt()
on both join and every resync, so every site's sso-manager can validly
sign a command for any agent enrolled anywhere -- the accepted tradeoff
discussed for this deployment's scale (blast radius for simplicity).
New SiteSpoke model tracks registered spokes (endpoint + pushToken);
registered it in models/index.js (a real bug the e2e test below caught --
SiteSpoke.list() 500'd with "Cannot read properties of null (reading
'adapter')" until the model was added to initORM's model list).
Verified end-to-end against docker-compose.multisite-e2e.yml: mint join
key -> join with selfUrl -> write a NEW resource on master post-join ->
poll the spoke -> it shows up within a few seconds via the resync push,
no manual re-join needed. MULTISITE E2E PASS.
Unit tests: nodejs/tests/site_replicate.test.js (concurrent fan-out, one
failing spoke doesn't block another, empty-registry and list()-throws
edge cases).
docker-compose.multisite-e2e.yml boots two full all-in-one instances
(master + spoke, each with bundled slapd) and a client that drives the
actual HTTP API: seeds admins, mints a join key, joins the spoke, and
verifies the spoke persisted its role across restart, adopted the
pre-join catalog, is enforced read-only, and reports WAN health.
Verified passing locally. Existing docker-compose.test.yml/e2e.yml split
ldap+redis+app into separate containers, which doesn't work here --
POST /api/site/export runs slapcat in-process, so master and spoke each
need their own bundled slapd (Dockerfile.openldap), not a shared one.
See CHANGELOG.md for the full breakdown. Summary:
- POST /api/v1/ldap/{bind,search}: LDAP-over-HTTPS so a client stops
speaking raw LDAP and instead calls the SSO, which binds/searches its
own OpenLDAP on the caller's behalf (DESIGN.md §3).
- LDAP byte-pump relay (utils/ldap_tunnel.js): forwards raw LDAP bytes
from an agent's local socket into OpenLDAP over the existing agent WSS
channel; the SSO never parses LDAP (DESIGN.md §4).
- POST /api/v1/agent/secrets: node-scoped OpenBao secret fetch for
agents, enforced to each agent's own secret/data/nodes/<id>/* prefix
(DESIGN.md §5).
- iam_apply signed command: push node-scoped IAM config (sudo rules, SSH
keys, access control, revocation) to an agent (DESIGN.md §6).
- Agent capability badges on the Directory Metrics tab, sourced from the
agent's own discovery frame.
- Join key management: GET /api/agent/join-keys/:id/agents (which hosts
enrolled through a key) plus a Manage join keys table in the Install
Agent modal with Revoke/Delete actions, confirmed inline per-row rather
than a blocking native confirm() or the shared app.messages.confirm()
banner (which desyncs across concurrent rows -- see CHANGELOG).
- docs/agents.md: capability matrix updated for the three new
capabilities, a full secrets-engine walkthrough with screenshots
(bash + Node consuming a rendered secret, plus the direct-API
alternative), and the join-key reuse/UI/audit questions answered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
docker-compose.test.yml spins up the all-in-one OpenLDAP image, a
standalone Redis, and a test-runner that seeds the test user and runs
jest against them. globalSetup honors REDIS_URL; tests/setup.js
initializes the ORM and flushes test Redis keys before the run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wmantly