The shipped join flow (v2.2.0-v2.3.0) was a one-time snapshot: a spoke's
catalog never updated after joining. This adds the two pieces that were
explicitly designed but missing:
- Live replication: a spoke registers its own endpoint with the master
right after joining (POST /api/site/spokes, Bearer join-key), receiving
a pushToken. Every successful catalog write on the master now fires a
fire-and-forget resync ping (utils/site_replicate.js) at every known
spoke, concurrently -- one unreachable spoke never blocks or delays
another (wired into the existing write-gate middleware in
api_directory_admin.js). The spoke's POST /api/site/resync handler
reuses the already-tested export+import path rather than applying a
partial diff.
- Identical directories: POST /api/site/export now best-effort includes
the master's agent-signing key; a spoke adopts it via agent_keys.adopt()
on both join and every resync, so every site's sso-manager can validly
sign a command for any agent enrolled anywhere -- the accepted tradeoff
discussed for this deployment's scale (blast radius for simplicity).
New SiteSpoke model tracks registered spokes (endpoint + pushToken);
registered it in models/index.js (a real bug the e2e test below caught --
SiteSpoke.list() 500'd with "Cannot read properties of null (reading
'adapter')" until the model was added to initORM's model list).
Verified end-to-end against docker-compose.multisite-e2e.yml: mint join
key -> join with selfUrl -> write a NEW resource on master post-join ->
poll the spoke -> it shows up within a few seconds via the resync push,
no manual re-join needed. MULTISITE E2E PASS.
Unit tests: nodejs/tests/site_replicate.test.js (concurrent fan-out, one
failing spoke doesn't block another, empty-registry and list()-throws
edge cases).
docker-compose.multisite-e2e.yml boots two full all-in-one instances
(master + spoke, each with bundled slapd) and a client that drives the
actual HTTP API: seeds admins, mints a join key, joins the spoke, and
verifies the spoke persisted its role across restart, adopted the
pre-join catalog, is enforced read-only, and reports WAN health.
Verified passing locally. Existing docker-compose.test.yml/e2e.yml split
ldap+redis+app into separate containers, which doesn't work here --
POST /api/site/export runs slapcat in-process, so master and spoke each
need their own bundled slapd (Dockerfile.openldap), not a shared one.
Completes the multi-site join layer on top of the v2.2.0 endpoints:
- UI (Master Site modal): a fresh install (canJoin) gets a 'Join an Existing
Site' form (master URL + stj_ key); a master gets a 'Site Join Keys' manager
(mint/revoke/list, key shown once); WAN Sync Health now reflects a live probe.
- POST /api/site/ping (Bearer stj_ key, no admin session): lightweight master
reachability probe for WAN health (cheap vs /export).
- Spoke read-only: directory-write routes (resources/edges/groups/secrets/
grants/driver-action/discovered) reject with 403 pointing at the master.
- Fresh-install guard: /api/site/join refuses unless no users beyond the
bootstrap admin and no enrolled agents (siteIsFresh), and site-status exposes
canJoin so the UI only offers join on a genuinely fresh install. The
bootstrap's seeded default resources are NOT the signal (they always exist).
- The spoke stores the join key (masterJoinKey) in /config/site.json so WAN
health (and a future write-proxy) can reach the master.
- Tests: siteIsFresh cases in tests/site_join.test.js.
Server endpoints for joining a spoke to a master directory (MULTI_SITE_SPEC.md).
This pass is server-only; setup.sh wiring and the UI are the next layer.
- Site join keys (SiteJoinKey model, stj_ prefix): mint/revoke/delete/list,
hashed at rest, shown once — the same model as agent join keys.
- POST /api/site/export (master, Bearer stj_ key, no admin session): returns the
local LDAP tree (slapcat LDIF) + resource catalog + siteSlug + baseDn.
- POST /api/site/join (spoke, admin): { masterUrl, joinKey } pulls the master
export, imports resources (upsert by slug) + LDAP (ldapadd -c), and persists
the spoke role. Refused if already a spoke.
- Persisted site role: utils/site_config.js keeps isMaster/masterUrl/siteSlug in
/config/site.json (env seeds defaults); site-status/site-promote now use it.
- Unit tests (site_join, site_config) with in-memory stubs, wired into npm test.
- docs/site-join.md + docs router entry.
- Repairs the corrupted multi-site emojis (crown/bolt) in directory.ejs.
- .gitguardian.yml ignores the generic-password false positive on reading the
LDAP bind credential from runtime config (never a hardcoded secret).
Fresh-install bug report fixes:
- 'Master Site' button error 'app.modal.show is not a function': the
multi-site status modal used the legacy app.modal.show() signature; the app
exposes app.modal.open({title, bodyHtml, size}). The site-status request
itself worked -- only the rendering call was wrong.
- Agents with no discovery yet showed a fake 'v2.0.0' (three hardcoded
fallbacks). Now 'unknown', so a host whose agent never connected isn't
presented as an old version.
- The default org name (browser tab title) is set by theta-suite's setup.sh;
that default is fixed separately there (CFG_ORG -> Theta Directory).
The Install Agent modal now emits Windows (PowerShell) one-liners for the
join-key, pre-register and custom-config flows. Each downloads the fully-offline
setup installer and passes the same values the bash flow uses:
join key : /SERVER_URL /JOIN_KEY
quick : /SERVER_URL /AUTH_TOKEN /PUBLIC_KEY
custom : /B64_CONFIG=<base64 agent.yml>
Binaries are no longer committed here: the setup.exe and loose agent binaries
are GitHub release artifacts (built by the theta-agent release workflow) and the
modal downloads the installer from releases/latest/download/. The SSO still
serves install.sh (the small Linux bootstrap script); the large binaries are
dropped from this repo.
ldapbuild now pulls ghcr.io/theta42/openldap-nestgroup:<pinned commit>
instead of compiling OpenLDAP from source on every build.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Extracts Dockerfile.openldap's `ldapbuild` stage (compile OpenLDAP from
source for the nestgroup overlay, ~5 min, dependent on git.openldap.org
being reachable) into its own Dockerfile, built and pushed to
ghcr.io/theta42/openldap-nestgroup by this workflow whenever the pinned
commit changes.
This commit only adds the new image + workflow; Dockerfile.openldap itself
still compiles from source. A follow-up change switches it to FROM the
published image once this workflow has run once and the image exists.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
- GET /api/directory-admin/resources let every kind:'host' resource through
regardless of promotion status, so "Auto-promote to Directory" unchecked on
a discovery plugin never kept unpromoted devices out of the Directory tab.
- GET /api/directory-admin/site-status queried the nonexistent Resource.subType
column instead of metadata.subType, throwing SequelizeDatabaseError.
- Added a "Show ignored" toggle to Discovered Inventory (off by default).
- Untracked nodejs/config/inventory.sqlite -- the app's default runtime DB,
not a fixture, committed by mistake across 13 prior releases.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Removed "Why this over the alternatives"; fixed stale links to the old
per-repo GitHub Pages site to point at the unified theta-suite docs site;
made explicit this is deployed as part of Theta Suite, not standalone; added
the agent capability/install screenshots to the gallery.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
release(v2.0.1): fix openbao container discovery, agent version API collection, console logs, cache invalidation, site status API 500, secrets filtering, and auto-group spawning
* feat(v2.0.0): bump version to 2.0.0, add Multi-Site Master badge, site-status API, and Master promotion UI
* fix(test): update test script to run unit tests without requiring live Redis socket
See CHANGELOG.md for the full breakdown. Summary:
- POST /api/v1/ldap/{bind,search}: LDAP-over-HTTPS so a client stops
speaking raw LDAP and instead calls the SSO, which binds/searches its
own OpenLDAP on the caller's behalf (DESIGN.md §3).
- LDAP byte-pump relay (utils/ldap_tunnel.js): forwards raw LDAP bytes
from an agent's local socket into OpenLDAP over the existing agent WSS
channel; the SSO never parses LDAP (DESIGN.md §4).
- POST /api/v1/agent/secrets: node-scoped OpenBao secret fetch for
agents, enforced to each agent's own secret/data/nodes/<id>/* prefix
(DESIGN.md §5).
- iam_apply signed command: push node-scoped IAM config (sudo rules, SSH
keys, access control, revocation) to an agent (DESIGN.md §6).
- Agent capability badges on the Directory Metrics tab, sourced from the
agent's own discovery frame.
- Join key management: GET /api/agent/join-keys/:id/agents (which hosts
enrolled through a key) plus a Manage join keys table in the Install
Agent modal with Revoke/Delete actions, confirmed inline per-row rather
than a blocking native confirm() or the shared app.messages.confirm()
banner (which desyncs across concurrent rows -- see CHANGELOG).
- docs/agents.md: capability matrix updated for the three new
capabilities, a full secrets-engine walkthrough with screenshots
(bash + Node consuming a rendered secret, plus the direct-API
alternative), and the join-key reuse/UI/audit questions answered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mail sending fell back to a hardcoded noreply@theta42.com From address when
smtp.from wasn't set, which authenticated relays reject with "Sender is not
same as SMTP authenticate username" since no relay authorized this account
to send as that address. Falls back to smtp.user first now.
Also: catalog card titles now read name-then-icon instead of icon-then-name,
and a handful of docs corrections found in an accuracy pass (configuration.md
missing the OpenBao/live-config layer, plugins.md undercounting plugin types,
vault.md describing OpenBao dev-mode/root-token access that doesn't reflect
the real production setup, orphaned discovery.md/vault.md pages linked in,
README's required-groups list missing app_sso_directory_admin).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0113gCdnfSCuZr6xvPDxTo3D
Test Email always failed with "Email.send is not a function":
models/email.js exports {Mail}, and the handler required the module and
called .send on it directly. Every other caller destructures it.
Test SMS failed with "Unexpected token '<'": it POSTed to
https://api.voip.ms/v1.0/sms/send with Basic auth, an endpoint that does
not exist. VoIP.ms's REST API is a GET against voip.ms/api/v1/rest.php
with api_username/api_password and method=sendSMS, so the fabricated URL
returned HTML and response.json() threw.
Worse, ALL SMS delivery was broken. models/sms.js called
PluginInstance.find({...}) but the ORM has no find -- the query method is
list({where}) -- so it threw on every send, before it could even fall
back to the direct VoIP.ms path. OTP-by-SMS and notifications were dead.
Both test endpoints now send through the same senders every real message
uses. A test that reimplements delivery proves nothing about whether real
delivery works, which is how two broken paths went unnoticed. Failures
report as 400 with the underlying reason rather than an opaque 500.
Adds a guard suite that fails the build on any call to a non-existent ORM
static, on requiring models/email without destructuring {Mail}, and on
any reference to the bogus api.voip.ms host.
Also: the Install Agent modal now leads with the join-key flow. v1.30.0
shipped join keys in the API and documented the modal as the place to get
one, but the modal still only did the pre-register flow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JOIN KEYS
v1.29.0 required an admin to pre-register every machine before its agent
would be spoken to. The security model was right; the workflow was not --
installing the agent should be enough to add a host.
POST /api/agent/join-keys mints one credential an operator hands out. A
host presenting it is enrolled automatically and immediately issued its
OWN per-agent token plus the public key it must pin, delivered in the
config frame. The join key is a bootstrap credential, never the host's
identity, so one key stays convenient without becoming a fleet-wide
skeleton key: every host remains individually revocable.
DIRECTORY
Collapsing the tree did nothing. applyTreeCollapse found the caret with
`.tree-caret i` and returned early when absent -- Font Awesome's SVG mode
rewrites <i> to <svg>, so that selector matched nothing and the early
return skipped setting hideBelowDepth. State now lives on the caret
button and is rotated by CSS.
The Discovery Plugins delete button called deleteDiscoveryPlugin(), which
was never defined. The pane also had no .actionMessage, and confirmations
render into one -- without it the promise never settles, so an awaited
confirmation hangs forever and the action silently never happens.
Plugin instances can now be edited.
DISCOVERY
A fresh install presented its own five containers as unmanaged
discoveries. The Docker plugin now recognises the stack's compose project
and attaches each container to the service it implements. Container slugs
came from the container id, which changes on recreate, so every deploy
minted a new resource and orphaned the old one.
DOCS
/docs/discovery 404'd (no slug entry) and `agents` pointed at plugins.md,
leaving docs/agents.md unreachable. Adds docs/discovery.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wmantly