Closes the gap where all of this session's new server-side capability
(live replication, coordinated promotion, identical signing keys) had
no UI at all -- an operator using the Master Site modal had no way to
know any of it existed or was working.
- Master Site modal: new "Live Replication" row (spoke) shows whether
this join actually registered for live updates or is stuck on a
one-time snapshot; new "Registered Spokes" row (master) shows how
many spokes are receiving live pushes.
- Join form: new "this site's own reachable URL" field, prefilled from
window.location.origin, wired to the selfUrl the join API already
supported but the UI never sent -- a UI-driven join previously NEVER
registered for live replication, only the setup.sh bootstrap path did.
The success toast now reports whether live replication actually
activated, not just "joined".
- Promote button: success toast now surfaces the handoff result (old
master demoted / unreachable / no previous master), so the operator
sees immediately whether the coordinated demotion actually happened.
- GET /api/site/config no longer returns masterJoinKey or
replicationPushToken in the response -- found while wiring this up:
live credentials were being sent straight to the browser for every
admin session. Replaced with boolean derivatives
(hasMasterJoinKey, liveReplication).
- GET /api/directory-admin/site-status gained liveReplication (spoke)
and registeredSpokesCount (master) so the modal has something to render.
Verified by actually driving it in a real browser against a live
container (not just code review): logged in, opened the modal, saw the
new rows, minted a real join key end-to-end, no console errors.
docs/site-join.md rewritten to cover live replication, signing-key sync,
coordinated promotion/demote, and the new endpoints -- it previously only
described the v2.2.0-v2.3.0 one-time-snapshot behavior.
Completes the multi-site join layer on top of the v2.2.0 endpoints:
- UI (Master Site modal): a fresh install (canJoin) gets a 'Join an Existing
Site' form (master URL + stj_ key); a master gets a 'Site Join Keys' manager
(mint/revoke/list, key shown once); WAN Sync Health now reflects a live probe.
- POST /api/site/ping (Bearer stj_ key, no admin session): lightweight master
reachability probe for WAN health (cheap vs /export).
- Spoke read-only: directory-write routes (resources/edges/groups/secrets/
grants/driver-action/discovered) reject with 403 pointing at the master.
- Fresh-install guard: /api/site/join refuses unless no users beyond the
bootstrap admin and no enrolled agents (siteIsFresh), and site-status exposes
canJoin so the UI only offers join on a genuinely fresh install. The
bootstrap's seeded default resources are NOT the signal (they always exist).
- The spoke stores the join key (masterJoinKey) in /config/site.json so WAN
health (and a future write-proxy) can reach the master.
- Tests: siteIsFresh cases in tests/site_join.test.js.
Server endpoints for joining a spoke to a master directory (MULTI_SITE_SPEC.md).
This pass is server-only; setup.sh wiring and the UI are the next layer.
- Site join keys (SiteJoinKey model, stj_ prefix): mint/revoke/delete/list,
hashed at rest, shown once — the same model as agent join keys.
- POST /api/site/export (master, Bearer stj_ key, no admin session): returns the
local LDAP tree (slapcat LDIF) + resource catalog + siteSlug + baseDn.
- POST /api/site/join (spoke, admin): { masterUrl, joinKey } pulls the master
export, imports resources (upsert by slug) + LDAP (ldapadd -c), and persists
the spoke role. Refused if already a spoke.
- Persisted site role: utils/site_config.js keeps isMaster/masterUrl/siteSlug in
/config/site.json (env seeds defaults); site-status/site-promote now use it.
- Unit tests (site_join, site_config) with in-memory stubs, wired into npm test.
- docs/site-join.md + docs router entry.
- Repairs the corrupted multi-site emojis (crown/bolt) in directory.ejs.
- .gitguardian.yml ignores the generic-password false positive on reading the
LDAP bind credential from runtime config (never a hardcoded secret).
Fresh-install bug report fixes:
- 'Master Site' button error 'app.modal.show is not a function': the
multi-site status modal used the legacy app.modal.show() signature; the app
exposes app.modal.open({title, bodyHtml, size}). The site-status request
itself worked -- only the rendering call was wrong.
- Agents with no discovery yet showed a fake 'v2.0.0' (three hardcoded
fallbacks). Now 'unknown', so a host whose agent never connected isn't
presented as an old version.
- The default org name (browser tab title) is set by theta-suite's setup.sh;
that default is fixed separately there (CFG_ORG -> Theta Directory).
The Install Agent modal now emits Windows (PowerShell) one-liners for the
join-key, pre-register and custom-config flows. Each downloads the fully-offline
setup installer and passes the same values the bash flow uses:
join key : /SERVER_URL /JOIN_KEY
quick : /SERVER_URL /AUTH_TOKEN /PUBLIC_KEY
custom : /B64_CONFIG=<base64 agent.yml>
Binaries are no longer committed here: the setup.exe and loose agent binaries
are GitHub release artifacts (built by the theta-agent release workflow) and the
modal downloads the installer from releases/latest/download/. The SSO still
serves install.sh (the small Linux bootstrap script); the large binaries are
dropped from this repo.
- GET /api/directory-admin/resources let every kind:'host' resource through
regardless of promotion status, so "Auto-promote to Directory" unchecked on
a discovery plugin never kept unpromoted devices out of the Directory tab.
- GET /api/directory-admin/site-status queried the nonexistent Resource.subType
column instead of metadata.subType, throwing SequelizeDatabaseError.
- Added a "Show ignored" toggle to Discovered Inventory (off by default).
- Untracked nodejs/config/inventory.sqlite -- the app's default runtime DB,
not a fixture, committed by mistake across 13 prior releases.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* feat(v2.0.0): bump version to 2.0.0, add Multi-Site Master badge, site-status API, and Master promotion UI
* fix(test): update test script to run unit tests without requiring live Redis socket
See CHANGELOG.md for the full breakdown. Summary:
- POST /api/v1/ldap/{bind,search}: LDAP-over-HTTPS so a client stops
speaking raw LDAP and instead calls the SSO, which binds/searches its
own OpenLDAP on the caller's behalf (DESIGN.md §3).
- LDAP byte-pump relay (utils/ldap_tunnel.js): forwards raw LDAP bytes
from an agent's local socket into OpenLDAP over the existing agent WSS
channel; the SSO never parses LDAP (DESIGN.md §4).
- POST /api/v1/agent/secrets: node-scoped OpenBao secret fetch for
agents, enforced to each agent's own secret/data/nodes/<id>/* prefix
(DESIGN.md §5).
- iam_apply signed command: push node-scoped IAM config (sudo rules, SSH
keys, access control, revocation) to an agent (DESIGN.md §6).
- Agent capability badges on the Directory Metrics tab, sourced from the
agent's own discovery frame.
- Join key management: GET /api/agent/join-keys/:id/agents (which hosts
enrolled through a key) plus a Manage join keys table in the Install
Agent modal with Revoke/Delete actions, confirmed inline per-row rather
than a blocking native confirm() or the shared app.messages.confirm()
banner (which desyncs across concurrent rows -- see CHANGELOG).
- docs/agents.md: capability matrix updated for the three new
capabilities, a full secrets-engine walkthrough with screenshots
(bash + Node consuming a rendered secret, plus the direct-API
alternative), and the join-key reuse/UI/audit questions answered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mail sending fell back to a hardcoded noreply@theta42.com From address when
smtp.from wasn't set, which authenticated relays reject with "Sender is not
same as SMTP authenticate username" since no relay authorized this account
to send as that address. Falls back to smtp.user first now.
Also: catalog card titles now read name-then-icon instead of icon-then-name,
and a handful of docs corrections found in an accuracy pass (configuration.md
missing the OpenBao/live-config layer, plugins.md undercounting plugin types,
vault.md describing OpenBao dev-mode/root-token access that doesn't reflect
the real production setup, orphaned discovery.md/vault.md pages linked in,
README's required-groups list missing app_sso_directory_admin).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0113gCdnfSCuZr6xvPDxTo3D
Test Email always failed with "Email.send is not a function":
models/email.js exports {Mail}, and the handler required the module and
called .send on it directly. Every other caller destructures it.
Test SMS failed with "Unexpected token '<'": it POSTed to
https://api.voip.ms/v1.0/sms/send with Basic auth, an endpoint that does
not exist. VoIP.ms's REST API is a GET against voip.ms/api/v1/rest.php
with api_username/api_password and method=sendSMS, so the fabricated URL
returned HTML and response.json() threw.
Worse, ALL SMS delivery was broken. models/sms.js called
PluginInstance.find({...}) but the ORM has no find -- the query method is
list({where}) -- so it threw on every send, before it could even fall
back to the direct VoIP.ms path. OTP-by-SMS and notifications were dead.
Both test endpoints now send through the same senders every real message
uses. A test that reimplements delivery proves nothing about whether real
delivery works, which is how two broken paths went unnoticed. Failures
report as 400 with the underlying reason rather than an opaque 500.
Adds a guard suite that fails the build on any call to a non-existent ORM
static, on requiring models/email without destructuring {Mail}, and on
any reference to the bogus api.voip.ms host.
Also: the Install Agent modal now leads with the join-key flow. v1.30.0
shipped join keys in the API and documented the modal as the place to get
one, but the modal still only did the pre-register flow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JOIN KEYS
v1.29.0 required an admin to pre-register every machine before its agent
would be spoken to. The security model was right; the workflow was not --
installing the agent should be enough to add a host.
POST /api/agent/join-keys mints one credential an operator hands out. A
host presenting it is enrolled automatically and immediately issued its
OWN per-agent token plus the public key it must pin, delivered in the
config frame. The join key is a bootstrap credential, never the host's
identity, so one key stays convenient without becoming a fleet-wide
skeleton key: every host remains individually revocable.
DIRECTORY
Collapsing the tree did nothing. applyTreeCollapse found the caret with
`.tree-caret i` and returned early when absent -- Font Awesome's SVG mode
rewrites <i> to <svg>, so that selector matched nothing and the early
return skipped setting hideBelowDepth. State now lives on the caret
button and is rotated by CSS.
The Discovery Plugins delete button called deleteDiscoveryPlugin(), which
was never defined. The pane also had no .actionMessage, and confirmations
render into one -- without it the promise never settles, so an awaited
confirmation hangs forever and the action silently never happens.
Plugin instances can now be edited.
DISCOVERY
A fresh install presented its own five containers as unmanaged
discoveries. The Docker plugin now recognises the stack's compose project
and attaches each container to the service it implements. Container slugs
came from the container id, which changes on recreate, so every deploy
minted a new resource and orphaned the old one.
DOCS
/docs/discovery 404'd (no slug entry) and `agents` pointed at plugins.md,
leaving docs/agents.md unreachable. Adds docs/discovery.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SECURITY
/api/agent/ws authenticated nothing. There was no agent registry, so any
client reaching the SSO could register as a node, publish discovery and
telemetry into the admin view, and receive commands -- including a signed
arbitrary_bash -- addressed to a token it guessed. Tokens were generated
in the BROWSER and never recorded server-side, so there was nothing to
validate against and no way to revoke one.
Agents are now rows in a new Agent table, authenticated by SHA-256 token
hash before the connection is registered or the welcome payload is sent.
Tokens are minted by POST /api/agent/enroll and shown once. Revoke and
rotate drop the live socket immediately. All agent actions are audited.
The Ed25519 command-signing key was generated in the AgentManager
constructor, so it changed on every restart and the public_key pinned in
an agent's agent.yml stopped matching. It now lives in OpenBao at
secret/agent/signing-key; if it cannot be loaded the SSO refuses to send
high-risk commands rather than signing with a key no agent has seen.
DIRECTORY
Agents bind to a host resource instead of being matched by hostname, and
a bound agent's discovery is written onto that resource -- previously the
one source running ON the host contributed nothing to the directory.
The resource tree is collapsible, with state persisted per browser.
DISCOVERY
The Proxmox plugin zipped MACs and IPs from two flat lists by index,
attributing addresses to the wrong NIC on multi-NIC guests. NICs are now
keyed by MAC. Adds an endpoint resource parenting each node, sourceId/
vmid/node identity, container-interface filtering, node IP/MAC, and
offline-node handling.
The reconciler could make a resource its own parent, named hosts after
their MAC address, had a dead isIp() regex (\\. matches a backslash),
merged across kinds, and re-read the whole inventory per resource.
Dockerfile.test-runner never copied nodejs/plugins, so every plugin test
suite failed in CI as "Cannot find module".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- group names match docs/GROUPS.md: {site}_{kind}_{name}_{level} (kind always present; services -> app kind); updated resolver + tests + access_request test
- site resource carries only god_admin + site-wide groups
- groups no longer appear 3x: idempotent ResourceGroup linking (self-heal was creating duplicates on every Directory load)
- /api/agent/* no longer 404s: REST router mounts unconditionally (was gated on the WS server)
- shared-secrets: slug regex allows underscores; GET list uses static pathFor (fixes 's.path is not a function')
- vault Apps tab: new GET /api/vault/apps + Minted apps list + purpose text; /docs/vault help link + docs cover Apps/Shared
- discovery promote: load instance and call update() (fixes 'Resource.update is not a function')
- discovery plugin cards: last-run time/status + Logs button
* feat: complete the group model (god_admin, site groups, aggregates), enforce naming, fix docs 500s + status dots (v1.26.0)
- seed god_admin + nest into app_super_admin; auto-provision site groups (S_super_admin, S_hosts_*/S_apps_* aggregates, S_everyone) on site create + self-heal on Directory load
- map service resources to the app kind (site_local_app_<slug>_*); nest per-resource groups into site aggregates (physical inheritance lattice)
- enforce the group naming convention server-side on POST /groups; surface god_admin + site groups on the site resource modal
- fix in-app /docs/<slug> 500s (Dockerfile never copied docs/); serve doc images at /docs/images
- fix Directory status dots (neutral grey when agent endpoint unreachable); align Profile/API cards full-width
- group resolver: keep the site slug verbatim (site_local not re-slugified)
- bump to 1.26.0
* fix: use verbatim resource slugs in group names (matches access-request tests + live convention)
The group naming inserts a kind segment (resourceGroupCns(site, kind, slug, level)),
but the access-request tests + the live directory convention are verbatim
({site}_{slug}_{level} -- the kind is carried in the resource slug, e.g. host_theta-env).
For bare test slugs this produced site_x_host_artest-host_x_access instead of the
expected site_x_artest-host_x_access, so the requester was never removed from the
auto-provisioned access group and every request 409'd. resourceGroupCns is now
(site, slug, level) with the verbatim slug; the kind is used only to pick the
aggregate the group nests into.
- Add utils/groups.js: the group schema + inheritance resolver (god_admin,
{site}_super_admin, {site}_hosts_*/{site}_apps_* aggregates, per-resource
admin/access/<capability>, meta everyone/{site}_everyone). admin implies
access; capabilities explicit; hosts/apps orthogonal; cross-site isolated.
- permission.js: recognize god_admin (legacy app_super_admin aliased) and add
onResource/requireResource for resource-level checks + everyone meta grants.
- user.js isAdmin: recognize god_admin + site-scoped super/app-admin groups.
- Remove the standalone Groups page (nav + route + view); groups are managed on
adopted Directory resources. Add a /docs/groups help link in the Directory
toolbar (GROUPS.md copied into the SSO docs).
- tests/groups.test.js: full resolver coverage (15 tests).
Co-Authored-By: Claude <noreply@anthropic.com>
- Merge theta-agent into Directory: remove the Agents page; add green/yellow/red
status dots to host rows and a Metrics tab (telemetry + discovery) to the
resource modal, joined to hosts by hostname, live via socket.io + 30s refresh.
- Discovery Plugins New-plugin modal: slug derived from name (field removed),
cron dropdown (hourly/daily/weekly/custom), configSchema-driven settings
(Proxmox url/tokenId/tokenSecret) sent as a populated config.
- Directory resource slug now read-only + derived from name.
- Vault page restyled to match the site.
- Navbar: username no longer underlined; only the active link is bold+underlined.
- docs/agents.md: document the Directory status/metrics + NAT troubleshooting.
Co-Authored-By: Claude <noreply@anthropic.com>
- /api/vault proxy now injects X-Vault-Token: the proxy declared its request
hook with http-proxy-middleware v3 syntax (on: { proxyReq }), which the
installed HPM v2 silently ignores — so every vault call reached OpenBao
unauthenticated (the recurring 403). Rewritten as v2 onProxyReq.
- Header injection ordered before fixRequestBody (the body write flushes
headers; setting X-Vault-Token after it failed on every POST/PUT).
- initORM add-only schema heal: sequelize.sync() never ALTERs, so newer columns
(PluginInstance.lastLog) are now added via describeTable + addColumn.
- Long-lived external-app tokens via sso-app role (768h periodic); VaultAppToken
stores each app token's accessor and renews it at boot + every 6h; re-minting
revokes the previous token via its accessor.
- Wire-level tests for the vault proxy + app-token accessor lifecycle.
- package.json + lockfile bumped to 1.23.0.
Co-Authored-By: Claude <noreply@anthropic.com>
- New admin Agents page (nav + /agents route + views/agents.ejs): live list of
connected theta-agent hosts with CPU/RAM/disk/ZFS/GPU telemetry and online
status, updated live over socket.io ('agent.telemetry'/'agent.discovery').
- Auth + admin-gate the /api/agent REST router (it was mounted without
middleware.auth — anyone could list nodes / send commands). The agent
WebSocket (/api/agent/ws) is unaffected (handled by the raw wss upgrade with
its own token auth).
- package.json + lockfile bumped to 1.22.0 to match the tag.
Co-Authored-By: Claude <noreply@anthropic.com>
The no_native_dialogs regression test forbids native alert/confirm/prompt in
views (they block browser events). Replace the native confirm() in deleteShared
with app.messages.confirm().
- vault_broker: always reconcile policy content before serving a cached
token (compare-and-skip), so stale stored policies can't cause a recurring
403 'permission denied'; policy content is parsed live by OpenBao, so edits
apply to existing tokens immediately.
- Shared secrets: publish to secret/shared/<owner>/<slug>; grant read to users
and apps by editing the grantee's policy content (live-applied). New
SharedSecret/SharedSecretGrant ORM models, /api/shared-secrets router, and a
Shared tab in the vault UI.
- package.json + lockfile bumped to 1.21.0 to match the tag.
Co-Authored-By: Claude <noreply@anthropic.com>
### Fixed
- **Navbar shows Catalog/Vault for unauthenticated users** — Changed nav
gating from `groups: []` (always visible) to `groups: ['login']` and
added synthetic 'login' group handling in app-base.js.
- **500 ENOENT: no such file or directory, open '/docs/directory.md'** —
Created the missing documentation file.
### Changed
- **SMTP configuration UI-only** — Removed SMTP from static config files
(conf/base.js, sso-secrets.js, setup.env.example). SMTP is now only
configurable via the runtime UI at /conf.
### Added
- **Test email/SMS capability** — Added POST /api/conf/test-email and
POST /api/conf/test-sms endpoints with UI buttons in the Configuration
page. Saves config first, then sends test message to verify settings.
### theta-env setup.sh
- **Non-interactive theta-agent configuration** — Added CFG_THETA_AGENT_ENABLE,
CFG_THETA_AGENT_LDAP_AUTH, and CFG_THETA_AGENT_FULL_CONTROL variables to
setup.env (all default to 1/enabled).
Co-Authored-By: Claude <noreply@anthropic.com>
GET /api/conf no longer returns smtp.pass / oauth.jwtSecret in cleartext
(masked to ********). POST treats a blank or ******** secret submission as
"keep the stored value," so editing the From address or token lifetimes no
longer requires re-entering or leaks the SMTP password / JWT secret. The /conf
form fields carry a leave-unchanged hint. Storage stays in OpenBao at
secret/sso-manager/conf (unchanged); no theta-suite policy change needed.
Co-Authored-By: Claude <noreply@anthropic.com>
Generalize the half-built discovery plugins into a real plugin system: plugin
TYPES (the plugins/<category>/<type>.js modules with manifests) and loadable,
configurable, multi-copy plugin INSTANCES (PluginInstance ORM model) managed
from a dedicated /plugins page and /api/plugins API, with per-instance secrets
in OpenBao at secret/plugins/<id>/conf.
- plugin_registry.js: getTypes/getModule/splitConfig/mask + required-field helpers
- PluginInstance model (Sequelize): id/pluginType/category/name/slug(unique)/
enabled/cron/config(json, non-secret)/lastRun*; registered in models/index.js
- plugin_secrets.js: read/write/remove/mergeForRun over @simpleworkjs/bao-conf
- scheduler.js: schedules from the DB registry; per-instance stable BullMQ
JobScheduler ids (plugin:<id>) for load/unload; legacy migration from
conf.discovery.plugins on first boot (idempotent, empty-table-guarded)
- api_plugins.js (replaces routes/plugins.js): types/list/get/create/update/
secrets/test/load/unload/run/delete/runs; admin-gated; secrets always masked
- /plugins page (plugins.ejs) + nav; Agents & Scheduler tab removed from
/directory; /docs/agents aliased to /docs/plugins
- proxmox/unifi/nmap gained manifests (configSchema/validate/run alias)
- tests/plugins.test.js: registry unit + plugin_secrets (mocked bao-conf) +
PluginInstance model round-trip/unique-slug
- docs (plugins.md, vault.md, _config.yml, API.md) + 1.16.1 -> 1.17.0
Requires theta-suite >= v1.30.1 for the sso-broker secret/plugins/* grant;
fails-soft with a clear error if absent.
Co-Authored-By: Claude <noreply@anthropic.com>
Both view routes did server-side auth via req.user, but this app's auth-token is
a header set by client JS (localStorage), not a cookie — so req.user is
undefined on a browser navigation. permission.byGroup(undefined,...) throws
status 401, and the middleware.auth gate on /vault threw Auth.errors.login()
(401) for the same reason.
Both routes now render the shell unconditionally (like /users, /directory) and
gate client-side. conf.ejs already called app.auth.forceLogin; vault.ejs now
derives isAdmin + personal namespace from /api/user/me after forceLogin
instead of server-rendering them. /api/conf and /api/vault still enforce
app_sso_admin + OpenBao scope server-side — only the view-route gating moved
client-side where the session lives. Also removed a dead duplicate /conf route.
Co-authored-by: Claude <noreply@anthropic.com>
Closes the end-user half of the directory and adds nested LDAP groups.
The directory could describe the lab but could not tell anyone what they had
or how to reach it, and several of the paths meant to do so were silently
returning nothing:
- GET /api/discovery/me resolved groups from req.user.groups, which does not
exist (req.user carries memberOf), so it returned only isPublic resources
for every human caller -- "My Services" was blank for everyone. The same
read made isDirectoryAdmin() false for real admins.
- The portal's "Discover More Services" called the admin-gated endpoint and
swallowed the 403, so it never rendered for non-admins at all.
- Services reported no address, because /me had reimplemented getMyAccess
without its parent-walking resolution.
Adds the catalog at /, self-service access requests, and admin access
visibility (per-resource counts, and the reverse "what can this user reach").
Nested groups come in two halves. groupOfNames.member already accepts a group
DN, so nesting needs no schema -- what it needs is resolution, which no
released OpenLDAP performs. The all-in-one image therefore builds slapd from a
pinned master commit for the nestgroup overlay, and the app computes the
closure itself when pointed at a server without it. Both paths are covered.
member-values is deliberately left out of nestgroup-flags: it expands `member`
when reading a group, which destroys the distinction between "listed here" and
"reachable through a nested group" and is not recoverable afterwards.
Full suite green in both resolution modes: 215 passed, 2 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- app_super_admin is a new cross-app LDAP group (also recognized by proxy
and jump-host) that grants full admin here regardless of app_sso_admin
membership: bypassed centrally in utils/permission.js's byGroup, folded
into GET /api/user/me's isAdmin flag, and added to nav/forceLogin gates
alongside app_sso_admin.
- Renamed the Executive page to Overview (route, view, API path
/api/metrics/overview, nav label, docs), keeping /executive as a 301
redirect alongside the existing /admin, /notifications, /dashboard
legacy redirects.
wmantly