Compare commits

..

1 Commits

Author SHA1 Message Date
wmantly 2333a145cc fix: drop legacy app_super_admin -- SUPER_ADMIN_GROUP is now god_admin (v1.26.1)
Pull Request Tests / Run Tests (18.x) (push) Failing after 1m38s
Pull Request Tests / Run Tests (20.x) (push) Failing after 25s
Pull Request Tests / Run Tests (22.x) (push) Failing after 26s
Pull Request Tests / Test Summary (push) Failing after 4s
god_admin now exists at boot (seeded by docker-entrypoint), so the canonical
cross-resource super group nested into every resource's _admin group is god_admin,
not the legacy app_super_admin. docker-entrypoint no longer seeds or nests
app_super_admin (god_admin nests into the app_sso_* groups directly). isSuperAdmin
still recognizes a pre-existing app_super_admin as a migration alias until rebuild.
2026-08-04 19:27:54 -04:00
58 changed files with 513 additions and 4922 deletions
-177
View File
@@ -1,180 +1,3 @@
# v1.31.0 - 2026-08-07
### Added
- **Resource Secrets Engine & Zero-View Security.** OpenBao KV-v2 encrypted secrets for directory resources (`secret/data/resources/<slug>/conf`). Zero-View UI & API model — secret values are never returned to admin browsers or UI templates, and delivered exclusively to authenticated `theta-agent` instances.
- **Strict Secret Key Regex Validation.** Secret keys are validated against `^[A-Za-z0-9_]+$` (Standard Environment Variable format, e.g. `DB_PASSWORD`).
- **Field-Populating Password Generator.** Cryptographic secret generator (`window.crypto.getRandomValues`) with length selector dropdown (8128 chars) populating input fields with security notices.
- **Multi-Level Secret Inheritance.** Dynamic secret resolution across any depth of the resource tree (`Services / Apps -> Hosts / Nodes -> Global Sites`).
- **Non-Blocking UI Confirmations.** Replaced browser blocking dialogs with async `app.messages.confirm()` banners.
- **UI Directory Layout Improvements.** Fixed Directory table resource name and badge order for enhanced readability.
### Fixed
- **SSSD `sshPublicKey` Mapping.** Included `ldap_user_ssh_public_key = sshPublicKey` in generated agent `sssd.conf` template.
# Unreleased — LDAP-over-HTTPS API + agent LDAP byte-pump relay
### Added
- **`POST /api/v1/ldap/bind` and `POST /api/v1/ldap/search`** — an LDAP-over-HTTPS
API (DESIGN.md §3). A client stops speaking LDAP and instead does an HTTPS call
to the SSO, which performs the real bind/search against its own OpenLDAP. This
kills the hostname / cross-network / LDAPS-cert-chain pain. Caller auth is a
Bearer token: an agent token or a self-service API token (PAT). `/search` is
restricted to agent callers (the SSSD user/group-resolution use case) and runs
under the admin bind — see DESIGN.md §9.5 for the scoped-service-account
follow-up.
- **LDAP byte-pump relay** (`utils/ldap_tunnel.js`) — the SSO relays raw LDAP
bytes from an agent's local socket into its real OpenLDAP and pipes the
response back, over the existing agent WSS channel (`ldap_tunnel` messages).
The SSO does not parse LDAP; it is a transparent socket relay. See DESIGN.md §4.
- **`POST /api/v1/agent/secrets`** — an agent fetches its own node-scoped OpenBao
secrets (DESIGN.md §5). The agent may only read under `secret/data/nodes/<id>/*`;
the SSO fetches with its own OpenBao access, so the agent never holds a Vault
token. Agent-token authed (not admin-gated).
- **`iam_apply` command** — the SSO pushes node-scoped IAM config (sudo rules,
SSH keys, access control, revocation) to an agent as a signed high-risk
command (DESIGN.md §6). Added to `HIGH_RISK_COMMANDS`.
- **Agent capabilities in the Directory UI** — the agent reports its enabled
capabilities in its `discovery` frame; the SSO stores them and the host's
Metrics tab renders them as green/gray badges, so an operator can see at a
glance what each agent is allowed to do.
- **`GET /api/agent/join-keys/:id/agents`** — which hosts enrolled through a
given join key. Matches on the trace `Agent.enroll` already leaves in
`description` ("Self-enrolled with join key `<prefix>`") rather than a stored
relation.
- **Join key management in the Install Agent modal** — a table (label, prefix,
created date, hosts joined, status) alongside the existing mint/select
dropdown, with **Revoke** and **Delete** actions and a click-through to see
which hosts joined via a given key. Previously these were API-only. Revoke
and Delete confirm inline within the row ("Revoke? Yes/No") rather than a
blocking native `confirm()` (freezes the whole tab) or the shared
`app.messages.confirm()` banner (a single `.actionMessage` shared by the
whole card, so a second click before the first resolves leaves a dangling
`$('body').one('click', ...)` handler from the first call and desyncs which
row the banner is actually confirming for).
# v1.30.2
### Fixed
- **Outbound mail (test email, invites, password resets, OTP-by-email, notifications) could be rejected by the SMTP relay with `554 5.7.1 ... Sender is not same as SMTP authenticate username`.** Many authenticated relays require the `From` address to match the authenticated account or they refuse the send outright. `models/email.js` fell back to a hardcoded `noreply@theta42.com` when `smtp.from` wasn't set, which no relay ever authorized this account to send as. It now falls back to `smtp.user` first — the address the account can actually prove it owns — before the hardcoded placeholder.
- **Catalog page card titles read icon-then-name.** Swapped to name-then-icon so the resource name leads.
### Docs
- `docs/configuration.md` didn't mention that OpenBao + the live Configuration UI sit above the four file/env config layers and win the merge — added.
- `docs/plugins.md` listed 3 of 4 discovery plugin types (missing `docker`) and didn't mention the `messaging` plugin category (`twilio`, `webhook`) at all — added both.
- `docs/vault.md` had no navigation (no frontmatter, no back-link, unreachable from the docs index) and described OpenBao as running in dev mode with API access via the root token — both wrong for a real deployment. Fixed navigation and corrected to describe the actual production setup (unsealed OpenBao, server-side scoped-token injection, personal API tokens for programmatic access).
- `docs/discovery.md` was unreachable from the docs index and missing its back-link — both fixed.
- `README.md`'s required-groups list was missing `app_sso_directory_admin` (gates Directory/Plugins/Agent admin).
# v1.30.1
### Fixed
- **Test Email always failed with `Email.send is not a function`.** `models/email.js` exports `{Mail}`; the handler required the module and called `.send` on it directly. Every other caller destructures it. The button could never have worked.
- **Test SMS failed with `Unexpected token '<', "<!DOCTYPE "...`.** It POSTed to `https://api.voip.ms/v1.0/sms/send` with Basic auth — an endpoint that does not exist. VoIP.ms's REST API is a GET against `https://voip.ms/api/v1/rest.php` with `api_username`/`api_password` and `method=sendSMS`, so the fabricated URL returned an HTML page and `response.json()` threw. It could never have sent anything.
- **All SMS delivery was broken, not just the test button.** `models/sms.js` called `PluginInstance.find({…})`, but @simpleworkjs/orm has no `find` — the query method is `list({where})`. It threw "is not a function" on every send, before it could even fall back to the direct VoIP.ms path, so OTP-by-SMS and notifications were dead too.
- Both test endpoints now send through the **same senders every real message uses** (`Mail.send`, `SMS.send`). A test that reimplements delivery proves nothing about whether real delivery works — which is exactly how two broken paths went unnoticed.
- The SMS credential check no longer demands `conf.voipms` when a messaging plugin is loaded; the plugin supplies its own credentials, and requiring both blocked a working setup from testing itself.
- Both endpoints report a failure as a `400` with the underlying reason (`VoIP.ms error: invalid_credentials`, `connect ECONNREFUSED …:587`) instead of an opaque `500`. A misconfiguration is the operator's to fix and the UI should be able to show it.
- test: a guard suite that fails the build on any call to a non-existent ORM static (`find`/`findOne`/`findAll`/`where`), on requiring `models/email` without destructuring `{Mail}`, and on any reference to the bogus `api.voip.ms` host.
### Added
- **Install Agent offers the join-key flow.** The modal now leads with "Join key" — mint one, copy a single install command, and the host enrolls itself. Pre-registering a specific host moved to a second tab. v1.30.0 shipped join keys in the API and documented the modal as the place to get one, but the modal itself still only did the pre-register flow.
# v1.30.0
Adds **join keys**: installing the agent with one key is now all it takes to add a host. Fixes a set of Directory/discovery defects found on a fresh `setup.sh` install.
### theta-agent — enrollment without pre-registering
- feat: **join keys.** `POST /api/agent/join-keys` mints one credential an operator hands out. A host presenting it is enrolled automatically and immediately issued **its own** per-agent token plus the public key it must pin, delivered in the `config` frame; the agent persists both and blanks the join key. v1.29.0 required an admin to pre-register every machine before its agent would be spoken to, which made adding a host a two-system chore — the security model was right, the workflow was not.
- feat: a join key is a bootstrap credential, never the host's identity, so one key stays convenient without becoming a fleet-wide skeleton key: every host remains individually revocable and a compromised host yields nothing that works elsewhere. Revoking a join key stops new hosts joining and leaves already-enrolled agents alone.
- feat: join keys support a label and optional expiry, record their use count, and are stored as a SHA-256 (`AgentJoinKey`). Issue/revoke/delete and every self-enrollment are audited.
### Directory
- fix: **collapsing the tree did nothing.** `applyTreeCollapse` located the caret with `$row.find('.tree-caret i')` and returned early when it found nothing. Font Awesome runs in SVG-with-JS mode and its mutation observer rewrites every `<i class="fa-…">` into an `<svg>`, so moments after a render that selector matched nothing — and the early return skipped setting `hideBelowDepth`, so no row was ever hidden. Collapse state now lives on the caret *button* and is rotated by CSS, and the hide decision is made from the collapsed set alone. Never key behaviour to an element another library is free to replace.
- fix: **the Discovery Plugins delete button did nothing.** It called `deleteDiscoveryPlugin()`, which was never defined — clicking it only threw a `ReferenceError`.
- fix: the plugins pane had no `.actionMessage` element, and `app.messages` confirmations render into one. Without it the returned promise **never settles**, so an awaited confirmation hangs forever and the action it gates silently never happens. Added, along with a note that any pane asking for confirmation needs it.
- feat: **discovery plugin instances can be edited.** Name, schedule, loaded state and configuration, with secrets on their own endpoint and left blank ("unchanged") rather than prefilled with the mask — submitting `********` back would otherwise store the asterisks as the secret.
### Discovery
- fix: **a fresh install no longer presents its own containers as things to triage.** The Docker plugin recognises containers belonging to the stack's own compose project, records them as managed, and attaches each to the service it implements. `setup.sh` deploys `sso-manager`, `proxy`, `jump-host`, `openbao` and `bao-renewer`; all five arrived as unmanaged discoveries awaiting promotion.
- fix: **Docker container slugs were derived from the container id**, which changes on every recreate — so each `docker compose up` minted a brand-new resource and orphaned the previous one. Slugs now come from compose project + service, falling back to the container name.
- feat: discovered containers carry `composeProject`, `composeService`, `containerName` and `sourceId`.
### Docs
- fix: `/docs/discovery` 404'd — the slug had no entry, though the Discovery tab's help icon linked to it. New `docs/discovery.md` covering the catalog/discovered distinction, how sources are matched and merged, naming precedence, promotion and garbage collection.
- fix: the `agents` slug pointed at `plugins.md`, so `docs/agents.md` was unreachable in the app.
# v1.29.0
**Breaking:** theta-agent enrollment is now mandatory. Agents installed before this release carry a browser-generated token the server never recorded and will be rejected until re-enrolled. Requires theta-suite ≥ v1.42.0 (the `sso-broker` OpenBao policy must grant `secret/agent/*`); re-run `./setup.sh`.
### Security — theta-agent channel
- **sec: `/api/agent/ws` accepted any token.** There was no agent registry, so the endpoint authenticated nothing: any client that could reach the SSO could register as a node, publish discovery/telemetry into the admin view, and receive commands — including a signed `arbitrary_bash` — addressed to a token it guessed. Tokens were generated in the *browser* (`generateRandomHexToken`) and never recorded server-side, so there was nothing to validate against and no way to revoke one. Agents are now rows in a new `Agent` table, authenticated by SHA-256 token hash before the connection is registered or the welcome payload is sent; unknown or revoked tokens are closed with `4001` and audited.
- **sec: the command signing key was ephemeral.** `AgentManager` generated an Ed25519 pair in its constructor, so it changed on every process start and the `public_key` an agent pinned in `agent.yml` stopped matching immediately. The key now lives in OpenBao at `secret/agent/signing-key` and survives restarts. If it cannot be loaded the SSO **refuses** to send high-risk commands rather than signing with a key no agent has seen (`signingAvailable: false` on `GET /api/agent/nodes`).
- **sec: commands are addressed by agent id, not token.** A credential has no business in a URL, an access log or browser history.
- **sec: agent actions are audited.** Enroll, update, rotate, revoke, delete, every command (with `signed`), and every rejected connection are emitted as structured `"component":"agent"` log records carrying the acting user.
### theta-agent — enrollment & resource binding
- feat: `POST /api/agent/enroll` mints the token server-side and returns it **once**; only its SHA-256 is stored. Plus `PUT /nodes/:id` (rename/rebind), `POST /nodes/:id/rotate`, `POST /nodes/:id/revoke`, `DELETE /nodes/:id`. Rotate, revoke and delete drop the live socket immediately (`4004`/`4003`) instead of waiting for a reconnect.
- feat: an agent binds to a **host resource** (`resourceId`). The Directory reads that link instead of guessing by hostname — the old `agentsByHost[name]` match silently failed whenever a Directory name differed from the machine's hostname, and aliased two hosts that shared one.
- feat: **agent discovery reaches the Directory.** A bound agent's facts (`os`, `kernel`, `cpu`, `ram_total_gb`, `disk_total_gb`, `ip`) are written onto its host resource, tagged `discovery_sources: ["theta-agent"]` with an `agentId` back-reference. An unbound agent goes through the normal reconciler. Previously `handleDiscovery` wrote to an in-memory record and updated nothing — the one source actually running *on* the host contributed nothing to the directory.
- feat: agent state is persisted, so an agent that is installed but **offline** is now distinguishable from one that never existed; enrollments survive a restart. The Directory status dot reflects this: red means "enrolled and not connected" (a fault), grey means no agent enrolled / revoked / service unreachable. Red previously covered both, making an ordinary directory of hosts look like an outage.
- feat: the Install Agent modal enrolls first and builds the install command from the result, including `--public-key`. `public_key` was never emitted into the generated `agent.yml` before, so no installed agent could verify anything.
- fix: `registerAgent` is synchronous. Awaiting a database write before attaching the WebSocket `message` listener lost every agent's first `discovery` frame, which it sends the instant the socket opens (`ws` drops events emitted with no listener attached).
### Directory
- feat: **the resource tree is collapsible.** Any row with children has a caret; the toolbar collapses/expands everything. State persists per browser, so the shape survives the self-heal reload that follows most edits. An active search overrides collapse so matches inside a folded subtree are never hidden.
- fix: **the Proxmox plugin mismatched MAC addresses to IPs.** It collected MACs and IPs into two flat lists and zipped them by index, so on any multi-NIC guest — or any guest where one NIC had no address — the directory recorded an address against the wrong MAC. NICs are now keyed by MAC, so a pairing can only come from the source that observed both together.
- feat: Proxmox discovery emits an **endpoint resource** (named from `/cluster/status`) with every node parented beneath it, so one endpoint is one subtree instead of several orphan roots. It deliberately carries no IP: giving it the address it is reached at made the reconciler merge it with the node answering on that address, producing a resource that was its own parent.
- feat: discovered guests carry `sourceId` (`<node>/qemu/<vmid>`), `node`, `vmid` and `macAddress`, so a row traces back to the exact guest on the exact hypervisor. Against a live 3-node cluster this took MAC coverage to 53/54 resources and `sourceId` to 54/54.
- fix: Proxmox interfaces belonging to something running *inside* a guest (`docker0`, `veth*`, `br-*`, VPN tunnels) are filtered out — one Home Assistant VM reported 16 of them alongside its single real NIC, and their 172.x addresses gave the reconciler spurious matches.
- fix: a stopped VM still reports its MAC (read from the VM config), a DHCP-configured LXC gets its address from the running container's interface list, and Proxmox **nodes** report their own IP/MAC (recovered from `enx<mac>` predictable names, since `/nodes/*/network` carries no `hwaddr`). Offline nodes are recorded with `status` instead of skipped, so a hypervisor that is down no longer looks decommissioned and get garbage-collected after a week.
- fix: **the reconciler could make a resource its own parent.** Two slugs in one payload can resolve to the same row once merged; the resulting self-edge renders as an infinitely nested tree and defeats every ancestor walk in the app. Self-edges and cycle-closing edges are now refused and logged.
- fix: **hosts were named after their MAC address.** `bestName` preferred the *longer* name, so UniFi's `ac:16:2d:b3:da:80` (17 chars) beat Proxmox's real hostname `dl380-0` (7). Names are now ranked (hostname > IP > MAC) with length only as a tie-break within a rank.
- fix: `isIp` never matched anything — `\\.` inside a regex literal matches a backslash, not a dot — so an IP-shaped placeholder name was never replaced by a real hostname a later source discovered.
- fix: a discovered device can only merge into a resource of the same kind. A VM named `gitea-runner` could match a hand-created *service* of the same name on the name rule and overwrite it.
- perf: the reconciler reads the inventory once per run instead of once per incoming resource — a ~55-resource Proxmox payload against a similar-sized inventory was doing quadratic full-table reads every run.
- fix: the Discovered Inventory table showed "Unknown IP" for almost everything, because it read `metadata.ip` while any source that enumerates interfaces stores addresses per-NIC. It now falls back to the first NIC address, and shows `vmid`, slug, `sourceId` and per-interface MAC/name.
### Profile
- fix: the API Tokens card is no longer wider than every other card on the site — the section sat outside the page's `.container`.
### Build & docs
- fix: `Dockerfile.test-runner` never copied `nodejs/plugins`, so every plugin test suite failed in CI as "Cannot find module" and plugin code was effectively untested. Suite count goes 27 → 29.
- docs: `docs/agents.md` rewritten for enrollment, the close-code table, resource binding, the persistent signing key, and a corrected `public_key` example (the documented `MCowBQYDK2VwAyEA...` was an SPKI PEM body — 44 bytes decoded — where the agent requires the raw 32).
- docs: `docs/directory.md` covers the collapsible tree and the corrected seed hierarchy; `docs/plugins.md` documents what the Proxmox plugin produces and why the endpoint has no IP.
# v1.28.0
- fix: `/api/agent/nodes` no longer 404s — the previous "unconditional mount" was still inside the post-listen `onListen` hook, so the REST router landed *behind* app.js's terminal 404 catch-all and every `/api/agent/*` request 404'd. The router is now mounted synchronously in `app.js` before the 404 handler; only the agent WebSocket setup runs on `onListen`.
- feat: promoting a discovered inventory resource now opens the resource form pre-filled with the discovered data (name, kind, IP, subtype, …) for review; the modal's Save confirms the promote (creates the LDAP groups + marks it managed) instead of silently promoting.
- fix: Directory table no longer goes stale after add/remove edge — `addEdge`/`removeEdge` called an undefined `loadData()`, which threw and left the host/parent linkage stale until a manual refresh; they now call `loadResources()`. `addGroup`/`removeGroup` also refresh so the Access column stays accurate.
- feat: Vault page states it's powered by OpenBao (header badge linking to openbao.org).
# v1.27.0
- fix: Directory group names now match `docs/GROUPS.md` exactly — per-resource groups are `{site}_{kind}_{name}_{level}` (`site_local_host_theta-env_access`, `site_local_app_sso-manager_access`), with the kind always present and the resource name slug stripped of its kind prefix. Services map to the `app` kind. The access-request + resolver tests were updated to the documented convention.
- fix: a site resource now carries only `god_admin` + the site-wide groups (`{site}_super_admin`, `{site}_everyone`); the kind-scoped aggregates are still created for nesting but are no longer surfaced on the site's modal.
- fix: groups no longer appear 3× under a resource — the Directory self-heal (which runs on every load) was creating duplicate `ResourceGroup` links; linking is now idempotent (check-then-create).
- fix: `/api/agent/nodes` no longer 404s — the agent REST router is mounted unconditionally instead of being gated on the WebSocket server being up.
- fix: `POST /api/shared-secrets/` rejected valid slugs — the slug regex now allows underscores (was hyphens-only).
- fix: `GET /api/shared-secrets/` crashed with `s.path is not a function` — the list spread dropped the instance's `path()` method; now uses the static `SharedSecret.pathFor`.
- fix: promoting a discovered inventory resource crashed with `Resource.update is not a function``update` is an instance method; the promote handler now loads an instance and calls `update()` on it.
- feat: Vault → Apps tab now lists minted app tokens (the "Minted apps" list) — each is a scoped OpenBao credential for an external service; sso renews them and the list shows renewal state, so a minted credential no longer vanishes after its once-only token display. New `GET /api/vault/apps`.
- feat: Vault page documents itself — a `/docs/vault` help icon in the header, and the doc now covers the Apps + Shared tabs.
- feat: discovery plugin cards show last-run time + status (ok/error) and a Logs button that opens the captured run log.
# v1.26.1
- fix: the legacy `app_super_admin` group is gone — `SUPER_ADMIN_GROUP` (nested into every resource's `_admin` group by auto-provisioning) is now `god_admin`, and `docker-entrypoint.sh` no longer seeds or nests `app_super_admin` (god_admin is nested into the `app_sso_*` groups directly). `isSuperAdmin` still recognizes a pre-existing `app_super_admin` as a migration alias, so an old deployment isn't stripped of rights until it's rebuilt.
-6
View File
@@ -22,10 +22,6 @@ COPY nodejs/conf ./conf
COPY nodejs/controller ./controller
COPY nodejs/middleware ./middleware
COPY nodejs/models ./models
# Without this the discovery/plugin suites cannot even load their subject and
# fail as "Cannot find module ../plugins/discovery/..." -- plugin code was
# effectively untested in CI.
COPY nodejs/plugins ./plugins
COPY nodejs/routes ./routes
COPY nodejs/services ./services
COPY nodejs/utils ./utils
@@ -47,8 +43,6 @@ COPY directory_spec.md /directory_spec.md
COPY test_seed.js ./test_seed.js
COPY test/seed-test-user.sh /usr/local/bin/seed-test-user
RUN chmod +x /usr/local/bin/seed-test-user
# End-to-end LDAP tunnel test client (docker-compose.e2e.yml)
COPY test/tunnel_e2e.js ./test/tunnel_e2e.js
# Default command: seed the test user, then run the test suite
CMD ["sh", "-c", "seed-test-user && npm test"]
+1 -2
View File
@@ -200,8 +200,7 @@ If you are pointing the app at your own existing LDAP server, see
`pw-sha2`, `ppolicy`, `memberof`, and `refint` modules plus a small custom
schema. The bundled Docker image and `install.sh` set all of that up for you.
Required groups: `app_sso_admin` (full admin), `app_sso_oauth_admin` (manage
OAuth clients only), `app_sso_invite` (invitation management),
`app_sso_directory_admin` (Directory/Plugins/Agent admin) — see
OAuth clients only), `app_sso_invite` (invitation management) — see
DEPLOYMENT.md for the full setup.
## Development
-93
View File
@@ -1,93 +0,0 @@
# End-to-end test of the LDAP byte-pump tunnel (DESIGN.md §4).
#
# Spins up OpenLDAP + Redis, a real SSO server (bin/www, so the WSS relay is
# live), and a client that simulates the agent: it enrolls one, connects over
# WSS, sends a real LDAP bind as raw bytes, and verifies the SSO relays it into
# OpenLDAP and pipes the response back.
#
# docker compose -f docker-compose.e2e.yml up --build --abort-on-container-exit
# # exit code 0 = tunnel works; the client prints E2E PASS.
services:
ldap:
build:
context: .
dockerfile: Dockerfile.openldap
environment:
- LDAP_BASE_DN=dc=test,dc=local
- LDAP_ADMIN_PASS=secret
- ORG_NAME=Test SSO
command: ["sleep", "infinity"]
healthcheck:
test: ["CMD-SHELL", "ldapsearch -x -H ldap://localhost:389 -b '' -s base '(objectClass=*)' >/dev/null 2>&1"]
interval: 2s
timeout: 3s
retries: 20
start_period: 5s
volumes:
- ldap-data:/var/lib/ldap
- ldap-certs:/etc/openldap/certs
redis:
image: redis:7-alpine
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 2s
timeout: 3s
retries: 15
sso:
build:
context: .
dockerfile: Dockerfile.test-runner
command: ["node", "bin/www"]
environment:
- NODE_ENV=test
- NODE_PORT=3001
# Test OpenBao (theta-test-bao) — sso-broker token so the SSO can sign
# high-risk agent commands and read node-scoped secrets.
- VAULT_ADDR=http://theta-test-bao:8200
- VAULT_TOKEN=${VAULT_TOKEN:-}
- app_ldap__url=ldap://ldap:389
- app_ldap__bindDN=cn=admin,dc=test,dc=local
- app_ldap__bindPassword=secret
- app_ldap__userBase=ou=people,dc=test,dc=local
- app_ldap__groupBase=ou=groups,dc=test,dc=local
- app_redis__redisConf__url=redis://redis:6379
- REDIS_URL=redis://redis:6379
- app_oauth__jwtSecret=test-jwt-secret-for-testing-only
- app_name=Test SSO
depends_on:
ldap:
condition: service_healthy
redis:
condition: service_healthy
client:
build:
context: .
dockerfile: Dockerfile.test-runner
command: ["sh", "-c", "seed-test-user && node test/tunnel_e2e.js"]
environment:
- NODE_ENV=test
- SSO_URL=http://sso:3001
- app_ldap__url=ldap://ldap:389
- app_ldap__bindDN=cn=admin,dc=test,dc=local
- app_ldap__bindPassword=secret
- app_ldap__userBase=ou=people,dc=test,dc=local
- app_ldap__groupBase=ou=groups,dc=test,dc=local
- app_redis__redisConf__url=redis://redis:6379
- REDIS_URL=redis://redis:6379
- app_oauth__jwtSecret=test-jwt-secret-for-testing-only
- app_name=Test SSO
depends_on:
sso:
condition: service_started
ldap:
condition: service_healthy
redis:
condition: service_healthy
volumes:
ldap-data:
ldap-certs:
+11 -372
View File
@@ -6,124 +6,7 @@ nav_order: 5
# Theta Agent & Endpoint Management
The **Theta Agent** (`theta-agent`) is a unified, 2-way Command & Control (C2)
endpoint management daemon written in Go for Linux hosts across your home lab,
infrastructure, or data center. It connects outbound via a long-lived WebSocket
connection to the central **SSO Manager** (`wss://<sso-host>/api/agent/ws`),
enabling real-time host telemetry, automated host discovery, and local-first
administrative management.
---
## Enrollment
An agent is only real if the SSO issued its credential. **Tokens the server did
not issue are rejected** at the WebSocket handshake.
There are two ways to get a host enrolled, and the first is the normal one.
### Join key — install the agent and the host appears
Hand the machine a **join key** and nothing else. On first connect the SSO
enrolls the host, issues it its own per-agent token plus the public key it must
pin, and the agent **writes both into its own `agent.yml`** and blanks the join
key. From then on it authenticates as itself.
```bash
curl -fsSL https://<SSO_HOST>/resources/theta-agent/install.sh | sh -s -- \
--url "https://<SSO_HOST>" --join-key "tjk_..."
```
That is the whole procedure — no pre-registering the machine, no copying a
public key by hand. `setup.sh` mints a key and configures the stack's own host
this way automatically.
The join key is a *bootstrap* credential, not the host's identity. That
distinction is what keeps one key convenient without making it a fleet-wide
skeleton key: every host still ends up individually revocable, and a compromised
host does not yield a credential that works anywhere else.
| Endpoint | Purpose |
| :--- | :--- |
| `GET /api/agent/join-keys` | List keys (prefix + usage only; never the key) |
| `POST /api/agent/join-keys` | Mint one — returned **once** |
| `POST /api/agent/join-keys/:id/revoke` | Stop it enrolling new hosts |
| `DELETE /api/agent/join-keys/:id` | Remove it |
| `GET /api/agent/join-keys/:id/agents` | Which hosts enrolled through this key |
Revoking a join key does **not** disconnect hosts that already joined; they hold
their own tokens by then. Revoke the agent itself to cut a specific host off.
**Reuse.** Yes — a join key is not consumed on use. `AgentJoinKey.authenticate`
only checks `revoked` and `expires_on`; it never invalidates the key itself.
Every use increments `use_count` and stamps `last_used_on`, but the key keeps
working until you revoke or delete it (or it expires) — "one key works for as
many hosts as you like" above is literal, not a figure of speech.
**UI.** The **Install Agent** modal (Directory → Install Agent → Join key tab)
has a **Manage join keys** table below the mint/select dropdown: label, prefix,
created date, hosts joined, status, and **Revoke**/**Delete** actions per key.
Clicking a key's "N hosts" link expands the list of hosts that joined through
it (name, online status, joined date, last seen).
**Audit.** Yes, both halves are logged as structured `"component":"agent"`
lines, and the hosts-joined list in the UI above is queryable directly:
- Minting: `action: "join_key_issued"` records the acting admin (`actor`),
`label`, and `keyPrefix`.
- Each enrollment through that key: `action: "join"` records `agentId`,
`agentName`, `remoteAddr`, `joinKeyLabel`, and `joinKeyPrefix`.
- `GET /api/agent/join-keys/:id/agents` returns the same "which hosts did key
X add" answer the UI shows — it matches on the trace `Agent.enroll` leaves in
each agent's `description` ("Self-enrolled with join key `<prefix>`") rather
than a stored foreign key, since a join key is exchanged for a per-agent
token immediately and from then on the agent's own identity is what matters.
### Pre-registering a host
When you want the agent bound to a specific Directory host up front, enroll it
from **Directory → Install Agent**:
1. Give the agent a name and **bind it to a host resource**. The binding is what
links telemetry, status and commands to a Directory entry.
2. Press **Enroll & issue token**. The SSO mints a 256-bit token, stores only its
SHA-256, and shows the raw value **once**.
3. Copy the generated install command — it already carries the token and the
server's public key.
A host that self-enrolls with a join key arrives unbound; bind it afterwards with
`PUT /api/agent/nodes/:id` or from the Directory.
Or via the API:
```bash
curl -X POST https://<SSO_HOST>/api/agent/enroll \
-H "Authorization: Bearer <admin-api-token>" \
-H 'Content-Type: application/json' \
-d '{"name": "web01", "resourceId": "<host-resource-uuid>"}'
```
The response contains `token` (once only) and `publicKey`.
| Endpoint | Purpose |
| :--- | :--- |
| `GET /api/agent/nodes` | Every enrolled agent, connected or not, plus the server public key |
| `POST /api/agent/enroll` | Mint an agent + token |
| `PUT /api/agent/nodes/:id` | Rename, or bind/unbind the host resource |
| `POST /api/agent/nodes/:id/rotate` | Issue a new token; the old one stops working immediately |
| `POST /api/agent/nodes/:id/revoke` | Disable the enrollment |
| `DELETE /api/agent/nodes/:id` | Remove the enrollment |
| `POST /api/agent/nodes/:id/command` | Send a command (signed automatically when high-risk) |
Revoke, rotate and delete **drop any live connection immediately** — they do not
wait for the agent to reconnect. Commands are addressed by agent **id**, never by
token: a token is a credential and has no business in a URL or a log.
Enrollment, revocation, rotation, every command, and every rejected connection
are written to the application log as structured `"component":"agent"` records
with the acting user.
> **Lost the token?** It cannot be recovered — only its hash is stored. Rotate
> the agent to issue a new one.
The **Theta Agent** (`theta-agent`) is a unified, 2-way Command & Control (C2) endpoint management daemon written in Go for Linux hosts across your home lab, infrastructure, or data center. It connects outbound via a long-lived WebSocket connection to the central **SSO Manager** (`wss://<sso-host>/api/agent/ws`), enabling real-time host telemetry, automated host discovery, and local-first administrative management.
---
@@ -158,31 +41,12 @@ Directory shows a status dot in the row:
| :--- | :--- |
| **Green** | Connected, healthy (CPU/RAM/disk within limits). |
| **Yellow** | Connected but under high load (CPU > 80% or RAM > 80% or disk > 90%). |
| **Red** | **Enrolled but not connected.** The agent exists and is expected — this is a fault. |
| **Grey** | No agent enrolled for this host, the enrollment is revoked, or the agent service is unreachable. |
Red and grey used to be the same colour, which made an ordinary directory of
hosts look like an outage. Because the enrollment now outlives the connection,
"installed but down" is distinguishable from "never had an agent".
| **Red** | Not connected (no agent, or the agent is offline). |
Opening a host's resource modal reveals a **Metrics** tab with the agent's live
telemetry (CPU/RAM/disk/ZFS/GPU) and discovery info (OS, kernel, IPs, location).
An agent attaches to its host by its **enrollment binding** (`resourceId`), set
when you enroll it or later via `PUT /api/agent/nodes/:id`. Agents enrolled
without a binding fall back to matching their reported hostname against the
resource name — the old behaviour, kept only as a fallback, because it silently
failed whenever a Directory name differed from the machine's hostname and
aliased two hosts that happened to share one.
### Agent discovery feeds the Directory
A bound agent's discovery payload is written onto its host resource (`os`,
`kernel`, `cpu`, `ram_total_gb`, `disk_total_gb`, `ip`), tagged with
`discovery_sources: ["theta-agent"]` and an `agentId` back-reference. An agent
runs *on* the host it describes, so it is the most authoritative source the
directory has. An unbound agent goes through the normal discovery reconciler
instead, matching like any other source.
The agent is joined to its host by hostname (`agent.discovery.hostname` ↔ the
resource name), so name the Directory host the same as the machine's hostname.
---
@@ -197,205 +61,17 @@ To protect hosts against unauthorized control, `theta-agent` enforces a **strict
| **Service Control** | `service_control` | High | Restarts systemd services listed in an explicit allowlist (e.g., `["nginx", "docker", "sssd"]`). |
| **Reboot** | `reboot` | High | Triggers an immediate system reboot (`systemctl reboot`). |
| **Arbitrary Bash** | `arbitrary_bash` | Critical | Executes raw bash scripts sent from the SSO Manager as `root` (used for automated GitOps). |
| **LDAP Tunnel** | `ldap_tunnel` | Moderate | Serves a local LDAP byte-pump socket (`ldap_socket`, default `/run/theta/ldap.sock`) for SSSD/PAM. The agent never parses LDAP — it forwards raw bytes to the SSO, which relays them into its own OpenLDAP. |
| **Secrets** | `secrets` | Moderate | Renders OpenBao secrets to local files from templates (see [Secrets Engine](#secrets-engine---rendering-openbao-secrets-to-local-files) below). |
| **IAM** | `iam` | Critical | Applies SSO-pushed node identity config: sudo rules, SSH `AuthorizedKeysCommand` keys, `/etc/security/access.conf`, and revocation (`sss_cache -E` + session kill). Every push is Ed25519-signed. |
---
## High-Risk Command Verification (Protocol v1.2.0)
## High-Risk Command Verification (Protocol v1.1.0)
High-risk management commands (`reboot`, `service_restart`, `configure_ldap`, `arbitrary_bash`, `update_binary`) are cryptographically verified using **Ed25519 signatures**:
1. The SSO Manager canonicalizes the command payload (sorted keys, no whitespace,
no HTML escaping, `signature` omitted).
1. The SSO Manager canonicalizes the command payload (sorted keys, no whitespace).
2. The payload is signed with the SSO Manager's Ed25519 private key.
3. The Base64 signature is appended to the message payload.
4. The agent verifies the signature against the configured `public_key` in `/etc/theta42/agent.yml` before executing the action.
**The signing key is persistent.** It lives in OpenBao at
`secret/agent/signing-key` and survives restarts, so the `public_key` you pin in
`agent.yml` keeps matching. (It used to be generated in memory at boot and
changed on every restart, which made pinning impossible.) If the SSO cannot load
or store a key it **refuses** to send high-risk commands rather than signing with
one no agent has seen — `GET /api/agent/nodes` reports this as
`signingAvailable: false`.
This requires the `sso-broker` OpenBao policy to grant `secret/agent/*`. Re-run
`./setup.sh` from theta-suite if you are upgrading.
**Verification is fail-closed on the agent.** An agent with no `public_key`
configured rejects every high-risk command. Earlier versions logged "skipping
signature verification" and executed them, so an agent installed without a key
would run `reboot`, `configure_ldap` and `arbitrary_bash` unverified.
---
## Secrets Engine — rendering OpenBao secrets to local files
The agent can render OpenBao secrets to local files that any process on the
host — a bash script, a systemd unit, a Node app, whatever — reads like an
ordinary env file. The agent never holds a Vault token: it asks the SSO for the
values over its existing WSS channel, and the SSO fetches them from OpenBao
using its own access, scoped so the agent can only ever read its own node's
secrets.
**Node scope.** Every path an agent can request must start with
`secret/data/nodes/<this-agent's-id>/`. The SSO enforces this server-side
(`POST /api/v1/agent/secrets`); a request for any other node's path is
rejected:
```
$ curl -sk https://sso.example.com/api/v1/agent/secrets \
-H "Authorization: Bearer <agent-token>" -H 'Content-Type: application/json' \
-d '{"paths":["secret/data/nodes/some-other-node-id/db"]}'
{"status":"error","message":"path outside node scope: secret/data/nodes/some-other-node-id/db"}
```
A compromised agent can therefore never reach another host's secrets, or
anything outside `secret/data/nodes/*`.
### Walkthrough: a 3rd-party app reads a secret the agent rendered
This walks through the whole path end to end, on a stack freshly brought up
from theta-suite's own `docs/fixtures.md` demo data — the same steps work on
any theta-suite install.
**1. Enroll the host.** Directory → Install Agent → mint a join key, run the
install command on the target host as root.
<a href="images/agent-install-join-key.png" target="_blank"><img src="images/agent-install-join-key.png" alt="Install Theta Agent modal with a freshly minted join key and install command" width="80%"></a>
On first connect the agent exchanges the join key for its own token + the
SSO's public key and writes both back into `/etc/theta42/agent.yml`. Note the
agent's id from `GET /api/agent/nodes` (or the Directory URL) — you need it for
the next step.
**2. Turn on the `secrets` capability and point it at a template.** Add to the
host's `/etc/theta42/agent.yml`:
```yaml
secrets:
- template: /etc/theta/templates/db.env.tpl
target: /etc/theta/rendered/db.env
reload: "" # optional: e.g. "systemctl reload myapp"
capabilities:
secrets: true
```
And the template itself, `/etc/theta/templates/db.env.tpl` — placeholders are
`{{ bao "secret/data/nodes/<agent-id>/<name>#<key>" }}`:
```
DB_USER="{{ bao "secret/data/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/db#username" }}"
DB_PASS="{{ bao "secret/data/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/db#password" }}"
```
Restart the agent to pick up the config change.
**3. Seed the secret.** From `theta-suite/` (theta-env), as the operator:
```
./setup.sh --seed-node-secret f9a30ab0-7d8a-4b77-a4c4-6a6383d084db db \
username=demoapp password=CorrectHorseBattery42
```
This writes to `secret/nodes/<agent-id>/db` in OpenBao (the CLI path — the HTTP
API the agent uses sees it as `secret/data/nodes/<agent-id>/db`, matched by the
node-scope check above). It's idempotent: it skips silently if that path is
already seeded.
**4. Trigger the render.** The Directory UI doesn't have a button for this yet
— push it the same way any admin command goes out, `POST
/api/agent/nodes/:id/command`. It's in the high-risk list, so the SSO signs it
automatically:
```
curl -X POST https://sso.example.com/api/agent/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/command \
-H "auth-token: <admin session token>" -H 'Content-Type: application/json' \
-d '{"command": "render_secrets", "payload": {}}'
```
The agent logs `Received command: render_secrets` / `Rendering secret
templates...` and atomically writes the target file at mode `0600`:
```
$ cat /etc/theta/rendered/db.env
DB_USER="demoapp"
DB_PASS="CorrectHorseBattery42"
```
Back in the Directory, the host's Metrics tab shows **Secrets** lit up green
among the reported capabilities:
<a href="images/agent-capabilities-metrics.png" target="_blank"><img src="images/agent-capabilities-metrics.png" alt="Directory Metrics tab showing live telemetry and the agent's reported capability badges, with Telemetry and Secrets lit green" width="80%"></a>
**5. Read it from a bash app on the same host.** The rendered file is just an
env file — no agent involvement needed to consume it:
```sh
#!/bin/sh
. /etc/theta/rendered/db.env
echo "DB_USER=$DB_USER"
echo "DB_PASS=$DB_PASS"
```
**6. Read it from a Node app on the same host:**
```js
const fs = require('fs');
const env = fs.readFileSync('/etc/theta/rendered/db.env', 'utf8');
const db = {};
for (const line of env.split('\n')) {
const m = /^(\w+)="(.*)"$/.exec(line.trim());
if (m) db[m[1]] = m[2];
}
console.log('DB_USER=' + db.DB_USER);
console.log('DB_PASS=' + db.DB_PASS);
```
Both print the same values the template resolved — `demoapp` /
`CorrectHorseBattery42` in this walkthrough. `theta-agent/demo/` in the
theta-agent repo has these two scripts ready to run.
### Alternative: calling the API directly
Rendering to a file is the normal path — it works for any app regardless of
language, and the secret never touches an HTTP client the app itself controls.
But an app can also fetch its node's secrets directly, bypassing the template
engine entirely (useful for debugging, or a process that wants to hold the
value only in memory). This uses the **agent's own bearer token**, not an admin
token — the same node-scope enforcement applies:
```sh
curl -sk https://sso.example.com/api/v1/agent/secrets \
-H "Authorization: Bearer <agent-token>" -H 'Content-Type: application/json' \
-d '{"paths":["secret/data/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/db"]}'
```
```js
const token = process.env.THETA_AGENT_TOKEN; // from /etc/theta42/agent.yml
fetch('https://sso.example.com/api/v1/agent/secrets', {
method: 'POST',
headers: { Authorization: 'Bearer ' + token, 'Content-Type': 'application/json' },
body: JSON.stringify({ paths: ['secret/data/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/db'] })
}).then(r => r.json()).then(d => console.log(d.secrets));
```
Both return:
```json
{
"status": "ok",
"secrets": {
"secret/data/nodes/f9a30ab0-7d8a-4b77-a4c4-6a6383d084db/db": {
"username": "demoapp",
"password": "CorrectHorseBattery42"
}
}
}
```
---
## Installation & Deployment
@@ -404,14 +80,9 @@ Both return:
Run the following command as `root` on the target Linux host:
```bash
curl -fsSL https://<SSO_HOST>/resources/theta-agent/install.sh | sh -s -- \
--url "https://<SSO_HOST>" --token "<ISSUED_TOKEN>" --public-key "<BASE64_PUBLIC_KEY>"
curl -fsSL https://<SSO_HOST>/resources/theta-agent/install.sh | sh -s -- --url "https://<SSO_HOST>" --token "<HOST_TOKEN>"
```
Both values come from enrollment. The **Install Agent** modal builds this line
for you with them already filled in. Omitting `--public-key` leaves the agent
able to report telemetry but unable to accept any high-risk command.
### Custom Config Wizard
You can generate a Base64-encoded custom configuration using the **Install Agent** button on the **Directory Management** page in the SSO Manager UI:
@@ -426,18 +97,9 @@ curl -fsSL https://<SSO_HOST>/resources/theta-agent/install.sh | sh -s -- "<BASE
```yaml
# /etc/theta42/agent.yml
server_url: "wss://sso.example.com"
# Issued by the SSO. Left empty when installing with a join key -- the agent
# fills it in itself once the server enrolls it.
auth_token: "c8181ce0e55bf7302b11d719a7ae39adcd7604de461e6e363f8bb4fadf126acb"
# Bootstrap credential. Used only while auth_token is empty, and blanked by the
# agent once it has its own token.
join_key: ""
auth_token: "your-unique-host-token"
location: "dc-01-rack-12"
# Base64 of the RAW 32-byte Ed25519 public key -- exactly the `publicKey` value
# from enrollment or GET /api/agent/nodes. Not a PEM body: a base64-decoded
# SPKI blob is 44 bytes, the agent requires 32, and it will refuse every signed
# command if this is wrong.
public_key: "D0cJB3iuStTzhXlu7tFDh/eEXFxRZwkuwQJJhFSqwlQ="
public_key: "MCowBQYDK2VwAyEA..."
capabilities:
telemetry: true
@@ -449,31 +111,6 @@ capabilities:
---
## Troubleshooting: agent is rejected (`close 4001`)
If the agent logs that the server rejected its token, the enrollment — not the
network — is the problem. The SSO accepts the WebSocket upgrade and then closes
with an application code:
| Code | Meaning | Fix |
| :--- | :--- | :--- |
| `4001` | Token unknown, or never issued by this server | Enroll the host and put the issued token in `agent.yml` |
| `4002` | Superseded — another connection authenticated as this agent | Normal; two copies of the agent are running |
| `4003` | Enrollment revoked or deleted | Re-enroll |
| `4004` | Token rotated; `agent.yml` has the old value | Copy the new token |
The agent backs off for 5 minutes on `4001`/`4003`/`4004` rather than retrying
every 5 seconds — a credential that is wrong will not fix itself, and hammering
the SSO only floods its audit log.
An agent installed before protocol v1.2.0 carries a token generated in the
browser that the server never recorded, so it will be rejected with `4001` until
re-enrolled. The quickest fix is to put a **join key** in its `agent.yml` as
`join_key` and blank `auth_token` — it will re-enroll itself on the next
reconnect.
---
## Troubleshooting: agent can't connect (`dial tcp ... i/o timeout`)
If the agent host logs `Dial error: dial tcp <ip>:443: i/o timeout` while
@@ -495,3 +132,5 @@ Fix options:
> sure the proxy has a **persistent Host record** for the real SSO domain — not
> just the `localtest.me` placeholder — so routing survives a proxy restart
> (an in-memory lookup cache can mask a missing Redis record for up to ~1h).
+1 -15
View File
@@ -16,27 +16,13 @@ deep-merges, in order (later wins):
`localhost`, `SSO Manager`).
2. `conf/<NODE_ENV>.js` — optional, environment-specific.
3. `conf/secrets.js` — gitignored; secrets + per-deployment values.
4. **`app_*` environment variables** — the highest-precedence layer among these
four.
4. **`app_*` environment variables** — the highest-precedence layer.
Any env var whose name starts with `app_` overrides the merged config. The rest
of the name splits on **double-underscore** (`__`) into a nested path. Values are
`JSON.parse`-coerced when possible (numbers, booleans, null, JSON) and kept as
raw strings otherwise.
### A fifth, higher-precedence layer: OpenBao + the Configuration UI
In a theta-suite deployment, `@simpleworkjs/bao-conf`'s `init()` deep-merges
`secret/sso-manager/conf` (from OpenBao) over the four layers above at boot —
this is the layer `setup.sh`/theta-suite actually manages, and it wins over
everything else here. On top of that, the admin **Configuration** page in the
UI writes straight to `secret/sso-manager/conf` (via `routes/api_conf.js`)
and applies the change to the live `conf` object immediately
(`applyToLiveConf`) — no restart, and it bypasses `conf/secrets.js` entirely.
If a value isn't behaving the way `conf/secrets.js` says it should, check the
Configuration UI / OpenBao before assuming a file edit didn't take — it's
almost certainly OpenBao (or a live UI edit) winning the merge.
## Examples
| Env var | Sets | Type |
+2 -22
View File
@@ -78,19 +78,7 @@ Requests are decided by the resource's `owner`, or by any directory admin. Mark
## Navigating the UI
The Directory Management interface nests your resources as a tree, making it easy
to comprehend your network topography at a glance. You can filter, search, and
sort your entire infrastructure inventory. Click the green `+` icon next to any
resource to add a child resource beneath it.
**Collapsing the tree.** Any resource with children carries a caret; click it to
fold that subtree away. The toolbar's double-chevron buttons expand or collapse
everything at once. Collapsed state is remembered per browser, so the shape you
arrange survives a refresh (and the self-heal reload that follows most edits).
While a search filter is active every match is shown regardless of collapsed
ancestors — otherwise searching for something inside a folded subtree would
silently return nothing. Clearing the box restores your saved shape.
The Directory Management interface provides a **Tree View** toggle that visually nests your resources, making it easy to comprehend your network topography at a glance. You can also filter, search, and sort your entire infrastructure inventory. From the tree view, you can click the green `+` icon next to any resource to instantly add a child resource beneath it.
<a href="images/directory.png" target="_blank"><img src="images/directory.png" alt="Directory & inventory list view" width="80%"></a>
@@ -114,17 +102,9 @@ You don't have to build the graph by hand — the theta42 tooling registers itse
- a **site** (name from `CFG_SITE_NAME` in `setup.env`, default `local` → slug `site_local`) marked as the current site
- the **host** the stack runs on (`host_<hostname>`), with IP, MAC address, OS, and kernel collected from the machine
- the **hosts** for the proxy and jump host (`host_theta-proxy`, `host_theta-jump`)
- the **services** it composes — SSO Manager, Proxy (management UI), OpenLDAP Directory (the LDAPS endpoint Linux hosts and LDAP-native apps bind to), OpenResty Edge (the 80/443 data plane), and the SSH Jump Host — each with its address, internal port, and git repo
- the **services** it composes — SSO Manager, Proxy (management UI), OpenLDAP Directory (the LDAPS endpoint Linux hosts and LDAP-native apps bind to), and OpenResty Edge (the 80/443 data plane) — each with its address, internal port, and git repo
- the proxy's auto-registered **OAuth client**, linked under its service
Services are parented to the host that actually runs them: Proxy and OpenResty
Edge under `host_theta-proxy`, the SSH Jump Host under `host_theta-jump`, and the
rest under the stack host. Installs seeded before this was fixed had all of them
under the stack host, leaving the two purpose-made host resources childless; the
seed re-parents those on its next run, and only when the current parent is the
one the old code set, so a layout you arranged deliberately is left alone.
The seed is idempotent and non-destructive: a resource whose slug already exists is considered operator-owned — the seed only fills in metadata fields you haven't set, and never overwrites your values.
### Linux hosts (ldap-client)
-117
View File
@@ -1,117 +0,0 @@
---
layout: default
title: Discovery & Inventory
nav_order: 6
---
# Discovery & Inventory
[← Back to Home](index.html)
The Directory holds two different kinds of thing, and the distinction matters
for every consumer of the directory:
- **Catalog resources** — what you have declared. Created by hand, seeded by
`setup.sh`, or *promoted* from a discovery result. These get LDAP access
groups, appear in the Catalog, and are the only hosts the
[jump host](https://github.com/theta42/jump-host) will connect you to.
- **Discovered resources** — what the network reports. Produced by
[discovery plugins](plugins.html) and shown on the **Discovered Inventory**
tab. They are a queue of "this exists, do you want to manage it?", not
infrastructure you have committed to.
A resource is discovery-only when its `metadata.discovery_sources` is non-empty
and it has never been promoted. Promoting sets `metadata.managed = true`, at
which point it becomes catalog content like any other resource.
> Nothing grants access to a discovered resource. It carries no groups until it
> is promoted, and the jump host applies the same rule — an unpromoted Proxmox
> guest is not a jump target.
---
## Where discovered data comes from
| Source | What it reports |
| :--- | :--- |
| [Proxmox](plugins.html) | The cluster endpoint, its nodes, and every VM/LXC with NICs, `vmid` and node |
| [UniFi](plugins.html) | Network devices and connected clients, by MAC |
| [nmap](plugins.html) | Hosts and open ports on a target range |
| [Docker](plugins.html) | Containers on a local or remote daemon |
| [theta-agent](agents.html) | The host it runs on — OS, kernel, CPU, RAM, disk, addresses |
| [ldap-client](directory.html) | A Linux host registering itself when it joins |
An agent is the most authoritative of these: it runs *on* the machine it
describes. A network scan is the least — it only knows what answered.
---
## How results are matched to existing resources
Every source runs through one reconciler, so two sources seeing the same
machine converge on one resource instead of creating duplicates. Matching is
tried in order of precision:
1. **MAC address** — the strongest signal, compared across every interface.
2. **IP address** — any address on any interface, plus `metadata.address`.
3. **Slug, name, or base hostname** — last resort.
A candidate must also be **the same kind**. Without that guard a discovered VM
named `gitea-runner` would match a hand-created *service* of the same name on
rule 3 and overwrite it. (`template` counts as `host`: converting a VM to a
template is the same machine.)
When a match is found the metadata is merged, interfaces are unioned by MAC, and
the source is added to `discovery_sources` — so a resource can legitimately read
`["unifi", "proxmox"]`, meaning two independent sources agree it exists.
### Naming
Sources disagree about names, so the most human one wins: a **hostname** beats
an **IP-shaped** name, which beats a **MAC-shaped** name; length is only a
tie-break within a rank. This is why a device UniFi knows only as
`ac:16:2d:b3:da:80` is renamed `dl380-0` once Proxmox reports it.
### Relationships
Plugins emit edges as well as resources (a Proxmox node under its cluster
endpoint, a guest under its node). The reconciler refuses any edge that would
make a resource its own parent, or that would close a loop — a cycle renders as
an infinitely nested tree and breaks every ancestor walk in the app.
---
## Promoting a discovered resource
On the **Discovered Inventory** tab, press **Promote**. The resource form opens
pre-filled with what was discovered — name, kind, address, subtype — so you can
correct it before committing. Saving marks it managed and provisions its
[LDAP groups](groups.html).
Each row shows what the directory knows about the device: its source(s), its
`vmid` where applicable, the identifier it has at that source (`sourceId`, e.g.
`dl380-0/qemu/234`), and every interface with its MAC and address. If a row
looks wrong, that detail is where to start.
---
## Stale results
Resources that are *only* auto-discovered are garbage-collected: if a source
stops reporting one for long enough it is marked
`lifecycle_state: "archived"` rather than deleted. Anything you created or
promoted is never touched — `manual` in `discovery_sources` exempts it.
A Proxmox node that is powered off is still reported (with its `status`), so
downtime does not look like decommissioning.
---
## What the stack discovers about itself
`setup.sh` seeds its own components as catalog resources — the site, the stack
host, `theta-proxy` and `theta-jump`, and the services under them. The Docker
discovery plugin then finds the containers backing them. Containers belonging to
the theta-suite compose project are recognised and attached to the service they
implement rather than appearing as unmanaged strangers, so a fresh install has an
empty Discovered Inventory rather than five things demanding attention.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 401 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 430 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 332 KiB

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 503 KiB

After

Width:  |  Height:  |  Size: 392 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 119 KiB

After

Width:  |  Height:  |  Size: 430 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 358 KiB

After

Width:  |  Height:  |  Size: 313 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 320 KiB

After

Width:  |  Height:  |  Size: 221 KiB

-3
View File
@@ -60,10 +60,7 @@ backend, that's the niche.
run the pieces separately via `app_*` env config.
- **Geo-Location Scaling** — built-in support for N-Way Multi-Master OpenLDAP [replication](replication.html) across physical sites.
- **[Directory & Inventory](directory.html)** — map sites, hosts, and services as a graph with rich metadata (IP/MAC, OS/kernel, ports, git repos), auto-provisioned access groups, and automatic registration from theta-env and ldap-client. Drives directory-aware tools like the [SSH jump host](https://theta42.github.io/jump-host/).
- **[Discovery](discovery.html)** — the catalog-vs-discovered distinction, how scanned assets are matched/merged into existing resources, and how a discovery gets promoted into the catalog (and becomes reachable through the jump host).
- **[Theta Agent & Endpoint C2](agents.html)** — 2-way Go daemon (`theta-agent`) for real-time telemetry (CPU, RAM, Disk, ZFS, GPU), automated host discovery, SSSD/LDAP configuration, and local capability-controlled management operations.
- **[Vault secrets](vault.html)** — an OpenBao-backed key-value store built into the UI, for stashing passwords/API keys/credentials with encryption and access control.
- **[API tokens](concepts-api-tokens.html)** — self-service personal access tokens for calling the management API from scripts/CI without a browser session.
## Get it
+1 -53
View File
@@ -17,63 +17,11 @@ needs theta-suite ≥ v1.30.1 (which grants the `sso-broker` OpenBao policy
A plugin type is a module under `nodejs/plugins/<category>/<type>.js`. The
filename basename (without `.js`) is the `type`; the parent directory is the
`category`. Two built-in categories ship today:
**`discovery`** — scheduled scans that sync external assets into the
directory catalog:
`category`. The built-ins ship under `plugins/discovery/`:
- `proxmox` — Proxmox VE (URL + API token)
- `unifi` — UniFi Network controller (URL + username/password)
- `nmap` — nmap OS + port scan (a target range; no credentials)
- `docker` — Docker daemon discovery (containers as directory resources)
**`messaging`** — on-demand delivery for alerts, 2FA codes, and
notifications:
- `twilio` — Twilio SMS
- `webhook` — universal REST webhook (custom JSON payload to Slack, Teams,
Discord, or any HTTP endpoint)
If no messaging plugin instance is enabled, the system falls back to the
legacy `voipms` integration configured directly in the SSO secrets.
### What the Proxmox plugin produces
One endpoint becomes one subtree:
```
Proxmox endpoint (cluster name, or the endpoint hostname)
└── node (hypervisor)
├── VM / template
└── LXC / template
```
The endpoint resource stands for the cluster, not a machine, so it carries the
API URL and a `sourceId` but deliberately no IP — giving it the address it is
reached at made the reconciler merge it with the node answering on that address,
which produced a resource that was its own parent.
Every guest carries:
- `interfaces[]` — one entry per NIC with its own `mac`, `ip`/`ips` and `name`.
The MAC and the address on it are read from the same source, so they cannot be
mismatched (an earlier version collected MACs and IPs into two flat lists and
zipped them by index, which attributed addresses to the wrong NIC on any
multi-NIC guest).
- `macAddress` / `ip` — the primary NIC's values, preferring one that actually
has an address.
- `vmid`, `node` and `sourceId` (`<node>/qemu/<vmid>` or `<node>/lxc/<vmid>`), so
a directory row traces back to the exact guest on the exact node.
Interfaces belonging to something running *inside* a guest — `docker0`, `veth*`,
`br-*`, VPN tunnels — are filtered out. They are not NICs of the host, and their
172.x addresses would otherwise give the reconciler spurious matches.
A stopped VM still reports its MAC (read from the VM config rather than the
guest agent), and a DHCP-configured LXC gets its address from the running
container's interface list. Offline nodes are recorded with `status` rather than
skipped, so a hypervisor that is down does not look decommissioned and get
garbage-collected after a week.
A module exports a **manifest**:
+4 -52
View File
@@ -1,13 +1,5 @@
---
layout: default
title: Vault Secrets
description: OpenBao-backed personal, shared, and external-app secret storage built into the SSO Manager UI.
---
# Vault Secrets Management
[← Back to Home](index.html)
The Vault Secrets feature integrates with OpenBao to provide a secure key-value store for your environment. It allows you to store sensitive information like passwords, API keys, and credentials, ensuring they are encrypted and access-controlled.
## Usage
@@ -34,53 +26,13 @@ You can access the Vault UI from the application's top navigation bar.
### OpenBao Integration
The secrets are stored in a real, initialized-and-unsealed OpenBao backend
(`setup.sh` handles init/unseal on first run) — not OpenBao's ephemeral dev
mode, which auto-unseals with an in-memory store and loses everything on
restart. The default KV (Key-Value) version 2 engine is mounted at `secret/`.
The built-in UI proxies through `/api/vault/secret/…`, authenticated the same
way as the rest of the app (session cookie or a personal API token) — the
server resolves your OpenBao access itself and injects the right scoped
token; you never see or handle a raw OpenBao token as a UI user.
## Apps tab (admin)
The **Apps** tab mints a scoped OpenBao token for an **external application** so it can read its own configuration out of OpenBao — a downstream-app credential, not a per-user secret.
1. Enter an app **name** (e.g. `my-service`) and click **Mint token**.
2. A token is shown **once** — copy it into the external app now; it cannot be recovered later. The app uses it as the `X-Vault-Token` header against `secret/apps/<name>/*` (see the connection convention shown on the page).
3. The **Minted apps** list shows every token you've created (metadata only — the token itself is never stored). sso keeps each token alive by renewing it periodically, so a downstream app's credential stays valid as long as sso runs. If an app shows a **renewal error**, re-mint it here — that revokes the old token and issues a fresh one.
The token is scoped to `secret/apps/<name>/*` only (policy `app-<name>`), so a compromised token can't touch any other secret.
## Shared tab
The **Shared** tab lets you share a secret with another user (or app) without copying the value around.
1. **New** — give the secret a name (slug) and its JSON data. The owner has full read/write on `secret/shared/<uid>/<slug>`.
2. Open a secret and use **Grants** to share it with a user or app; the grantee's OpenBao policy is edited immediately so the share takes effect with no token re-mint. Revoking a grant removes access at the ACL.
3. The data itself is read through the normal Vault proxy using each user's own session, so OpenBao enforces read access per-request.
The secrets are stored in an OpenBao backend configured in development mode. The default KV (Key-Value) version 2 engine is mounted at `secret/`. The built-in UI uses the `/api/vault/secret/` API endpoints to interact with OpenBao.
## API Access
To read your own secrets programmatically, call the `/api/vault` proxy with
a [personal API token](concepts-api-tokens.html) — **not** a raw OpenBao
token. The server authenticates the request, resolves your own scoped
OpenBao access, and injects the real `X-Vault-Token` itself:
If you need to programmatically access the secrets, you can interact directly with the OpenBao API using the root token (in dev mode):
```bash
# Example: Read a secret via the API (KV-v2, so the path includes /data/)
curl -H "Authorization: Bearer sso_<id>_<secret>" \
https://<your-sso-host>/api/vault/secret/data/<your-secret-path>
# Example: Read a secret via the API
curl -H "X-Vault-Token: root" -H "Authorization: Bearer <your-sso-token>" http://<your-sso-host>/api/vault/secret/data/<your-secret-path>
```
An **external app** reading its own config uses the scoped token minted for
it on the **Apps** tab instead of a personal token — see *Apps tab (admin)*
above for how that token is minted and what it's confined to.
Using the OpenBao **root token** directly (bypassing the SSO entirely) is
never the intended path for day-to-day secret access — it's an
operator/maintenance credential (seeding, disaster recovery), kept in
`setup.env` and never passed to a service container. See
[theta-env's Secrets doc](https://theta42.github.io/theta-env/secrets.html)
for the full token/policy model.
+3 -20
View File
@@ -44,10 +44,9 @@ app.onListen.push(function(){
});
});
// Initialize Theta Agent WebSockets. The REST router is already mounted
// synchronously above (see the /api/agent mount); this hook only wires the WS.
require('./routes/api_agent').initAgentWebSockets(app);
});
// Initialize Theta Agent WebSockets
require('./routes/api_agent')(app);
});
// Gzip text responses (HTML/JS/CSS/JSON). The admin UI loads ~13 separate,
// uncompressed vendor JS/CSS files on every full page navigation (a
@@ -106,22 +105,6 @@ app.use('/api/conf', middleware.auth, require('./routes/api_conf'));
// Self-service API tokens (PATs) — owner-scoped, no admin group required.
app.use('/api/api-token', middleware.auth, require('./routes/api_token'));
// theta-agent REST API. Mounted SYNCHRONOUSLY (before the 404 catch-all below),
// not from an onListen hook — a router registered post-listen would sit behind
// the terminal 404 handler and make every /api/agent/* request 404. The agent
// WebSocket handler (routes/api_agent.initAgentWebSockets) still runs on onListen.
app.use('/api/agent', require('./routes/api_agent'));
// LDAP-over-HTTPS API (DESIGN.md §3). Bearer-authed (agent token or PAT); the
// SSO performs the real LDAP bind/search against its own OpenLDAP. Mounted
// synchronously for the same reason as /api/agent — it must sit before the 404
// catch-all.
app.use('/api/v1/ldap', require('./routes/api_ldap'));
// Agent-facing operations (DESIGN.md §5, §6): node-scoped secrets, IAM. The
// caller is the agent itself (Bearer agent token), not an admin session.
app.use('/api/v1/agent', require('./routes/api_agent_ops'));
// OAuth 2.0 / OpenID Connect
app.use('/oauth', oauthRouter);
app.use('/api/oauth', middleware.auth, oauthApiRouter);
-182
View File
@@ -1,182 +0,0 @@
'use strict';
const crypto = require('crypto');
const { Model } = require('@simpleworkjs/orm');
// A theta-agent enrolled against this SSO.
//
// Before this model existed the "agent token" was generated in the browser and
// never recorded anywhere, so the server had no way to tell an agent it issued
// from one someone invented -- /api/agent/ws accepted any string, and there was
// no way to revoke a token or to know that an agent existed while it was
// offline. The row is now the authority: an agent is only real if it is here.
//
// The raw token is shown exactly once, at enrollment. Only its SHA-256 lands in
// the database, so a database disclosure does not hand over working agent
// credentials. `tokenPrefix` is the first 8 characters, kept in the clear so the
// UI and logs can identify an agent without holding the secret.
class Agent extends Model {
// Tokens are compared by hash on every WebSocket connect. SHA-256 (not
// bcrypt) is deliberate: this runs on the connection path and the token is a
// 256-bit random value, not a human-chosen password, so there is nothing for
// a slow KDF to protect against here.
static hashToken(raw) {
return crypto.createHash('sha256').update(String(raw || ''), 'utf8').digest('hex');
}
static generateToken() {
return crypto.randomBytes(32).toString('hex');
}
// Resolve a presented token to its (non-revoked) agent, or null. Every
// caller that authenticates an agent must go through here.
static async authenticate(rawToken) {
if (!rawToken || typeof rawToken !== 'string') return null;
const tokenHash = this.hashToken(rawToken);
const matches = await this.list({ where: { tokenHash } });
const agent = matches && matches[0];
if (!agent) return null;
if (agent.revoked) return null;
return agent;
}
// Enroll a new agent and return { agent, token }. The caller is responsible
// for showing `token` to the operator once and never storing it.
static async enroll({ name, resourceId, enrolledBy, description }) {
const token = this.generateToken();
const agent = await this.create({
id: crypto.randomUUID(),
name: name || 'theta-agent',
description: description || null,
tokenHash: this.hashToken(token),
tokenPrefix: token.slice(0, 8),
resourceId: resourceId || null,
revoked: false,
enrolled_by: enrolledBy || null,
enrolled_on: Math.floor(Date.now() / 1000)
});
return { agent, token };
}
// Issue a fresh token for an existing agent, invalidating the old one.
async rotateToken() {
const token = Agent.generateToken();
await this.update({
tokenHash: Agent.hashToken(token),
tokenPrefix: token.slice(0, 8),
revoked: false
});
return token;
}
static fields = {
id: { type: 'uuid', primaryKey: true },
name: { type: 'string', isRequired: true },
description: { type: 'text' },
// Never the raw token. See hashToken above.
tokenHash: { type: 'string', isRequired: true },
tokenPrefix: { type: 'string' },
// The host this agent runs on. Nullable so an agent can be enrolled
// before its host exists in the Directory, but the UI pushes for it:
// without this link there is nothing to hang resource control off, and
// the old code had to guess by matching hostnames to slugs.
resource: { type: 'hasOne', model: 'Resource' }, // creates resourceId
revoked: { type: 'boolean', default: false },
enrolled_by: { type: 'string' },
enrolled_on: { type: 'integer' },
// Survives a restart, which the in-memory map did not: an agent that is
// installed but currently down is now distinguishable from one that was
// never enrolled.
last_seen: { type: 'integer' },
last_ip: { type: 'string' },
lastDiscovery: { type: 'json', default: {} },
lastTelemetry: { type: 'json', default: {} }
};
// The shape the admin API returns. Never includes tokenHash.
toPublic(liveState) {
const data = this.toJSON ? this.toJSON() : { ...this };
delete data.tokenHash;
return {
...data,
connected: !!(liveState && liveState.connected),
// "Online" is a live-connection fact, not a stored one. A row with a
// last_seen from an hour ago is an installed agent that is down.
isOnline: !!(liveState && liveState.connected),
lastResponse: (liveState && liveState.lastResponse) || null
};
}
}
// A join key: the one credential an operator hands out so a host can enroll
// itself. Requiring an admin to pre-register every machine before the agent
// would talk to them made adding a host a two-system chore -- installing the
// agent should be enough.
//
// A join key is NOT the agent's long-term credential. On first connect the
// server auto-enrolls the host and issues it a unique per-agent token, which
// the agent persists and uses from then on (PROTOCOL.md 1.2). That keeps the
// operator experience to "one key" while still giving every host its own
// revocable identity -- revoking a single agent means something, and a host
// that is compromised does not hand over the credential for the whole fleet.
class AgentJoinKey extends Model {
static hashKey(raw) {
return crypto.createHash('sha256').update(String(raw || ''), 'utf8').digest('hex');
}
static generateKey() {
// `tjk_` so an operator can tell a join key from an agent token at a
// glance -- they are handled very differently.
return 'tjk_' + crypto.randomBytes(32).toString('hex');
}
// Resolve a presented key to a usable join key, or null. Expiry and
// revocation are both enforced here so no caller can forget one.
static async authenticate(rawKey) {
if (!rawKey || typeof rawKey !== 'string') return null;
const keyHash = this.hashKey(rawKey);
const matches = await this.list({ where: { keyHash } });
const key = matches && matches[0];
if (!key) return null;
if (key.revoked) return null;
if (key.expires_on && key.expires_on < Math.floor(Date.now() / 1000)) return null;
return key;
}
static async issue({ label, createdBy, expiresInDays }) {
const raw = this.generateKey();
const key = await this.create({
id: crypto.randomUUID(),
label: label || 'default',
keyHash: this.hashKey(raw),
keyPrefix: raw.slice(0, 12),
revoked: false,
created_by: createdBy || null,
created_on: Math.floor(Date.now() / 1000),
expires_on: expiresInDays ? Math.floor(Date.now() / 1000) + expiresInDays * 86400 : null,
use_count: 0
});
return { key, raw };
}
static fields = {
id: { type: 'uuid', primaryKey: true },
label: { type: 'string', isRequired: true },
keyHash: { type: 'string', isRequired: true },
keyPrefix: { type: 'string' },
revoked: { type: 'boolean', default: false },
created_by: { type: 'string' },
created_on: { type: 'integer' },
expires_on: { type: 'integer' },
use_count: { type: 'integer', default: 0 },
last_used_on: { type: 'integer' }
};
toPublic() {
const data = this.toJSON ? this.toJSON() : { ...this };
delete data.keyHash;
return data;
}
}
module.exports = { Agent, AgentJoinKey };
+1 -8
View File
@@ -33,15 +33,8 @@ Mail.send = function(to, subject, message, from){
var transporter = nodemailer.createTransport(transportOpts);
// Most authenticated SMTP relays (and this bit the field: "554 5.7.1
// ...: Sender is not same as SMTP authenticate username") require the
// envelope/header From to equal the authenticated user, or reject the
// send outright. If the operator hasn't set an explicit smtp.from,
// defaulting to the SMTP username is far more likely to actually send
// than a made-up noreply@theta42.com address that no relay authorized
// this account to send as.
var mailOpts = {
from: from || conf.smtp.from || conf.smtp.user || `${conf.name} Accounts <noreply@theta42.com>`,
from: from || conf.smtp.from || `${conf.name} Accounts <noreply@theta42.com>`,
to: to,
subject: subject,
html: message
+1 -2
View File
@@ -20,7 +20,6 @@ const { PluginInstance } = require('./plugin_instance');
const { SharedSecret } = require('./shared_secret');
const { SharedSecretGrant } = require('./shared_secret_grant');
const { VaultAppToken } = require('./vault_app_token');
const { Agent, AgentJoinKey } = require('./agent');
async function initORM() {
const ormConf = conf.orm || {
dialect: 'sqlite',
@@ -35,7 +34,7 @@ async function initORM() {
conf: { orm: ormConf },
models: [
Resource, ResourceEdge, ResourceGroup, AccessRequest, Webhook, PluginInstance,
SharedSecret, SharedSecretGrant, VaultAppToken, Agent, AgentJoinKey,
SharedSecret, SharedSecretGrant, VaultAppToken,
Token, AuthToken, InviteToken, ImpersonationToken, PasswordResetToken, OtpToken, ServiceToken
]
});
-18
View File
@@ -191,24 +191,6 @@ class Resource extends Model {
}
return null;
}
// Walk all parent ResourceEdges upwards recursively to find all ancestor
// resources (Host, Cluster, Site, etc.).
static async findAllAncestors(resourceId, visited = new Set()) {
if (visited.has(resourceId)) return [];
visited.add(resourceId);
const ancestors = [];
const parentEdges = await ResourceEdge.list({ where: { childId: resourceId } }).catch(() => []);
for (const edge of parentEdges) {
const parent = await this.get(edge.parentId).catch(() => null);
if (!parent) continue;
ancestors.push(parent);
const higher = await this.findAllAncestors(parent.id, visited);
ancestors.push(...higher);
}
return ancestors;
}
}
class ResourceEdge extends Model {
+1 -6
View File
@@ -14,12 +14,7 @@ async function send(to, message) {
const registry = require('../services/plugin_registry');
const pluginSecrets = require('../utils/plugin_secrets');
// @simpleworkjs/orm has no `find` -- the query method is `list({where})`.
// `PluginInstance.find(...)` threw "is not a function" on EVERY call into
// this sender, so SMS delivery never worked at all: not the test button, not
// OTP-by-SMS, not notifications. It failed before it could even fall back to
// the direct VoIP.ms path below.
const instances = await PluginInstance.list({ where: { category: 'messaging', enabled: true } });
const instances = await PluginInstance.find({ category: 'messaging', enabled: true });
if (instances.length > 0) {
const inst = instances[0];
const manifest = registry.getManifest(inst.pluginType);
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "t42-sso-manager",
"version": "1.30.2",
"version": "1.26.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "t42-sso-manager",
"version": "1.30.2",
"version": "1.26.1",
"license": "MIT",
"dependencies": {
"@fortawesome/fontawesome-free": "^7.3.0",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "t42-sso-manager",
"version": "1.31.0",
"version": "1.26.1",
"description": "A very simple LDAP management and SSO system",
"author": [
{
+6 -48
View File
@@ -7,15 +7,7 @@ module.exports = {
description: 'Discover running containers and networks from a local or remote Docker daemon.',
configSchema: [
{ key: 'socketPath', label: 'Docker Socket Path', type: 'text', required: false, placeholder: '/var/run/docker.sock' },
{ key: 'tcpHost', label: 'TCP Host (e.g., http://10.0.0.1:2375)', type: 'url', required: false, placeholder: '' },
// Containers in this compose project are the stack's own. They are already
// represented in the catalog as services, so they are recorded as managed
// and linked to the service they implement instead of arriving as
// unmanaged strangers a fresh install has to triage.
{ key: 'stackProject', label: 'Own compose project', type: 'text', required: false, placeholder: 'theta-suite' },
// The catalog host these containers run on, so they land in the tree
// instead of as roots.
{ key: 'hostSlug', label: 'Parent host slug', type: 'text', required: false, placeholder: 'host_<hostname>' }
{ key: 'tcpHost', label: 'TCP Host (e.g., http://10.0.0.1:2375)', type: 'url', required: false, placeholder: '' }
],
validate: async (config) => {
@@ -56,57 +48,23 @@ module.exports = {
const resources = [];
const edges = [];
const stackProject = (config.stackProject || '').trim();
const hostSlug = (config.hostSlug || '').trim();
for (const c of containers) {
const labels = c.Labels || {};
const composeProject = labels['com.docker.compose.project'] || '';
const composeService = labels['com.docker.compose.service'] || '';
const name = c.Names && c.Names.length > 0 ? c.Names[0].replace(/^\//, '') : c.Id.substring(0, 12);
// A container id changes every time the container is recreated,
// so an id-derived slug made `docker compose up` mint a brand-new
// resource on every deploy and orphan the previous one. Prefer
// identifiers that survive a recreate: the compose project+service
// it belongs to, else its name.
const stableKey = composeProject && composeService
? `${composeProject}-${composeService}`
: (name || c.Id.substring(0, 12));
const slug = `docker-${stableKey.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '')}`;
const slug = `docker-cnt-${c.Id.substring(0, 12)}`;
const ports = (c.Ports || []).map(p => p.PublicPort ? `${p.PublicPort}:${p.PrivatePort}` : `${p.PrivatePort}`).join(', ');
const isOwnStack = !!(stackProject && composeProject === stackProject);
resources.push({
kind: 'container',
name: composeService || name,
name: name,
slug: slug,
metadata: {
image: c.Image,
state: c.State,
status: c.Status,
ports: ports,
composeProject: composeProject || undefined,
composeService: composeService || undefined,
containerName: name,
sourceId: stableKey,
// Part of the deployment we are running inside: already
// accounted for, not something to promote.
managed: isOwnStack ? true : undefined
ports: ports
}
});
// Attach the container to the service it implements when the
// catalog already has one under that slug (the bootstrap seeds
// `sso-manager`, `proxy`, `jump-host`, … using the same names
// compose uses). The reconciler drops an edge whose parent does
// not resolve, so an unmatched name is simply not linked.
if (isOwnStack && composeService) {
edges.push({ parentSlug: composeService, childSlug: slug, relation: 'runs' });
} else if (hostSlug) {
edges.push({ parentSlug: hostSlug, childSlug: slug, relation: 'hosts' });
}
}
resolve({ resources, edges });
+33 -237
View File
@@ -6,81 +6,6 @@ const agent = new https.Agent({
rejectUnauthorized: false
});
// Accumulates a guest's NICs, keyed by MAC, merging what several Proxmox
// endpoints each know a piece of: the guest agent knows MAC+IP together, the
// VM/LXC config knows the MAC even while the guest is stopped, and the LXC
// interfaces endpoint knows the DHCP-assigned IP. Keying by MAC is what keeps
// the pairing honest -- the previous code collected MACs and IPs into two flat
// lists and zipped them by index, which mismatched them on any multi-NIC guest.
class Interfaces {
constructor() { this.byMac = new Map(); this.anonymous = []; }
// Interfaces that belong to something running INSIDE the guest -- container
// engines, overlay networks, VPNs -- rather than to the guest itself. A
// Home Assistant VM reported 16 of these (docker0, hassio, 14x veth*)
// alongside its one real NIC, which is noise in the directory and, worse,
// gives the reconciler a pile of 172.x addresses to match unrelated hosts on.
// Only applied to guests; a hypervisor's own bridges are how you reach it.
static VIRTUAL_IFACE_RE = /^(lo|docker\d*|hassio|veth|br-|virbr|tap|fwbr|fwln|fwpr|cni|flannel|cali|kube|weave|zt|tailscale|wg|tun|utun)/i;
static isVirtualName(name) {
return !!name && Interfaces.VIRTUAL_IFACE_RE.test(name);
}
// A udev "predictable" name of the form enx<12 hex> encodes the MAC. It is
// the only place the Proxmox node network API exposes a physical NIC's MAC
// (/nodes/{node}/network carries no hwaddr field at all), so parse it out
// rather than leaving every hypervisor MAC-less.
static macFromIfaceName(name) {
const m = /^enx([0-9a-f]{12})$/i.exec(name || '');
if (!m) return null;
return m[1].toLowerCase().match(/.{2}/g).join(':');
}
static normalizeMac(mac) {
const m = (mac || '').toLowerCase().trim();
if (!/^([0-9a-f]{2}:){5}[0-9a-f]{2}$/.test(m)) return null;
if (m === '00:00:00:00:00:00') return null;
return m;
}
// `ips` are the addresses observed on this one NIC (may be empty for a
// stopped guest, where only the MAC is known).
add(mac, ips, name) {
const key = Interfaces.normalizeMac(mac);
const addrs = (ips || []).filter(Boolean);
if (!key) {
// An IP with no usable MAC is still worth keeping; a NIC with neither is not.
if (addrs.length) this.anonymous.push({ mac: null, ip: addrs[0], ips: addrs, name: name || null });
return;
}
const existing = this.byMac.get(key);
if (existing) {
for (const ip of addrs) if (!existing.ips.includes(ip)) existing.ips.push(ip);
existing.ip = existing.ips[0] || null;
if (!existing.name && name) existing.name = name;
return;
}
this.byMac.set(key, { mac: key, ip: addrs[0] || null, ips: addrs, name: name || null });
}
toArray() { return [...this.byMac.values(), ...this.anonymous]; }
// The address/MAC the directory shows in its single-value columns, and what
// the reconciler matches on. Prefer a NIC that actually has an address.
primaryIp() {
const withIp = this.toArray().find(i => i.ip);
return withIp ? withIp.ip : null;
}
primaryMac() {
const withIp = this.toArray().find(i => i.ip && i.mac);
if (withIp) return withIp.mac;
const first = this.toArray().find(i => i.mac);
return first ? first.mac : null;
}
}
module.exports = {
// Plugin manifest — see nodejs/services/plugin_registry.js. `configSchema`
// drives the admin UI form and validation; fields flagged `secret:true` are
@@ -125,45 +50,6 @@ module.exports = {
const resources = [];
const edges = [];
// 0. The Proxmox endpoint itself. Without it a multi-node cluster produces
// several unrelated roots in the Directory tree and nothing says where any
// of them came from. Every node discovered below is parented to this, so
// one endpoint == one subtree.
const endpointHost = (() => {
try { return new URL(url).hostname; } catch (e) { return url.replace(/^https?:\/\//, '').split('/')[0]; }
})();
const clusterName = await (async () => {
// /cluster/status names the cluster when one exists; a standalone node
// has no cluster entry, in which case the endpoint hostname is the name.
try {
const res = await fetch(`${url}/api2/json/cluster/status`, { headers, agent });
if (!res.ok) return null;
const entry = ((await res.json()).data || []).find(d => d.type === 'cluster');
return entry ? entry.name : null;
} catch (e) { return null; }
})();
const endpointSlug = `pve-${endpointHost.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '')}`;
resources.push({
kind: 'host',
name: clusterName || `Proxmox (${endpointHost})`,
slug: endpointSlug,
metadata: {
subType: 'proxmox',
address: url,
os: 'Proxmox VE',
isProduction: true,
sourceId: url,
// Deliberately NO `ip`/`interfaces`: this resource stands for the
// cluster (the API endpoint), not for a machine. Giving it the address
// it is reached at made the reconciler match it to the very node that
// answers on that address -- the endpoint and the node collapsed into
// one row, which then became its own parent. The cluster is identified
// by slug + sourceId instead, which nothing else can collide with.
interfaces: []
}
});
// 1. Get Nodes
const resNodes = await fetch(`${url}/api2/json/nodes`, { headers, agent });
if(!resNodes.ok) {
@@ -173,57 +59,9 @@ module.exports = {
const nodes = (await resNodes.json()).data;
for (const node of nodes) {
// An offline node is still a real hypervisor that belongs in the
// directory -- skipping it entirely used to make it look decommissioned
// and let the reconciler's garbage collector archive it after a week of
// downtime. Record it, mark it down, and skip only the guest enumeration
// (which needs the node to answer).
const online = node.status === 'online';
if (node.status !== 'online') continue;
const nodeSlug = `pve-node-${node.node}`;
// A hypervisor with no address is not actionable. Read its bridges/NICs
// so the node lands in the directory reachable and MAC-identified like
// any other host. Unlike a guest, a node's bridges are kept: vmbrN is
// normally the address you actually reach the hypervisor on.
const nodeIfaces = new Interfaces();
try {
const netRes = online
? await fetch(`${url}/api2/json/nodes/${node.node}/network`, { headers, agent })
: { ok: false };
if (netRes.ok) {
const ifaceList = (await netRes.json()).data || [];
for (const iface of ifaceList) {
if (iface.iface === 'lo') continue;
const ip = iface.address || iface.cidr;
// This endpoint has no hwaddr field, so the MAC has to be recovered
// from a predictable interface name -- either this interface's own
// or, for a bridge, one of the physical ports beneath it.
let mac = Interfaces.macFromIfaceName(iface.iface);
if (!mac) {
for (const alt of (iface.altnames || [])) {
mac = Interfaces.macFromIfaceName(alt);
if (mac) break;
}
}
if (!mac && iface.bridge_ports) {
for (const port of String(iface.bridge_ports).split(/\s+/).filter(Boolean)) {
mac = Interfaces.macFromIfaceName(port);
if (mac) break;
// The port may itself only carry the MAC in an altname.
const portDef = ifaceList.find(i => i.iface === port);
for (const alt of ((portDef && portDef.altnames) || [])) {
mac = Interfaces.macFromIfaceName(alt);
if (mac) break;
}
if (mac) break;
}
}
nodeIfaces.add(mac || iface.hwaddr, ip ? [String(ip).split('/')[0]] : [], iface.iface);
}
}
} catch (e) {}
resources.push({
kind: 'host',
name: node.node,
@@ -232,19 +70,9 @@ module.exports = {
subType: 'hypervisor',
os: 'Proxmox VE',
isProduction: true,
status: node.status,
sourceId: `${node.node}`,
node: node.node,
interfaces: nodeIfaces.toArray(),
macAddress: nodeIfaces.primaryMac(),
ip: nodeIfaces.primaryIp()
interfaces: []
}
});
edges.push({ parentSlug: endpointSlug, childSlug: nodeSlug, relation: 'hosts' });
// Everything below asks the node itself; an offline node answers none of
// it, and its guests are already recorded from previous runs.
if (!online) continue;
// 2. Get VMs for this node
const resVms = await fetch(`${url}/api2/json/nodes/${node.node}/qemu`, { headers, agent });
@@ -254,12 +82,10 @@ module.exports = {
const vmSlug = `vm-${vm.vmid}`;
const isTemplate = vm.template === 1;
const ifaces = new Interfaces();
// Enrich from QEMU guest agent if running. The agent is the only source
// that knows which IP sits on which NIC, so pair them here rather than
// accumulating two flat lists (zipping those by index attributed IPs to
// the wrong MAC on any guest with more than one NIC).
let ips = [];
let macs = [];
// Enrich from QEMU guest agent if running
if (vm.status === 'running') {
try {
const agentRes = await fetch(`${url}/api2/json/nodes/${node.node}/qemu/${vm.vmid}/agent/network-get-interfaces`, { headers, agent });
@@ -267,35 +93,35 @@ module.exports = {
const agentData = (await agentRes.json()).data;
if (agentData && agentData.result) {
for (const iface of agentData.result) {
// Docker bridges, veth pairs and VPN tunnels are the
// guest's own plumbing, not NICs of the guest.
if (Interfaces.isVirtualName(iface.name)) continue;
const ips = (iface['ip-addresses'] || [])
.filter(ip => ip['ip-address-type'] === 'ipv4' && ip['ip-address'] !== '127.0.0.1')
.map(ip => ip['ip-address']);
ifaces.add(iface['hardware-address'], ips, iface.name);
if (iface['hardware-address'] && iface['hardware-address'] !== '00:00:00:00:00:00') macs.push(iface['hardware-address']);
if (iface['ip-addresses']) {
for (const ip of iface['ip-addresses']) {
if (ip['ip-address-type'] === 'ipv4' && ip['ip-address'] !== '127.0.0.1') {
ips.push(ip['ip-address']);
}
}
}
}
}
}
} catch(e) {}
}
// Enrich from VM config: the MAC is declared there whether or not the
// guest agent answered, so a stopped VM still gets a stable identity.
// Enrich from VM config to at least get MAC if agent failed/stopped
try {
const configRes = await fetch(`${url}/api2/json/nodes/${node.node}/qemu/${vm.vmid}/config`, { headers, agent });
if (configRes.ok) {
const confData = (await configRes.json()).data;
for (let i = 0; i < 10; i++) {
if (confData[`net${i}`]) {
const m = confData[`net${i}`].match(/(?:virtio|e1000e?|rtl8139|vmxnet3)=([0-9a-fA-F:]{17})/);
if(m) ifaces.add(m[1], [], `net${i}`);
const m = confData[`net${i}`].match(/(?:virtio|e1000|rtl8139|vmxnet3)=([0-9a-fA-F:]+)/);
if(m) macs.push(m[1].toLowerCase());
}
}
}
} catch(e) {}
const interfaces = ifaces.toArray();
const interfaces = [...new Set(macs)].map((mac, i) => ({ mac, ip: ips[i] || null }));
resources.push({
kind: isTemplate ? 'template' : 'host',
@@ -304,14 +130,9 @@ module.exports = {
metadata: {
subType: isTemplate ? 'template' : 'vm',
vmid: vm.vmid,
// The Proxmox-side identity, so a resource can be traced back to the
// exact guest on the exact node it was discovered from.
sourceId: `${node.node}/qemu/${vm.vmid}`,
node: node.node,
isProduction: vm.status === 'running',
interfaces,
macAddress: ifaces.primaryMac(),
ip: ifaces.primaryIp()
ip: ips[0] || null
}
});
edges.push({ parentSlug: nodeSlug, childSlug: vmSlug, relation: 'hosts' });
@@ -325,45 +146,26 @@ module.exports = {
const lxcSlug = `lxc-${lxc.vmid}`;
const isTemplate = lxc.template === 1;
const ifaces = new Interfaces();
// Enrich from LXC config. Each netN line carries its own hwaddr and ip,
// so read them off the same line instead of into parallel lists.
let ips = [];
let macs = [];
// Enrich from LXC config
try {
const configRes = await fetch(`${url}/api2/json/nodes/${node.node}/lxc/${lxc.vmid}/config`, { headers, agent });
if (configRes.ok) {
const confData = (await configRes.json()).data;
for (let i = 0; i < 10; i++) {
const line = confData[`net${i}`];
if (!line) continue;
const hwMatch = line.match(/hwaddr=([0-9a-fA-F:]{17})/);
// `ip=` is either a CIDR address or the literal `dhcp`/`manual`.
const ipMatch = line.match(/\bip=(\d+\.\d+\.\d+\.\d+)/);
const nameMatch = line.match(/\bname=([^,]+)/);
if (hwMatch || ipMatch) {
ifaces.add(hwMatch && hwMatch[1], ipMatch ? [ipMatch[1]] : [], nameMatch ? nameMatch[1] : `net${i}`);
if (confData[`net${i}`]) {
const hwMatch = confData[`net${i}`].match(/hwaddr=([0-9a-fA-F:]+)/);
const ipMatch = confData[`net${i}`].match(/ip=([0-9\.]+)/); // Ignores dhcp
if(hwMatch) macs.push(hwMatch[1].toLowerCase());
if(ipMatch) ips.push(ipMatch[1]);
}
}
}
} catch(e) {}
// A DHCP-configured container has no IP in its config. Ask the running
// container's interface list so it lands in the directory addressable
// instead of as an IP-less row.
if (lxc.status === 'running' && !ifaces.primaryIp()) {
try {
const ifRes = await fetch(`${url}/api2/json/nodes/${node.node}/lxc/${lxc.vmid}/interfaces`, { headers, agent });
if (ifRes.ok) {
for (const iface of ((await ifRes.json()).data || [])) {
if (Interfaces.isVirtualName(iface.name)) continue;
const ip = (iface.inet || '').split('/')[0];
ifaces.add(iface.hwaddr, ip ? [ip] : [], iface.name);
}
}
} catch(e) {}
}
const interfaces = ifaces.toArray();
const interfaces = [...new Set(macs)].map((mac, i) => ({ mac, ip: ips[i] || null }));
resources.push({
kind: isTemplate ? 'template' : 'host',
@@ -372,12 +174,9 @@ module.exports = {
metadata: {
subType: isTemplate ? 'template' : 'lxc',
vmid: lxc.vmid,
sourceId: `${node.node}/lxc/${lxc.vmid}`,
node: node.node,
isProduction: lxc.status === 'running',
interfaces,
macAddress: ifaces.primaryMac(),
ip: ifaces.primaryIp()
ip: ips[0] || null
}
});
edges.push({ parentSlug: nodeSlug, childSlug: lxcSlug, relation: 'hosts' });
@@ -391,8 +190,5 @@ module.exports = {
// `discover` as their implementation name for back-compat, and `run` is just
// an alias. Referenced via module.exports (not `this`) so it survives being
// detached and called as a bare function reference.
run: async (config) => module.exports.discover(config),
// Exported for unit tests only -- not part of the plugin contract.
_Interfaces: Interfaces
run: async (config) => module.exports.discover(config)
};
+20 -65
View File
@@ -1,7 +1,9 @@
#!/bin/sh
#!/bin/bash
set -e
# --- Configuration ---
# In a real environment, these would be derived from the script's download URL
# or passed as additional arguments. For now, we use the most recent release.
BINARY_URL="${BINARY_URL:-}"
CONFIG_DIR="/etc/theta42"
CONFIG_FILE="$CONFIG_DIR/agent.yml"
@@ -13,50 +15,20 @@ RED='\033[0;31m'
GREEN='\033[0;32m'
NC='\033[0m' # No Color
log() { echo "${GREEN}[+]${NC} $1"; }
error() { echo "${RED}[!]${NC} $1"; exit 1; }
log() { echo -e "${GREEN}[+]${NC} $1"; }
error() { echo -e "${RED}[!]${NC} $1"; exit 1; }
# 1. Root check
if [ "$(id -u 2>/dev/null || echo 1)" -ne 0 ]; then
if [ "$EUID" -ne 0 ]; then
error "This script must be run as root."
fi
# Install SSSD and PAM integration packages if missing
install_sssd_deps() {
if ! command -v sssd >/dev/null 2>&1; then
log "Installing SSSD and PAM integration dependencies..."
if command -v apt-get >/dev/null 2>&1; then
DEBIAN_FRONTEND=noninteractive apt-get update -qq || true
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq sssd sssd-ldap libnss-sss libpam-sss libsss-sudo libpam-runtime || \
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq sssd sssd-ldap libnss-sss libpam-sss || true
if command -v pam-auth-update >/dev/null 2>&1; then
pam-auth-update --package --enable mkhomedir sss || pam-auth-update --enable mkhomedir || true
fi
elif command -v dnf >/dev/null 2>&1; then
dnf install -y sssd sssd-ldap sssd-tools || true
elif command -v yum >/dev/null 2>&1; then
yum install -y sssd sssd-ldap sssd-tools || true
elif command -v pacman >/dev/null 2>&1; then
pacman -S --noconfirm sssd || true
elif command -v zypper >/dev/null 2>&1; then
zypper in -y sssd || true
fi
else
log "SSSD is already installed."
fi
mkdir -p /etc/sssd
chmod 755 /etc/sssd
}
# 2. Argument Parsing
URL=""
TOKEN=""
JOIN_KEY=""
PUBLIC_KEY=""
B64_CONFIG=""
INSTALL_SSSD=0
while [ $# -gt 0 ]; do
while [[ $# -gt 0 ]]; do
case $1 in
--url)
URL="$2"
@@ -66,18 +38,6 @@ while [ $# -gt 0 ]; do
TOKEN="$2"
shift 2
;;
--public-key)
PUBLIC_KEY="$2"
shift 2
;;
--join-key)
JOIN_KEY="$2"
shift 2
;;
--install-sssd|--ldap)
INSTALL_SSSD=1
shift
;;
*)
B64_CONFIG="$1"
shift
@@ -85,9 +45,12 @@ while [ $# -gt 0 ]; do
esac
done
# Validation: require credentials ONLY if config file does not already exist
if [ ! -f "$CONFIG_FILE" ] && [ -z "$B64_CONFIG" ] && { [ -z "$URL" ] || { [ -z "$TOKEN" ] && [ -z "$JOIN_KEY" ]; }; }; then
error "Missing required configuration. Provide a base64 encoded config, or --url with either --join-key or --token."
# Validation
if [ -z "$B64_CONFIG" ] && [ -z "$URL" ] || [ -z "$B64_CONFIG" ] && [ -z "$TOKEN" ]; then
error "Missing required configuration. Either provide a base64 encoded config, or both --url and --token."
echo "Usage examples:"
echo " sh install.sh \"BASE64_CONFIG\""
echo " sh install.sh --url \"https://sso.local\" --token \"secret-token\""
exit 1
fi
@@ -108,9 +71,8 @@ if [ -z "$BINARY_URL" ]; then
fi
log "Downloading binary from $BINARY_URL..."
curl -fsSL "$BINARY_URL" -o "$BIN_PATH.tmp" || error "Failed to download binary."
chmod +x "$BIN_PATH.tmp"
mv -f "$BIN_PATH.tmp" "$BIN_PATH"
curl -fsSL "$BINARY_URL" -o "$BIN_PATH" || error "Failed to download binary."
chmod +x "$BIN_PATH"
# 4. Setup configuration
log "Preparing configuration directory $CONFIG_DIR..."
@@ -120,32 +82,23 @@ chmod 755 "$CONFIG_DIR"
if [ -n "$B64_CONFIG" ]; then
log "Decoding and writing configuration from base64..."
echo "$B64_CONFIG" | base64 -d > "$CONFIG_FILE" || error "Failed to decode base64 configuration."
elif [ ! -f "$CONFIG_FILE" ]; then
else
log "Generating minimal configuration from arguments..."
# Create a minimal yaml with the provided URL and Token
cat <<EOF > "$CONFIG_FILE"
server_url: "$URL"
auth_token: "$TOKEN"
join_key: "$JOIN_KEY"
public_key: "$PUBLIC_KEY"
location: "unknown"
capabilities:
telemetry: true
configure_ldap: true
ldap_tunnel: true
configure_ldap: false
reboot: false
service_control: []
arbitrary_bash: false
EOF
else
log "Preserving existing configuration at $CONFIG_FILE"
fi
chmod 600 "$CONFIG_FILE"
# 4b. Ensure SSSD dependencies are installed if configure_ldap is enabled
if [ "$INSTALL_SSSD" -eq 1 ] || grep -qE -i 'configure_ldap:[[:space:]]*true' "$CONFIG_FILE" 2>/dev/null; then
install_sssd_deps
fi
# 5. Setup systemd service
log "Creating systemd service unit..."
cat <<EOF > "$SERVICE_FILE"
@@ -158,6 +111,8 @@ Type=simple
ExecStart=$BIN_PATH
Restart=always
RestartSec=5
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=theta-agent
[Install]
+80 -461
View File
@@ -4,502 +4,121 @@ const express = require('express');
const middleware = require('../middleware/auth');
const permission = require('../utils/permission');
const agentManager = require('../utils/agent_manager');
const agentKeys = require('../utils/agent_keys');
const ldapTunnel = require('../utils/ldap_tunnel');
const { Agent, AgentJoinKey } = require('../models/agent');
const ADMIN_GROUPS = ['app_sso_admin', 'app_super_admin', 'app_sso_directory_admin'];
// Commands that can change or run code on the host. They are signed with the
// SSO's persisted Ed25519 key and the agent verifies against the key pinned in
// its agent.yml.
const HIGH_RISK_COMMANDS = ['reboot', 'service_restart', 'configure_ldap', 'arbitrary_bash', 'update_binary', 'render_secrets', 'iam_apply'];
// ── REST API (mounted synchronously in app.js, BEFORE the 404 catch-all) ──
// This is a plain Express Router exported directly so app.js can
// `app.use('/api/agent', require('./routes/api_agent'))` at require time. It
// must NOT be mounted from the onListen hook (which runs after the 404
// catch-all is already on the stack): a router registered behind that terminal
// handler would make every /api/agent/* request 404, no matter the WS server
// state. The WebSocket handler is separate (initAgentWebSockets below) and is
// the only part that needs the post-listen onListen hook.
const router = express.Router();
// Structured audit line for anything that reaches a host. The agent channel can
// run arbitrary bash, so "who told which host to do what" has to be recoverable
// after the fact; previously nothing was recorded at all.
function logAgentAudit(action, details) {
console.log(JSON.stringify({
timestamp: new Date().toISOString(),
component: 'agent',
action,
...details
}));
}
// The agent WebSocket (/api/agent/ws) authenticates its own token against the
// Agent table (see initAgentWebSockets). These REST routes are admin-facing, so
// they're auth + admin gated.
router.use(middleware.auth);
router.use(async (req, res, next) => {
try {
await permission.byGroup(req.user, ADMIN_GROUPS);
next();
} catch (err) {
if (err && (err.status === 401 || err.name === 'Insufficient Permission')) {
return res.status(403).json({ status: 'error', message: 'admin only' });
}
next(err);
module.exports = function initAgentWebSockets(app) {
if (!app.wss) {
console.warn("WebSocket server for agents is not initialized.");
return;
}
});
// --- Fleet ---
router.get('/nodes', async (req, res, next) => {
try {
const keyStatus = agentKeys.status();
res.json({
status: 'ok',
agents: await agentManager.listAgents(),
// Base64 of the raw 32-byte key: what goes into agent.yml's `public_key`.
publicKey: await agentManager.publicKeyBase64(),
publicKeyPem: await agentManager.publicKeyPem(),
signingAvailable: agentKeys.status().available,
signingError: keyStatus.error || null
});
} catch (err) { next(err); }
});
// --- Enrollment ---
// The token is minted HERE, not in the browser. It is returned exactly once;
// only its hash is stored, so it cannot be recovered afterwards -- rotate to
// get a new one.
router.post('/enroll', async (req, res, next) => {
try {
const { name, resourceId, description } = req.body || {};
if (!name || !String(name).trim()) {
return res.status(400).json({ status: 'error', message: 'name is required' });
}
if (resourceId) {
const { Resource } = require('../models/resource');
const resource = await Resource.get(resourceId);
if (!resource) return res.status(400).json({ status: 'error', message: 'resourceId does not exist' });
if (resource.kind !== 'host') {
return res.status(400).json({ status: 'error', message: 'an agent can only be bound to a host resource' });
}
}
const { agent, token } = await Agent.enroll({
name: String(name).trim(),
description,
resourceId: resourceId || null,
enrolledBy: req.user.uid
});
logAgentAudit('enroll', { actor: req.user.uid, agentId: agent.id, agentName: agent.name, resourceId: resourceId || null });
const publicKey = await agentManager.publicKeyBase64();
res.json({
status: 'ok',
agent: agent.toPublic(agentManager.liveState(agent.id)),
// Shown once. The UI must make that clear.
token,
publicKey,
signingAvailable: agentKeys.status().available
});
} catch (err) { next(err); }
});
router.put('/nodes/:id', async (req, res, next) => {
try {
const agent = await Agent.get(req.params.id);
if (!agent) return res.status(404).json({ status: 'error', message: 'agent not found' });
const patch = {};
if (req.body.name !== undefined) patch.name = req.body.name;
if (req.body.description !== undefined) patch.description = req.body.description;
if (req.body.resourceId !== undefined) {
if (req.body.resourceId) {
const { Resource } = require('../models/resource');
const resource = await Resource.get(req.body.resourceId);
if (!resource) return res.status(400).json({ status: 'error', message: 'resourceId does not exist' });
if (resource.kind !== 'host') {
return res.status(400).json({ status: 'error', message: 'an agent can only be bound to a host resource' });
}
}
patch.resourceId = req.body.resourceId || null;
}
const updated = await agent.update(patch);
logAgentAudit('update', { actor: req.user.uid, agentId: agent.id, fields: Object.keys(patch) });
res.json({ status: 'ok', agent: updated.toPublic(agentManager.liveState(agent.id)) });
} catch (err) { next(err); }
});
// Revoke: the token stops authenticating immediately and any live socket is
// dropped, so revocation takes effect without waiting for a reconnect.
router.post('/nodes/:id/revoke', async (req, res, next) => {
try {
const agent = await Agent.get(req.params.id);
if (!agent) return res.status(404).json({ status: 'error', message: 'agent not found' });
await agent.update({ revoked: true });
agentManager.disconnect(agent.id, 4003, 'Enrollment revoked');
logAgentAudit('revoke', { actor: req.user.uid, agentId: agent.id, agentName: agent.name });
res.json({ status: 'ok' });
} catch (err) { next(err); }
});
router.post('/nodes/:id/rotate', async (req, res, next) => {
try {
const agent = await Agent.get(req.params.id);
if (!agent) return res.status(404).json({ status: 'error', message: 'agent not found' });
const token = await agent.rotateToken();
// The old token is dead the moment it is replaced; drop the socket that was
// using it so the agent reconnects with the new one.
agentManager.disconnect(agent.id, 4004, 'Token rotated');
logAgentAudit('rotate', { actor: req.user.uid, agentId: agent.id, agentName: agent.name });
res.json({ status: 'ok', token, publicKey: await agentManager.publicKeyBase64() });
} catch (err) { next(err); }
});
router.delete('/nodes/:id', async (req, res, next) => {
try {
const agent = await Agent.get(req.params.id);
if (!agent) return res.status(404).json({ status: 'error', message: 'agent not found' });
agentManager.disconnect(agent.id, 4003, 'Enrollment deleted');
await agent.delete();
logAgentAudit('delete', { actor: req.user.uid, agentId: agent.id, agentName: agent.name });
res.json({ status: 'ok' });
} catch (err) { next(err); }
});
// --- Join keys ---
// One key an operator hands out; hosts that present it enroll themselves and
// are immediately issued their own per-agent token. Listing never returns the
// key itself -- only its prefix and usage.
router.get('/join-keys', async (req, res, next) => {
try {
const keys = await AgentJoinKey.list();
res.json({ status: 'ok', joinKeys: keys.map(k => k.toPublic()) });
} catch (err) { next(err); }
});
// Which hosts enrolled through a given key. There is no stored relation --
// join keys are exchanged for a per-agent token immediately, and from then on
// the agent's own identity is what matters -- so this matches on the
// human-readable trace `Agent.enroll` already leaves in `description`
// ("Self-enrolled with join key <prefix>") rather than a foreign key. Prefixes
// are 12 random hex chars, so a collision is not a practical concern.
router.get('/join-keys/:id/agents', async (req, res, next) => {
try {
const key = await AgentJoinKey.get(req.params.id);
if (!key) return res.status(404).json({ status: 'error', message: 'join key not found' });
const marker = `join key ${key.keyPrefix}`;
const agents = await Agent.list();
const matches = agents.filter(a => (a.description || '').includes(marker));
res.json({ status: 'ok', agents: matches.map(a => a.toPublic(agentManager.liveState(a.id))) });
} catch (err) { next(err); }
});
router.post('/join-keys', async (req, res, next) => {
try {
const { label, expiresInDays } = req.body || {};
const { key, raw } = await AgentJoinKey.issue({
label: (label && String(label).trim()) || 'default',
createdBy: req.user.uid,
expiresInDays: expiresInDays ? Number(expiresInDays) : null
});
logAgentAudit('join_key_issued', { actor: req.user.uid, label: key.label, keyPrefix: key.keyPrefix });
// Shown once; only the hash is stored.
res.json({ status: 'ok', joinKey: key.toPublic(), key: raw });
} catch (err) { next(err); }
});
router.post('/join-keys/:id/revoke', async (req, res, next) => {
try {
const key = await AgentJoinKey.get(req.params.id);
if (!key) return res.status(404).json({ status: 'error', message: 'join key not found' });
await key.update({ revoked: true });
logAgentAudit('join_key_revoked', { actor: req.user.uid, label: key.label, keyPrefix: key.keyPrefix });
// Agents already enrolled keep working -- they hold their own tokens now,
// which is the whole point of exchanging the join key rather than using it
// as the long-term credential.
res.json({ status: 'ok' });
} catch (err) { next(err); }
});
router.delete('/join-keys/:id', async (req, res, next) => {
try {
const key = await AgentJoinKey.get(req.params.id);
if (!key) return res.status(404).json({ status: 'error', message: 'join key not found' });
await key.delete();
logAgentAudit('join_key_deleted', { actor: req.user.uid, label: key.label, keyPrefix: key.keyPrefix });
res.json({ status: 'ok' });
} catch (err) { next(err); }
});
// --- Commands ---
// Addressed by agent id, not by token: a token is a credential and has no
// business travelling in a URL, being logged, or sitting in browser history.
router.post('/nodes/:id/command', async (req, res, next) => {
const { command, payload, isHighRisk } = req.body || {};
if (!command) {
return res.status(400).json({ status: 'error', message: 'Command type is required' });
}
try {
const agent = await Agent.get(req.params.id);
if (!agent) return res.status(404).json({ status: 'error', message: 'agent not found' });
if (agent.revoked) return res.status(403).json({ status: 'error', message: 'agent enrollment is revoked' });
const requiresSigning = isHighRisk || HIGH_RISK_COMMANDS.includes(command);
const msg = await agentManager.sendCommand(agent, command, payload || {}, requiresSigning);
logAgentAudit('command', {
actor: req.user.uid,
agentId: agent.id,
agentName: agent.name,
resourceId: agent.resourceId || null,
command,
signed: requiresSigning
});
res.json({ status: 'ok', sentMessage: msg });
} catch (err) {
logAgentAudit('command_failed', { actor: req.user && req.user.uid, agentId: req.params.id, command, error: err.message });
res.status(400).json({ status: 'error', message: err.message });
}
});
module.exports = router;
module.exports.HIGH_RISK_COMMANDS = HIGH_RISK_COMMANDS;
module.exports.initAgentWebSockets = function initAgentWebSockets(app) {
// WebSocket handler only needs the WS server; runs from the onListen hook.
if (!app.wss) return;
// Warm the signing key at boot so a misconfigured OpenBao policy is a loud
// startup error rather than a surprise the first time someone reboots a host.
agentKeys.load().then(keys => {
if (!keys) console.error(`[Theta Agent] signing key unavailable — high-risk commands will be refused. ${agentKeys.status().error || ''}`);
});
app.wss.on('connection', async (ws, req) => {
app.wss.on('connection', (ws, req) => {
const url = new URL(req.url, `http://${req.headers.host || 'localhost'}`);
const token = url.searchParams.get('token') || req.headers['authorization'];
if (!token) {
ws.close(4001, 'Unauthorized: Missing token');
return;
}
const remoteAddr = req.socket.remoteAddress;
console.log(`[Theta Agent] Agent connected from ${remoteAddr} with token ${token.substring(0, 8)}...`);
// Authenticate BEFORE doing anything else: no registration, no welcome
// payload, no acknowledgement that the token was close. Until this passes
// the peer is an anonymous stranger, and the old code treated it as a
// trusted node purely for presenting a non-empty string.
let agent = null;
let issuedToken = null; // set when this connection auto-enrolled
try {
agent = await Agent.authenticate(token);
agentManager.registerAgent(token, ws, remoteAddr);
// Not a known agent token -- try it as a join key. This is what makes
// "install the agent with a key and the host appears" work without an
// admin pre-registering every machine. The join key is exchanged for a
// per-agent token below, so it never becomes the host's long-term
// credential.
if (!agent) {
const joinKey = await AgentJoinKey.authenticate(token);
if (joinKey) {
const hostname = (url.searchParams.get('hostname') || '').trim();
let existingAgent = null;
if (hostname) {
const matches = await Agent.list({ where: { name: hostname } });
existingAgent = matches && matches.find(a => !a.revoked);
}
if (existingAgent) {
const newToken = await existingAgent.rotateToken();
agent = existingAgent;
issuedToken = newToken;
} else {
const enrolled = await Agent.enroll({
name: hostname || `agent-${Date.now().toString(36)}`,
description: `Self-enrolled with join key ${joinKey.keyPrefix}`,
enrolledBy: `join-key:${joinKey.label}`
});
agent = enrolled.agent;
issuedToken = enrolled.token;
}
await joinKey.update({
use_count: (joinKey.use_count || 0) + 1,
last_used_on: Math.floor(Date.now() / 1000)
}).catch(() => {});
logAgentAudit('join', {
agentId: agent.id, agentName: agent.name, remoteAddr,
joinKeyLabel: joinKey.label, joinKeyPrefix: joinKey.keyPrefix
});
console.log(`[Theta Agent] "${agent.name}" self-enrolled with join key ${joinKey.keyPrefix}`);
}
}
} catch (err) {
console.error('[Theta Agent] authentication lookup failed:', err.message);
try { ws.close(1011, 'Authentication unavailable'); } catch (e) {}
return;
}
if (!agent) {
// Deliberately indistinguishable for unknown vs revoked vs missing: a
// caller probing tokens learns nothing about which part was wrong.
logAgentAudit('auth_rejected', { remoteAddr, tokenPrefix: token ? String(token).slice(0, 8) : null });
try { ws.close(4001, 'Unauthorized'); } catch (e) {}
return;
}
console.log(`[Theta Agent] "${agent.name}" (${agent.id}) connected from ${remoteAddr}`);
logAgentAudit('connected', { agentId: agent.id, agentName: agent.name, remoteAddr });
// Must stay synchronous, and the listeners below must be attached in this
// same tick: the agent sends `discovery` the instant the socket opens, and
// `ws` discards messages emitted while no listener is attached.
agentManager.registerAgent(agent, ws, remoteAddr);
if (issuedToken) {
const publicKey = await agentManager.publicKeyBase64();
try {
ws.send(JSON.stringify({
type: 'config',
payload: {
enrolled: true,
auth_token: issuedToken,
public_key: publicKey
}
}));
console.log(`[Theta Agent] Sent auto-enrollment credentials to "${agent.name}"`);
} catch (err) {
console.error(`[Theta Agent] Failed to send auto-enrollment config to "${agent.name}":`, err.message);
}
}
ws.on('message', async (message) => {
ws.on('message', (message) => {
try {
const data = JSON.parse(message);
if (!data || typeof data.type !== 'string') return;
// Re-read the row per message so a revoke mid-session takes effect on
// the next thing the agent says, not only on reconnect.
const current = await Agent.get(agent.id).catch(() => null);
if (!current || current.revoked) {
try { ws.close(4003, 'Enrollment revoked'); } catch (e) {}
return;
}
const payload = data.payload || {};
switch (data.type) {
case 'discovery':
await agentManager.handleDiscovery(current, payload);
if (app.io) app.io.emit('agent.discovery', { agentId: current.id, payload });
if (payload.capabilities && payload.capabilities.configure_ldap) {
const conf = require('@simpleworkjs/conf');
const os = require('os');
const ssoHost = (conf.stack && conf.stack.ssoHost) || 'sso.laptop-dev.vm42.us';
const ldapBaseDn = (conf.stack && conf.stack.ldapBaseDn) || 'dc=laptop-dev,dc=vm42,dc=us';
const lanIps = [];
const ifaces = os.networkInterfaces();
for (const dev in ifaces) {
for (const details of ifaces[dev]) {
if (!details.internal && details.family === 'IPv4') lanIps.push(details.address);
}
}
const uriList = [
`ldapi://%2frun%2ftheta%2fldap.sock`,
`ldap://127.0.0.1:3890`,
`ldap://127.0.0.1:389`,
`ldap://${ssoHost}:389`,
`ldaps://${ssoHost}:636`,
...lanIps.map(ip => `ldap://${ip}:389`)
];
const ldapUris = [...new Set(uriList)].join(', ');
const sssdConfig = `[sssd]
config_file_version = 2
domains = default
[domain/default]
id_provider = ldap
auth_provider = ldap
chpass_provider = ldap
sudo_provider = ldap
ldap_uri = ${ldapUris}
ldap_search_base = ${ldapBaseDn}
ldap_user_search_base = ou=people,${ldapBaseDn}
ldap_group_search_base = ou=groups,${ldapBaseDn}
ldap_sudo_search_base = ou=people,${ldapBaseDn}
ldap_schema = rfc2307bis
ldap_user_object_class = posixAccount
ldap_user_name = uid
ldap_user_ssh_public_key = sshPublicKey
ldap_group_object_class = groupOfNames
ldap_group_member = member
ldap_id_mapping = false
ldap_id_use_start_tls = false
ldap_tls_reqcert = never
cache_credentials = true
entry_cache_timeout = 600
entry_cache_user_timeout = 600
entry_cache_group_timeout = 600
entry_cache_sudo_timeout = 600
refresh_expired_interval = 300
`;
agentManager.sendCommand(current, 'configure_ldap', { config: sssdConfig }, true).then(() => {
console.log(`[Theta Agent] Pushed auto configure_ldap to "${current.name}"`);
}).catch(err => {
console.error(`[Theta Agent] Auto push configure_ldap to "${current.name}" failed:`, err.message);
});
}
agentManager.handleDiscovery(token, payload);
if (app.io) app.io.emit('agent.discovery', { token, payload });
break;
case 'telemetry':
await agentManager.handleTelemetry(current, payload);
if (app.io) app.io.emit('agent.telemetry', { agentId: current.id, payload });
agentManager.handleTelemetry(token, payload);
if (app.io) app.io.emit('agent.telemetry', { token, payload });
break;
case 'heartbeat':
await agentManager.handleHeartbeat(current, payload, ws);
agentManager.handleHeartbeat(token, payload, ws);
break;
case 'response':
await agentManager.handleResponse(current, payload);
if (app.io) app.io.emit('agent.response', { agentId: current.id, payload });
break;
case 'ldap_tunnel':
// Raw LDAP bytes from the agent's local socket → relay into OpenLDAP
// and pipe the response back (DESIGN.md §4).
ldapTunnel.handleTunnel(current.id, ws, payload);
agentManager.handleResponse(token, payload);
if (app.io) app.io.emit('agent.response', { token, payload });
break;
default:
console.log(`[Theta Agent] Received message type '${data.type}' from ${current.id}`);
console.log(`[Theta Agent] Received message type '${data.type}' from ${token}`);
}
} catch (err) {
console.error('[Theta Agent] Error handling message:', err);
console.error("[Theta Agent] Error parsing message:", err);
}
});
ws.on('close', () => {
console.log(`[Theta Agent] "${agent.name}" (${agent.id}) disconnected`);
agentManager.unregisterAgent(agent.id, ws);
ldapTunnel.cleanup(agent.id);
console.log(`[Theta Agent] Agent disconnected (${token})`);
agentManager.unregisterAgent(token, ws);
});
// Send initial welcome/config payload. When this connection enrolled via a
// join key it also carries the credentials the agent should persist and use
// from now on: its own token, and the public key it must pin to verify
// signed commands. Handing the public key over here is what removes the
// last manual step -- an agent installed with only a join key ends up fully
// configured without anyone copying values between two machines.
// Send initial welcome/config payload
try {
const payload = {
message: 'Connected to SSO Manager C2',
protocol_version: '1.2.0',
agent_id: agent.id
};
if (issuedToken) {
payload.enrolled = true;
payload.auth_token = issuedToken;
payload.public_key = await agentManager.publicKeyBase64();
}
ws.send(JSON.stringify({ type: 'config', payload }));
ws.send(JSON.stringify({
type: 'config',
payload: {
message: 'Connected to SSO Manager C2',
protocol_version: '1.1.0'
}
}));
} catch (e) {}
});
// REST API routes for Agent Management (mounted under /api/agent). The agent
// WebSocket (/api/agent/ws) is handled by the raw `wss` upgrade server in
// bin/www with its own ?token= auth — unaffected by the express middleware
// here. These REST routes are admin-facing, so they're auth + admin gated.
const router = express.Router();
router.use(middleware.auth);
router.use(async (req, res, next) => {
try {
await permission.byGroup(req.user, ADMIN_GROUPS);
next();
} catch (err) {
if (err && (err.status === 401 || err.name === 'Insufficient Permission')) {
return res.status(403).json({ status: 'error', message: 'admin only' });
}
next(err);
}
});
router.get('/nodes', (req, res) => {
res.json({
status: 'ok',
agents: agentManager.getConnectedAgents(),
publicKey: agentManager.publicKeyPem
});
});
router.post('/nodes/:token/command', (req, res) => {
const { token } = req.params;
const { command, payload, isHighRisk } = req.body;
if (!command) {
return res.status(400).json({ status: 'error', message: 'Command type is required' });
}
try {
const HIGH_RISK_COMMANDS = ['reboot', 'service_restart', 'configure_ldap', 'arbitrary_bash', 'update_binary'];
const requiresSigning = isHighRisk || HIGH_RISK_COMMANDS.includes(command);
const msg = agentManager.sendCommand(token, command, payload || {}, requiresSigning);
res.json({ status: 'ok', sentMessage: msg });
} catch (err) {
res.status(400).json({ status: 'error', message: err.message });
}
});
app.use('/api/agent', router);
};
-115
View File
@@ -1,115 +0,0 @@
'use strict';
// Agent-facing operations (DESIGN.md §5, §6). These are NOT admin-gated: the
// caller is the agent itself, authenticated by its own token (the same one it
// presents on its WSS channel). Mounted at /api/v1/agent.
const express = require('express');
const baoConf = require('@simpleworkjs/bao-conf');
const { authenticateAgent } = require('../utils/agent_auth');
const router = express.Router();
// POST /secrets — fetch node-scoped OpenBao secrets for the agent's own node.
//
// { paths: ["secret/data/nodes/<agent-id>/db"] }
// -> { status: "ok", secrets: { "secret/data/nodes/<agent-id>/db": { key: value } } }
//
// The agent may only read under its own node prefix (secret/data/nodes/<id>/*),
// so a compromised agent cannot reach other nodes' or shared secrets. The SSO
// fetches with its own OpenBao access (SSO_VAULT_TOKEN); the agent never holds a
// Vault token.
const { Resource } = require('../models/resource');
const { SharedSecretGrant } = require('../models/shared_secret_grant');
const { SharedSecret } = require('../models/shared_secret');
router.post('/secrets', async (req, res, next) => {
try {
const agent = await authenticateAgent(req);
if (!agent) return res.status(401).json({ status: 'error', message: 'unauthorized' });
let { paths } = req.body || {};
let boundResource = null;
if (agent.resourceId) {
boundResource = await Resource.get(agent.resourceId).catch(() => null);
}
if (!Array.isArray(paths) || paths.length === 0) {
paths = [`secret/data/nodes/${agent.id}/conf`];
if (boundResource && boundResource.slug) {
paths.push(`secret/data/resources/${boundResource.slug}/conf`);
}
}
// Allowed prefixes for this agent:
// 1. Node scope: secret/data/nodes/<agent.id>/
// 2. Bound Resource scope: secret/data/resources/<resource.slug>/
// 3. Shared Resource Grants: secret/data/resources/<grantee-slug>/
const allowedPrefixes = [`secret/data/nodes/${agent.id}/`];
if (boundResource && boundResource.slug) {
allowedPrefixes.push(`secret/data/resources/${boundResource.slug}/`);
}
// Add granted shared resources
if (boundResource) {
const grants = await SharedSecretGrant.listForGrantee('resource', boundResource.id).catch(() => []);
for (const g of grants) {
const sharedSec = await SharedSecret.get(g.secretId).catch(() => null);
if (sharedSec && sharedSec.slug) {
allowedPrefixes.push(`secret/data/resources/${sharedSec.slug}/`);
allowedPrefixes.push(`secret/data/shared/${sharedSec.ownerUid}/${sharedSec.slug}/`);
}
}
}
const secrets = {};
for (let p of paths) {
if (typeof p !== 'string') continue;
// Normalize human shorthand "resources/foo/bar" -> "secret/data/resources/foo/bar"
if (p.startsWith('resources/')) {
p = `secret/data/resources/${p.slice('resources/'.length)}`;
}
const isAllowed = allowedPrefixes.some(prefix => p.startsWith(prefix));
if (!isAllowed) {
return res.status(403).json({ status: 'error', message: `path outside authorized scope: ${p}` });
}
const r = await baoConf.request('GET', p);
if (r.ok) {
const body = await r.json().catch(() => ({}));
const rawMap = (body.data && body.data.data) || {};
const resolvedMap = {};
for (const [k, v] of Object.entries(rawMap)) {
const strV = String(v || '');
if (strV.startsWith('INHERIT:')) {
const parts = strV.split(':');
if (parts.length >= 3) {
const targetSlug = parts[1];
const targetKey = parts[2];
const parentR = await baoConf.request('GET', `secret/data/resources/${targetSlug}/conf`);
if (parentR.ok) {
const parentBody = await parentR.json().catch(() => ({}));
const parentMap = (parentBody.data && parentBody.data.data) || {};
resolvedMap[k] = parentMap[targetKey] || '';
} else {
resolvedMap[k] = '';
}
} else {
resolvedMap[k] = '';
}
} else {
resolvedMap[k] = v;
}
}
secrets[p] = resolvedMap;
} else {
secrets[p] = {};
}
}
return res.json({ status: 'ok', secrets });
} catch (err) { next(err); }
});
module.exports = router;
+31 -40
View File
@@ -136,25 +136,15 @@ router.post('/test-email', async (req, res, next) => {
return res.status(400).json({ error: 'Recipient email address is required' });
}
// Send through the SAME sender every other feature uses (password reset,
// invites, OTP-by-email, notifications). A "test" that reimplements
// delivery proves nothing about whether real mail works.
//
// models/email.js exports `{Mail}`; requiring the module and calling
// `.send` on it directly -- as this did -- always threw
// "Email.send is not a function", so the button could never succeed.
const { Mail } = require('../models/email');
// Use the email model to send the test message
const Email = require('../models/email');
const testSubject = subject || 'SSO Manager Test Email';
const testBody = body || `<p>This is a test email from SSO Manager.</p><p>If you received this, your SMTP configuration is working correctly.</p><p>Sent at: ${new Date().toISOString()}</p>`;
await Mail.send(to, testSubject, testBody);
await Email.send(to, testSubject, testBody);
res.json({ success: true, message: `Test email sent to ${to}` });
} catch(err) {
// A failed test is almost always a misconfiguration (wrong host, refused
// connection, bad credentials) -- the operator's to fix, and something the
// UI should be able to show them. Surfacing it as a 400 with the reason
// beats an opaque 500 carrying a raw stack-trace name.
return res.status(400).json({ error: err.message || 'Failed to send test email' });
next(err);
}
});
@@ -166,37 +156,38 @@ router.post('/test-sms', async (req, res, next) => {
return res.status(400).json({ error: 'Recipient phone number is required' });
}
// Send through models/sms.js -- the same path every real SMS takes. It
// prefers a configured messaging plugin and falls back to VoIP.ms, and it
// normalizes the destination to E.164 digits.
//
// This used to POST to `https://api.voip.ms/v1.0/sms/send` with Basic auth.
// No such endpoint exists: VoIP.ms's REST API is a GET against
// `https://voip.ms/api/v1/rest.php` with `api_username`/`api_password` and
// `method=sendSMS`. The fabricated URL returned an HTML page, so
// `response.json()` threw `Unexpected token '<', "<!DOCTYPE "...` and the
// button reported that as the failure. It could never have sent anything.
const { SMS } = require('../models/sms');
const { PluginInstance } = require('../models/plugin_instance');
// A messaging plugin, when present, supplies its own credentials -- so
// requiring conf.voipms unconditionally would block a perfectly working
// setup from testing itself.
const messagingPlugins = await PluginInstance.list({ where: { category: 'messaging', enabled: true } }).catch(() => []);
const voipmsConf = conf.voipms || {};
if (!messagingPlugins.length && (!voipmsConf.username || !voipmsConf.password || !voipmsConf.did)) {
return res.status(400).json({ error: 'No messaging plugin is loaded and VoIP.ms credentials are not configured. Set username, DID and password in the SMS tab, or load a messaging plugin.' });
if (!voipmsConf.username || !voipmsConf.password || !voipmsConf.did) {
return res.status(400).json({ error: 'VoIP.ms credentials not configured. Please configure username, DID, and password in the SMS tab.' });
}
const testMessage = message || `SSO Manager Test SMS: This is a test message from ${conf.name}. If you received this, your SMS configuration is working correctly.`;
const testMessage = message || `SSO Manager Test SMS: This is a test message from ${conf.name}. If you received this, your VoIP.ms configuration is working correctly.`;
await SMS.send(to, testMessage);
res.json({ success: true, message: `Test SMS sent to ${to}` });
// VoIP.ms SMS API endpoint
const voipmsApiUrl = 'https://api.voip.ms/v1.0';
const authHeader = Buffer.from(`${voipmsConf.username}:${voipmsConf.password}`).toString('base64');
const response = await fetch(`${voipmsApiUrl}/sms/send`, {
method: 'POST',
headers: {
'Authorization': `Basic ${authHeader}`,
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
did: voipmsConf.did,
to: to,
message: testMessage
})
});
const result = await response.json();
if (result.status === 'success') {
res.json({ success: true, message: `Test SMS sent to ${to}` });
} else {
res.status(400).json({ error: `VoIP.ms API error: ${result.message || 'Unknown error'}` });
}
} catch(err) {
// The sender rejects with a useful reason (`VoIP.ms error: <status>`, or a
// plugin's own error). Surface it as a 400 the UI can display rather than
// an opaque 500 -- a misconfiguration is the operator's to fix, not a bug.
return res.status(400).json({ error: err.message || 'Failed to send test SMS' });
next(err);
}
});
+29 -178
View File
@@ -61,17 +61,6 @@ async function ensureGroup(name, ownerDn, description) {
}
}
// Link a group to a resource only if that link doesn't already exist. The
// ResourceGroup table has no unique constraint on (resourceId, groupCn), so a
// naive create on every Directory self-heal (which runs ensureSiteGroups /
// provisionResourceGroups on each load) was accumulating duplicate links -- the
// "groups appear 3x under a resource" bug. Always check first.
async function ensureResourceGroup(resourceId, groupCn, accessLevel) {
const existing = await ResourceGroup.list({ where: { resourceId, groupCn } });
if (existing.length) return existing[0];
return ResourceGroup.create({ resourceId, groupCn, accessLevel });
}
// Provision the site-level groups + the aggregates the per-resource groups nest
// into. Idempotent -- called on every directory list so a site seeded by an
// older release gets its groups without a rebuild:
@@ -90,20 +79,19 @@ async function ensureSiteGroups(siteSlug, ownerDn, siteName, siteResourceId) {
// groups as member.
const link = async (cn, isAdmin) => {
if (!siteResourceId) return;
await ensureResourceGroup(siteResourceId, cn, isAdmin ? 'owner' : 'member');
await ResourceGroup.create({ resourceId: siteResourceId, groupCn: cn, accessLevel: isAdmin ? 'owner' : 'member' }).catch(() => {});
};
const sAdmin = groups.siteSuperAdminCns(siteSlug);
await ensureGroup(sAdmin, ownerDn, `Site admin for ${siteName || siteSlug}`);
await link(sAdmin, true);
// The kind-scoped aggregates are CREATED here (per-resource groups nest into
// them), but are NOT linked to the site resource: a site carries only the god
// and site-wide groups (S_super_admin, S_everyone), per the user's model. The
// aggregates have no modal home; site-wide access is granted via S_super_admin
// and per-resource access via the host/app groups.
for (const kind of ['host', 'app']) {
await ensureGroup(groups.aggregateGroupCns(siteSlug, kind, 'admin'), ownerDn, `Admin on all ${kind}s at ${siteSlug}`);
await ensureGroup(groups.aggregateGroupCns(siteSlug, kind, 'access'), ownerDn, `Access to all ${kind}s at ${siteSlug}`);
const aggAdmin = groups.aggregateGroupCns(siteSlug, kind, 'admin');
const aggAccess = groups.aggregateGroupCns(siteSlug, kind, 'access');
await ensureGroup(aggAdmin, ownerDn, `Admin on all ${kind}s at ${siteSlug}`);
await ensureGroup(aggAccess, ownerDn, `Access to all ${kind}s at ${siteSlug}`);
await link(aggAdmin, true);
await link(aggAccess, false);
}
await ensureGroup(groups.siteEveryoneCns(siteSlug), ownerDn, `All users at ${siteSlug}`);
await link(groups.siteEveryoneCns(siteSlug), false);
@@ -126,30 +114,29 @@ async function ensureSiteGroups(siteSlug, ownerDn, siteName, siteResourceId) {
// Provision the per-resource groups for a host/app and nest them into the site
// aggregates (so a site/aggregate admin reaches this resource by membership).
// Group names follow docs/GROUPS.md §2: `{site}_{kind}_{nameSlug}_{level}` where
// nameSlug is the resource name with the kind prefix stripped (`host_theta-env` ->
// `theta-env`). `kind` (host/app) both goes in the name and selects the aggregate:
// The specific group name uses the resource's slug verbatim
// (`{site}_{slug}_{level}` -- the kind is carried in the slug, e.g. `host_theta-env`);
// `kind` (host/app) selects which aggregate the group nests into:
//
// {site}_{kind}_{slug}_admin -> {site}_{kind}_{slug}_access
// {site}_{kind}_{slug}_admin -> {site}_{kind}s_admin (aggregate)
// {site}_{kind}_{slug}_access -> {site}_{kind}s_access (aggregate)
// god_admin -> {site}_{kind}_{slug}_admin (global super admin)
// {site}_{slug}_admin -> {site}_{slug}_access
// {site}_{slug}_admin -> {site}_{kind}s_admin (aggregate)
// {site}_{slug}_access -> {site}_{kind}s_access (aggregate)
// god_admin -> {site}_{slug}_admin (global super admin)
async function provisionResourceGroups(resource, kind, siteSlug, ownerDn) {
const nameSlug = groups.resourceNameSlug(resource.slug);
const accessCn = groups.resourceGroupCns(siteSlug, kind, nameSlug, 'access');
const adminCn = groups.resourceGroupCns(siteSlug, kind, nameSlug, 'admin');
const accessCn = groups.resourceGroupCns(siteSlug, resource.slug, 'access');
const adminCn = groups.resourceGroupCns(siteSlug, resource.slug, 'admin');
await ensureGroup(accessCn, ownerDn, `Access group for ${resource.name}`);
await ensureGroup(adminCn, ownerDn, `Admin group for ${resource.name}`);
// Link both groups to the resource so the Directory can show/revoke them.
await ensureResourceGroup(resource.id, accessCn, 'member');
await ensureResourceGroup(resource.id, adminCn, 'owner');
await ResourceGroup.create({ resourceId: resource.id, groupCn: accessCn, accessLevel: 'member' }).catch(() => {});
await ResourceGroup.create({ resourceId: resource.id, groupCn: adminCn, accessLevel: 'owner' }).catch(() => {});
await nestGroup(adminCn, accessCn); // administering implies using
await nestGroup(adminCn, groups.aggregateGroupCns(siteSlug, kind, 'admin')); // aggregate admin reaches this resource
await nestGroup(accessCn, groups.aggregateGroupCns(siteSlug, kind, 'access')); // aggregate access reaches this resource
await nestGroup(SUPER_ADMIN_GROUP, adminCn); // global super admin
await nestGroup(SUPER_ADMIN_GROUP, adminCn); // legacy cross-app super admin
}
// The group CNs it is valid to associate with a given resource (docs/GROUPS.md
@@ -160,24 +147,25 @@ async function provisionResourceGroups(resource, kind, siteSlug, ownerDn) {
// capability groups following the same shapes.
function validGroupCnsForResource(resource, siteSlug) {
const valid = new Set();
// A site resource only carries god_admin (added by the route) + the site-wide
// groups (S_super_admin, S_everyone). The kind-scoped host/app aggregates and
// specific groups belong to host/app resources, not to the site.
if (resource.kind === 'site') {
valid.add(groups.siteSuperAdminCns(siteSlug));
valid.add(groups.siteEveryoneCns(siteSlug));
return { valid, capRe: new RegExp(`^${siteSlug}_super_admin$|^${siteSlug}_everyone$`) };
for (const k of ['host', 'app']) {
valid.add(groups.aggregateGroupCns(siteSlug, k, 'admin'));
valid.add(groups.aggregateGroupCns(siteSlug, k, 'access'));
}
return { valid, capRe: new RegExp(`^${siteSlug}_(hosts|apps)_[a-z0-9-]+$`) };
}
const kind = groupKind(resource); // 'host'|'app'|null
if (kind) {
const nameSlug = groups.resourceNameSlug(resource.slug);
valid.add(groups.resourceGroupCns(siteSlug, kind, nameSlug, 'admin'));
valid.add(groups.resourceGroupCns(siteSlug, kind, nameSlug, 'access'));
const slug = resource.slug; // verbatim (kind is carried in the slug)
valid.add(groups.resourceGroupCns(siteSlug, slug, 'admin'));
valid.add(groups.resourceGroupCns(siteSlug, slug, 'access'));
valid.add(groups.aggregateGroupCns(siteSlug, kind, 'admin'));
valid.add(groups.aggregateGroupCns(siteSlug, kind, 'access'));
valid.add(groups.siteSuperAdminCns(siteSlug));
valid.add(groups.siteEveryoneCns(siteSlug));
return { valid, capRe: new RegExp(`^${siteSlug}_${kind}_${nameSlug}_[a-z0-9-]+$|^${siteSlug}_${kind}s_[a-z0-9-]+$`) };
return { valid, capRe: new RegExp(`^${siteSlug}_(${slug}_|${kind}s_)[a-z0-9-]+$`) };
}
// oauth/container etc. — only the global god_admin makes sense to pin here.
valid.add(groups.siteSuperAdminCns(siteSlug));
@@ -199,7 +187,6 @@ router.get('/resources', async (req, res, next) => {
try {
let resources = await Resource.list();
resources = resources.filter(r => {
if (r.kind === 'host' || r.kind === 'site') return true;
const isAuto = r.metadata?.discovery_sources?.length > 0 && !r.metadata.discovery_sources.includes('manual');
const isManaged = r.metadata?.managed === true;
return !isAuto || isManaged;
@@ -452,7 +439,7 @@ router.post('/groups', async (req, res, next) => {
}
}
const g = await ensureResourceGroup(req.body.resourceId, groupCn, req.body.accessLevel);
const g = await ResourceGroup.create(req.body);
res.json({ results: g });
} catch (err) { next(err); }
});
@@ -601,140 +588,4 @@ router.get('/audit-logs', async (req, res, next) => {
} catch (err) { next(err); }
});
// ── Resource Secrets API (OpenBao KV-v2 under secret/data/resources/<slug>/conf) ──
const SECRET_KEY_REGEX = /^[A-Za-z0-9_]+$/;
router.get('/resources/:id/secrets', async (req, res, next) => {
try {
const resource = await Resource.get(req.params.id);
if (!resource) return res.status(404).json({ status: 'error', message: 'resource not found' });
const baoConf = require('@simpleworkjs/bao-conf');
// Read resource secrets from OpenBao
const path = `secret/data/resources/${resource.slug}/conf`;
const r = await baoConf.request('GET', path);
let secretsMap = {};
if (r.ok) {
const body = await r.json().catch(() => ({}));
secretsMap = (body.data && body.data.data) || {};
}
// Zero-View Security: Return metadata only, NEVER return raw secret values
const secrets = Object.keys(secretsMap).map(key => {
const val = String(secretsMap[key] || '');
let isInherited = false;
let parentSlug = null;
let parentKey = null;
if (val.startsWith('INHERIT:')) {
isInherited = true;
const parts = val.split(':');
if (parts.length >= 3) {
parentSlug = parts[1];
parentKey = parts[2];
} else if (parts.length === 2) {
parentKey = parts[1];
}
}
return {
key,
hasValue: val.length > 0,
isInherited,
parentSlug,
parentKey
};
});
// Find all ancestor resources across any depth (Host, Site, etc.) + Global Sites
const parentSecrets = [];
const seenAncestors = new Set();
const ancestors = await Resource.findAllAncestors(resource.id).catch(() => []);
const sites = await Resource.list({ where: { kind: 'site' } }).catch(() => []);
const allAncestors = [...ancestors, ...sites];
for (const parent of allAncestors) {
if (!parent || parent.id === resource.id || seenAncestors.has(parent.id)) continue;
seenAncestors.add(parent.id);
const parentPath = `secret/data/resources/${parent.slug}/conf`;
const parentR = await baoConf.request('GET', parentPath);
if (parentR.ok) {
const parentBody = await parentR.json().catch(() => ({}));
const pMap = (parentBody.data && parentBody.data.data) || {};
for (const pKey of Object.keys(pMap)) {
parentSecrets.push({
parentSlug: parent.slug,
parentName: `${parent.name} (${parent.kind ? parent.kind.toUpperCase() : 'PARENT'})`,
key: pKey
});
}
}
}
res.json({ status: 'ok', resourceId: resource.id, slug: resource.slug, secrets, parentSecrets });
} catch (err) { next(err); }
});
router.post('/resources/:id/secrets', async (req, res, next) => {
try {
const resource = await Resource.get(req.params.id);
if (!resource) return res.status(404).json({ status: 'error', message: 'resource not found' });
const secrets = (req.body.secrets && typeof req.body.secrets === 'object') ? req.body.secrets : {};
// Validate key names (Standard Env Var format: A-Z, 0-9, underscores)
for (const key of Object.keys(secrets)) {
if (!SECRET_KEY_REGEX.test(key)) {
return res.status(400).json({
status: 'error',
message: `Invalid secret key '${key}'. Keys must contain only letters, numbers, and underscores (e.g. DB_PASSWORD)`
});
}
}
const baoConf = require('@simpleworkjs/bao-conf');
const path = `secret/data/resources/${resource.slug}/conf`;
const r = await baoConf.request('POST', path, { data: secrets });
if (!r.ok) {
return res.status(500).json({ status: 'error', message: 'failed to save secrets to OpenBao' });
}
res.json({ status: 'ok' });
} catch (err) { next(err); }
});
router.get('/resources/:id/grants', async (req, res, next) => {
try {
const { SharedSecretGrant } = require('../models/shared_secret_grant');
const { SharedSecret } = require('../models/shared_secret');
const resource = await Resource.get(req.params.id);
if (!resource) return res.status(404).json({ status: 'error', message: 'resource not found' });
const grants = await SharedSecretGrant.listForGrantee('resource', resource.id);
const sharedSecretIds = grants.map(g => g.secretId);
const secrets = sharedSecretIds.length ? await SharedSecret.list({ where: { id: { in: sharedSecretIds } } }) : [];
res.json({ status: 'ok', grants: secrets.map(s => ({ id: s.id, slug: s.slug, description: s.description })) });
} catch (err) { next(err); }
});
router.post('/resources/:id/grants', async (req, res, next) => {
try {
const { SharedSecretGrant } = require('../models/shared_secret_grant');
const { SharedSecret } = require('../models/shared_secret');
const resource = await Resource.get(req.params.id);
if (!resource) return res.status(404).json({ status: 'error', message: 'resource not found' });
const { secretSlug, action } = req.body || {};
const secret = await SharedSecret.getBySlug(secretSlug);
if (!secret) return res.status(404).json({ status: 'error', message: `shared secret '${secretSlug}' not found` });
if (action === 'revoke') {
const existing = await SharedSecretGrant.list({ where: { secretId: secret.id, granteeType: 'resource', granteeId: resource.id } });
for (const g of existing) await g.delete();
return res.json({ status: 'ok', message: 'grant revoked' });
} else {
await SharedSecretGrant.grant({ secretId: secret.id, granteeType: 'resource', granteeId: resource.id, grantedBy: req.user.uid });
return res.json({ status: 'ok', message: 'grant created' });
}
} catch (err) { next(err); }
});
module.exports = router;
-105
View File
@@ -1,105 +0,0 @@
'use strict';
// LDAP-over-HTTPS API (DESIGN.md §3).
//
// The whole point of this API is that a client stops speaking LDAP and instead
// does an HTTPS call to the SSO, where the directory is reachable. That kills
// the hostname / cross-network / LDAPS-cert-chain pain: no LDAP protocol, no
// cert to trust, no firewall rule.
//
// POST /api/v1/ldap/bind {username, password} -> 200 {dn, uid} | 401
// POST /api/v1/ldap/search {base_dn, scope, filter, attributes} -> 200 {entries}
//
// Caller auth: a Bearer token in the Authorization header. Two kinds of caller
// are accepted, reusing existing credentials:
// - an agent token (the same one the agent presents on its WSS channel) — the
// caller is a node acting for SSSD;
// - a self-service API token (PAT, `sso_...`) — the caller is a user/app.
// The API authorizes the *caller*; OpenLDAP enforces the actual directory ACLs.
//
// Security note on /search: it runs under the directory admin bind (withClient),
// so it can read the whole tree. It is therefore restricted to agent callers
// (the SSSD user/group-resolution use case) and must eventually move to a
// scoped read-only service account rather than the admin bind. See DESIGN.md §9.
const express = require('express');
const { createLdapClient } = require('@simpleworkjs/ldap');
const conf = require('@simpleworkjs/conf').ldap;
const { Agent } = require('../models/agent');
const { ApiToken } = require('../models/api_token');
const router = express.Router();
const ldap = createLdapClient(conf);
// Resolve a Bearer token to a caller identity, or null. Tries the agent token
// first, then a PAT. Every failure collapses to null so a probing caller learns
// nothing about which credential was wrong.
async function authenticateCaller(req) {
const auth = req.headers['authorization'] || '';
const m = /^Bearer\s+(.+)$/i.exec(auth);
if (!m) return null;
const token = String(m[1]).trim();
if (!token) return null;
try {
const agent = await Agent.authenticate(token);
if (agent) return { kind: 'agent', id: agent.id, name: agent.name };
} catch (_) {}
try {
const pat = await ApiToken.authenticate(token);
if (pat) return { kind: 'user', id: pat.created_by };
} catch (_) {}
return null;
}
// POST /bind — authenticate a username/password against the directory.
router.post('/bind', async (req, res, next) => {
try {
const caller = await authenticateCaller(req);
if (!caller) return res.status(401).json({ status: 'error', message: 'unauthorized' });
const { username, password } = req.body || {};
if (!username || !password) {
return res.status(400).json({ status: 'error', message: 'username and password are required' });
}
// Resolve the username to a DN, then simple-bind as that DN. A missing user
// and a wrong password both surface as 401 (no user-existence oracle).
const user = await ldap.getUser(String(username));
if (!user) return res.status(401).json({ status: 'error', message: 'invalid credentials' });
const ok = await ldap.checkPassword(user.dn, String(password));
if (!ok) return res.status(401).json({ status: 'error', message: 'invalid credentials' });
return res.json({ status: 'ok', dn: user.dn, uid: user.uid });
} catch (err) { next(err); }
});
// POST /search — run a directory search. Agent callers only (see header note).
router.post('/search', async (req, res, next) => {
try {
const caller = await authenticateCaller(req);
if (!caller) return res.status(401).json({ status: 'error', message: 'unauthorized' });
if (caller.kind !== 'agent') {
return res.status(403).json({ status: 'error', message: 'search is restricted to agents' });
}
const { base_dn, scope, filter, attributes } = req.body || {};
if (!filter) return res.status(400).json({ status: 'error', message: 'filter is required' });
const entries = await ldap.withClient(async (client) => {
const { searchEntries } = await client.search(base_dn || conf.userBase, {
scope: scope || 'sub',
filter: String(filter),
attributes: Array.isArray(attributes) && attributes.length ? attributes : undefined,
});
return searchEntries;
});
return res.json({ status: 'ok', entries });
} catch (err) { next(err); }
});
module.exports = router;
+3 -8
View File
@@ -24,10 +24,7 @@ const { SharedSecretGrant } = require('../models/shared_secret_grant');
const vaultBroker = require('../utils/vault_broker');
const ADMIN_GROUPS = ['app_sso_admin', 'app_super_admin', 'app_sso_directory_admin'];
// Allow hyphens AND underscores (matching the plugin-instance slug convention);
// only reject values that can't be a sane secret path segment (spaces, slashes,
// leading non-alnum, too long).
const SLUG_RE = /^[a-z0-9][a-z0-9_-]{0,63}$/;
const SLUG_RE = /^[a-z0-9][a-z0-9-]{0,63}$/;
const router = express.Router();
@@ -80,9 +77,7 @@ router.get('/', async (req, res, next) => {
if (byId.has(g.id)) continue; // already owner
byId.set(g.id, { role: 'grantee', ...g });
}
// The `{ role, ...s }` spread above copies only own properties, so the
// instance method `path()` is dropped -- call the static builder instead.
res.json({ items: [...byId.values()].map(s => ({ id: s.id, slug: s.slug, ownerUid: s.ownerUid, description: s.description, path: SharedSecret.pathFor(s.ownerUid, s.slug), role: s.role })) });
res.json({ items: [...byId.values()].map(s => ({ id: s.id, slug: s.slug, ownerUid: s.ownerUid, description: s.description, path: s.path(), role: s.role })) });
} catch (e) { next(e); }
});
@@ -91,7 +86,7 @@ router.post('/', async (req, res, next) => {
try {
const uid = req.user.uid;
const slug = String(req.body.slug || '').trim().toLowerCase();
if (!SLUG_RE.test(slug)) return res.status(400).json({ error: 'slug must be lowercase letters/digits/hyphens/underscores, 1-64 chars' });
if (!SLUG_RE.test(slug)) return res.status(400).json({ error: 'slug must be lowercase letters/digits/hyphens, 1-64 chars' });
const description = String(req.body.description || '').trim();
const data = (req.body.data && typeof req.body.data === 'object') ? req.body.data : {};
+2 -5
View File
@@ -186,11 +186,8 @@ router.post('/promote/:slug', async (req, res, next) => {
const meta = resource.metadata || {};
meta.managed = true;
// `Resource.update` is not a static — `update` is an instance method
// (@simpleworkjs/orm). Load a fresh instance and call it on that.
const inst = await Resource.get(resource.id);
await inst.update({ metadata: meta });
await Resource.update(resource.id, { metadata: meta });
res.json(envelope({ success: true, groups: [accessGroup, adminGroup] }));
} catch (err) { next(err); }
});
+1 -5
View File
@@ -34,12 +34,8 @@ const DOCS = {
'oauth-apps': {title: 'Connecting Apps (SSO)', file: path.join(__dirname, '../../docs/concepts-oauth-apps.md')},
'api-tokens': {title: 'API Tokens', file: path.join(__dirname, '../../docs/concepts-api-tokens.md')},
directory: {title: 'Directory & Inventory', file: path.join(__dirname, '../../docs/directory.md')},
// `agents` pointed at plugins.md, so docs/agents.md -- the theta-agent
// guide the Directory links to -- was unreachable in the app.
agents: {title: 'Theta Agent', file: path.join(__dirname, '../../docs/agents.md')},
agents: {title: 'Plugins', file: path.join(__dirname, '../../docs/plugins.md')},
plugins: {title: 'Plugins', file: path.join(__dirname, '../../docs/plugins.md')},
// The Discovery tab's help icon links here; without an entry it 404'd.
discovery: {title: 'Discovery & Inventory', file: path.join(__dirname, '../../docs/discovery.md')},
vault: {title: 'Vault Secrets', file: path.join(__dirname, '../../docs/vault.md')},
groups: {title: 'Groups & Permissions', file: path.join(__dirname, '../../docs/groups.md')},
+13 -101
View File
@@ -2,64 +2,26 @@ const { Resource, ResourceEdge, ResourceGroup } = require('../models/resource');
const { WebhookEmitter } = require('./webhook_emitter');
const crypto = require('crypto');
// Is `candidateId` at or below `rootId` in the edge graph? Used to refuse an
// edge that would close a loop. Carries its own visited set so it terminates
// even if the stored graph already contains a cycle from an older release.
function isDescendant(candidateId, rootId, edges) {
const seen = new Set();
const stack = [rootId];
while (stack.length) {
const id = stack.pop();
if (id === candidateId) return true;
if (seen.has(id)) continue;
seen.add(id);
for (const e of edges) if (e.parentId === id) stack.push(e.childId);
}
return false;
}
class DiscoveryReconciler {
static async reconcile(sourceName, payload) {
const { resources = [], edges = [] } = payload;
let newDevices = 0;
const normalizeMac = (m) => (m || '').toLowerCase().replace(/[^a-f0-9]/g, '');
const normalizeHost = (h) => (h || '').toLowerCase().split('.')[0].trim();
// Read the inventory ONCE, not once per incoming resource. A Proxmox
// cluster reports ~55 resources against an inventory of similar size, so
// the per-iteration Resource.list() was doing quadratic full-table reads
// every discovery run. Newly created rows are pushed onto this list as we
// go, so later resources in the same payload still match against them.
const allRes = await Resource.list();
for (const res of resources) {
if (!res.metadata) res.metadata = {};
res._originalSlug = res.slug; // Keep track for edge mapping
let existing = null;
const normalizeMac = (m) => (m || '').toLowerCase().replace(/[^a-f0-9]/g, '');
const normalizeHost = (h) => (h || '').toLowerCase().split('.')[0].trim();
// A discovered device may only merge into a resource of the same kind
// (or into a placeholder from an earlier, kind-less discovery). Without
// this a VM called "gitea-runner" matches a hand-created *service* of
// the same name on rule 3 and silently overwrites it -- the discovered
// host's metadata lands on a service row, and the operator's entry is
// gone. `template` counts as `host`: a VM converted to a template is the
// same device, and it should update in place rather than fork a row.
const kindClass = (k) => (k === 'template' ? 'host' : k);
const incomingKind = kindClass(res.kind || 'unmanaged_device');
const kindCompatible = (r) => {
const k = kindClass(r.kind);
if (k === 'unmanaged_device' || incomingKind === 'unmanaged_device') return true;
return k === incomingKind;
};
const candidates = allRes.filter(kindCompatible);
const allRes = await Resource.list();
// 1. Attempt matching by MAC (highest precision)
if (res.metadata.interfaces && res.metadata.interfaces.length > 0) {
const macs = res.metadata.interfaces.map(i => normalizeMac(i.mac)).filter(m => m.length === 12);
if (macs.length > 0) {
existing = candidates.find(r =>
existing = allRes.find(r =>
r.metadata && (
(r.metadata.macAddress && macs.includes(normalizeMac(r.metadata.macAddress))) ||
(r.metadata.interfaces && r.metadata.interfaces.some(i => macs.includes(normalizeMac(i.mac))))
@@ -80,7 +42,7 @@ class DiscoveryReconciler {
ipsToMatch = [...new Set(ipsToMatch.filter(Boolean))];
if (!existing && ipsToMatch.length > 0) {
existing = candidates.find(r => {
existing = allRes.find(r => {
if (!r.metadata) return false;
if (r.metadata.ip && ipsToMatch.includes(r.metadata.ip)) return true;
if (r.metadata.address) {
@@ -95,7 +57,7 @@ class DiscoveryReconciler {
// 3. Fallback matching by Slug, Name, or Base Hostname
if (!existing && (res.slug || res.name)) {
const inputName = normalizeHost(res.name || res.slug);
existing = candidates.find(r => {
existing = allRes.find(r => {
if (res.slug && r.slug === res.slug) return true;
if (res.name && r.name && r.name.toLowerCase() === res.name.toLowerCase()) return true;
if (inputName && r.name && normalizeHost(r.name) === inputName) return true;
@@ -131,33 +93,10 @@ class DiscoveryReconciler {
mergedMeta.last_seen = Date.now();
// Pick the most human name across sources. Rank first, length only as
// a tie-break within a rank -- comparing lengths alone let a UniFi
// client named after its MAC ("ac:16:2d:b3:da:80", 17 chars) beat the
// hypervisor's real hostname from Proxmox ("dl380-0", 7), so the
// Directory listed MAC addresses where host names belong.
//
// NB: `\\.` inside a regex LITERAL matches a backslash, not a dot, so
// the old isIp returned false for every input and IP-shaped names were
// never replaced either. It is `\.` here.
const isIp = (str) => /^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$/.test(str || '');
const isMac = (str) => /^([0-9a-f]{2}[:-]){5}[0-9a-f]{2}$/i.test((str || '').trim());
// 2 = a real name, 1 = an IP (at least routable/recognizable), 0 = a
// MAC or nothing (pure machine identifier, the worst thing to show).
const nameRank = (str) => {
if (!str || !String(str).trim()) return 0;
if (isMac(str)) return 0;
if (isIp(str)) return 1;
return 2;
};
const isIp = (str) => /^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$/.test(str || '');
let bestName = existing.name;
if (res.name) {
const incoming = nameRank(res.name);
const current = nameRank(bestName);
if (incoming > current || (incoming === current && res.name.length > (bestName || '').length)) {
bestName = res.name;
}
if (res.name && (!bestName || isIp(bestName) || res.name.length > bestName.length && !isIp(res.name))) {
bestName = res.name;
}
await existing.update({
@@ -186,17 +125,12 @@ class DiscoveryReconciler {
newDevices++;
res._actualId = created.id; // Map original slug to actual ID
// Make it visible to the rest of THIS payload: a Proxmox run reports
// the endpoint, then its nodes, then their guests, and two of them can
// legitimately share a MAC/IP. Without this the same device could be
// created twice in a single run.
allRes.push(created);
WebhookEmitter.emit('discovery.new_device', created.toJSON());
}
}
// Now process edges. `allRes` above is already current -- rows created in
// the loop were pushed onto it -- so no second full read is needed.
// Now process edges
const allRes = await Resource.list();
const existingEdges = await ResourceEdge.list();
for (const edge of edges) {
@@ -220,37 +154,15 @@ class DiscoveryReconciler {
if (childResInDb) childId = childResInDb.id;
}
// Two slugs in one payload can resolve to the SAME resource once the
// matcher has merged them -- a Proxmox endpoint reached at the address
// of the node that answers for it is the case that produced this. The
// edge would then make a resource its own parent, which renders as an
// infinitely nested tree and defeats every ancestor walk in the app
// (findAncestorSiteSlug, withResolvedAddress) that relies on a cycle
// guard to terminate rather than to be correct.
if (parentId && childId && parentId === childId) {
console.warn(`[DiscoveryReconciler] ${sourceName}: dropping self-edge on ${edge.parentSlug} -> ${edge.childSlug} (both resolved to the same resource)`);
continue;
}
// Likewise refuse an edge that closes a loop: if the proposed parent is
// already a descendant of the proposed child, adding this makes a cycle.
if (parentId && childId && isDescendant(parentId, childId, existingEdges)) {
console.warn(`[DiscoveryReconciler] ${sourceName}: dropping ${edge.parentSlug} -> ${edge.childSlug} (would create a cycle)`);
continue;
}
if (parentId && childId) {
const edgeExists = existingEdges.find(e => e.parentId === parentId && e.childId === childId && e.relation === edge.relation);
if (!edgeExists) {
const created = await ResourceEdge.create({
await ResourceEdge.create({
id: crypto.randomUUID(),
parentId,
childId,
relation: edge.relation
});
// Keep the in-memory edge list current so the cycle check above sees
// edges added earlier in this same payload.
existingEdges.push(created);
}
}
}
+5 -7
View File
@@ -44,10 +44,9 @@ beforeAll(async () => {
expect(host.status).toBe(200);
hostId = host.body.results.id;
// Creating a host auto-provisions <site>_host_<slug>_access / _admin
// (docs/GROUPS.md §2 — the kind is part of the name).
accessGroupCn = `${siteSlug}_host_${hostSlug}_access`;
const adminGroupCn = `${siteSlug}_host_${hostSlug}_admin`;
// Creating a host auto-provisions <site>_<slug>_access / _admin.
accessGroupCn = `${siteSlug}_${hostSlug}_access`;
const adminGroupCn = `${siteSlug}_${hostSlug}_admin`;
// The creator is seeded into both groups -- groupOfNames requires at least
// one member, so Group.add puts the owner's DN there -- and _admin is nested
@@ -214,9 +213,8 @@ describe('Access requests — withdrawal', () => {
expect(host.status).toBe(200);
// Same as the top-level setup: step out of the auto-created groups the
// creator is seeded into (docs/GROUPS.md §2 — kind is part of the name),
// or this is a request for access already held.
for (const cn of [`${siteSlug}_host_${slug}_admin`, `${siteSlug}_host_${slug}_access`]) {
// creator is seeded into, or this is a request for access already held.
for (const cn of [`${siteSlug}_${slug}_admin`, `${siteSlug}_${slug}_access`]) {
await request(app)
.delete(`/api/group/${encodeURIComponent(cn)}/test`)
.set('auth-token', token);
+44 -150
View File
@@ -1,45 +1,11 @@
'use strict';
const crypto = require('crypto');
// In-memory stand-in for OpenBao. The signing key lives at secret/agent/
// signing-key in production; here we only need it to persist across calls so
// the "same key every time" property is actually exercised rather than mocked
// away.
const mockBaoStore = new Map();
jest.mock('@simpleworkjs/bao-conf', () => ({
get: jest.fn(async (path) => mockBaoStore.get(path) || null),
set: jest.fn(async (path, value) => { mockBaoStore.set(path, value); }),
request: jest.fn(async () => ({ ok: true, status: 200 }))
}));
const agentManager = require('../utils/agent_manager');
const agentKeys = require('../utils/agent_keys');
// The manager is now keyed by enrolled Agent rows rather than by a bare token
// string, so these use a stub row with the same surface the real model gives:
// an id, and an update() that records what would be persisted.
function stubAgent(overrides = {}) {
const row = {
id: overrides.id || crypto.randomUUID(),
name: overrides.name || 'test-agent',
resourceId: overrides.resourceId || null,
revoked: false,
persisted: {},
...overrides
};
row.update = jest.fn(async (patch) => {
Object.assign(row.persisted, patch);
Object.assign(row, patch);
return row;
});
return row;
}
describe('AgentManager PROTOCOL.md v1.2.0 Compliance', () => {
describe('AgentManager PROTOCOL.md v1.1.0 Compliance', () => {
let mockWs;
let sentMessages;
let agent;
beforeEach(() => {
sentMessages = [];
@@ -48,30 +14,23 @@ describe('AgentManager PROTOCOL.md v1.2.0 Compliance', () => {
send: jest.fn((msg) => sentMessages.push(JSON.parse(msg))),
close: jest.fn()
};
agent = stubAgent();
});
test('registers an agent and reports it as connected', () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
const state = agentManager.liveState(agent.id);
expect(state.connected).toBe(true);
expect(state.ipAddress).toBe('192.168.1.100');
expect(agentManager.isConnected(agent.id)).toBe(true);
test('registers agent and tracks initial connection state', () => {
const record = agentManager.registerAgent('test-token-123', mockWs, '192.168.1.100');
expect(record.token).toBe('test-token-123');
expect(record.ipAddress).toBe('192.168.1.100');
const agents = agentManager.getConnectedAgents();
const found = agents.find(a => a.token === 'test-token-123');
expect(found).toBeDefined();
expect(found.isOnline).toBe(true);
});
// registerAgent must not be async: the WS `message` listener is attached in
// the same tick, and `ws` drops events emitted before a listener exists. An
// awaited DB write here swallowed every agent's first discovery frame, which
// is the one it sends immediately on connect.
test('registerAgent is synchronous so no message can be missed', () => {
const result = agentManager.registerAgent(agent, mockWs, '10.0.0.1');
expect(result).toBeUndefined();
expect(agentManager.isConnected(agent.id)).toBe(true);
});
test('processes discovery payload per PROTOCOL.md v1.1.0 Section 3.1', () => {
agentManager.registerAgent('test-token-123', mockWs, '192.168.1.100');
test('persists discovery to the agent row (Section 3.1)', async () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
await agentManager.handleDiscovery(agent, {
const discoveryPayload = {
hostname: 'node-01.local',
ip_addresses: ['192.168.1.100', '10.0.0.5'],
os: 'Ubuntu 24.04 LTS',
@@ -80,34 +39,41 @@ describe('AgentManager PROTOCOL.md v1.2.0 Compliance', () => {
ram_total_gb: 32.0,
disk_total_gb: 500.0,
location: 'dc-chicago-rack-4'
});
};
const saved = agent.persisted.lastDiscovery;
expect(saved.hostname).toBe('node-01.local');
expect(saved.os).toBe('Ubuntu 24.04 LTS');
expect(saved.ip_addresses).toEqual(['192.168.1.100', '10.0.0.5']);
// Durable, not just in memory: an agent that goes offline keeps its facts.
expect(agent.persisted.last_seen).toEqual(expect.any(Number));
agentManager.handleDiscovery('test-token-123', discoveryPayload);
const agents = agentManager.getConnectedAgents();
const agent = agents.find(a => a.token === 'test-token-123');
expect(agent.hostname).toBe('node-01.local');
expect(agent.discovery.os).toBe('Ubuntu 24.04 LTS');
expect(agent.discovery.ip_addresses).toEqual(['192.168.1.100', '10.0.0.5']);
});
test('persists telemetry to the agent row (Section 3.2)', async () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
await agentManager.handleTelemetry(agent, {
test('processes telemetry payload per PROTOCOL.md v1.1.0 Section 3.2', () => {
agentManager.registerAgent('test-token-123', mockWs, '192.168.1.100');
const telemetryPayload = {
cpu_usage_percent: 14.5,
ram_usage_percent: 42.1,
disk_usage_percent: 68.0,
zfs_health: 'ONLINE',
gpu_usage_percent: -1.0,
timestamp: new Date().toISOString()
});
};
expect(agent.persisted.lastTelemetry.cpu_usage_percent).toBe(14.5);
expect(agent.persisted.lastTelemetry.zfs_health).toBe('ONLINE');
agentManager.handleTelemetry('test-token-123', telemetryPayload);
const agents = agentManager.getConnectedAgents();
const agent = agents.find(a => a.token === 'test-token-123');
expect(agent.telemetry.cpu_usage_percent).toBe(14.5);
expect(agent.telemetry.zfs_health).toBe('ONLINE');
});
test('responds to heartbeat with heartbeat_ack (Section 3.3)', async () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
await agentManager.handleHeartbeat(agent, { timestamp: new Date().toISOString() }, mockWs);
test('responds to heartbeat with heartbeat_ack per Section 3.3', () => {
agentManager.registerAgent('test-token-123', mockWs, '192.168.1.100');
agentManager.handleHeartbeat('test-token-123', { timestamp: new Date().toISOString() }, mockWs);
expect(mockWs.send).toHaveBeenCalled();
const lastMsg = sentMessages[sentMessages.length - 1];
@@ -115,92 +81,20 @@ describe('AgentManager PROTOCOL.md v1.2.0 Compliance', () => {
expect(lastMsg.payload.timestamp).toBeDefined();
});
test('canonicalizes and signs high-risk commands with Ed25519 (Section 5)', async () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
test('canonicalizes payload and signs high-risk commands using Ed25519 per Section 5', () => {
agentManager.registerAgent('test-token-123', mockWs, '192.168.1.100');
const rawPayload = { script: 'uptime', location: 'datacenter' };
const msg = await agentManager.sendCommand(agent, 'arbitrary_bash', rawPayload, true);
const msg = agentManager.sendCommand('test-token-123', 'arbitrary_bash', rawPayload, true);
expect(msg.type).toBe('arbitrary_bash');
expect(msg.payload.signature).toBeDefined();
expect(typeof msg.payload.signature).toBe('string');
const keys = await agentKeys.load();
const isValid = crypto.verify(
null,
Buffer.from(agentManager.canonicalize(rawPayload), 'utf8'),
crypto.createPublicKey(keys.publicKeyPem),
Buffer.from(msg.payload.signature, 'base64')
);
// Verify signature with public key
const signatureBuffer = Buffer.from(msg.payload.signature, 'base64');
const canonicalStr = agentManager.canonicalize(rawPayload);
const isValid = crypto.verify(null, Buffer.from(canonicalStr, 'utf8'), agentManager.publicKeyPem, signatureBuffer);
expect(isValid).toBe(true);
});
// The canonical form has to match the Go agent's byte for byte. Go's
// encoding/json escapes <, > and & by default and JSON.stringify does not, so
// the agent uses SetEscapeHTML(false); this pins the server's half of that
// contract. See theta-agent TestCanonicalizeMatchesServerForm.
test('canonical form is sorted, unescaped, and omits the signature', () => {
const canonical = agentManager.canonicalize({
script: 'echo a > b && c',
comment: 'x&y',
signature: 'should-not-appear'
});
expect(canonical).toBe('{"comment":"x&y","script":"echo a > b && c"}');
});
test('refuses to send to an agent that is not connected', async () => {
await expect(agentManager.sendCommand(agent, 'reload_config', {}, false))
.rejects.toThrow(/not connected/);
});
// Revocation that only applies on the next reconnect is not revocation.
test('disconnect drops the live socket immediately', () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
expect(agentManager.isConnected(agent.id)).toBe(true);
const dropped = agentManager.disconnect(agent.id, 4003, 'Enrollment revoked');
expect(dropped).toBe(true);
expect(mockWs.close).toHaveBeenCalledWith(4003, 'Enrollment revoked');
expect(agentManager.isConnected(agent.id)).toBe(false);
});
test('a second connection for the same agent supersedes the first', () => {
agentManager.registerAgent(agent, mockWs, '192.168.1.100');
const secondWs = { readyState: 1, send: jest.fn(), close: jest.fn() };
agentManager.registerAgent(agent, secondWs, '192.168.1.101');
expect(mockWs.close).toHaveBeenCalledWith(4002, 'Superseded by new connection');
expect(agentManager.liveState(agent.id).ipAddress).toBe('192.168.1.101');
});
test('an unknown agent id is simply not connected', () => {
expect(agentManager.isConnected('no-such-agent')).toBe(false);
expect(agentManager.liveState('no-such-agent')).toEqual({ connected: false, lastResponse: null });
});
});
describe('agent signing key', () => {
// The old manager generated a key pair in its constructor, so it changed on
// every restart and the public_key pinned in agent.yml stopped matching.
test('the same key is returned across repeated loads', async () => {
const first = await agentKeys.load();
const second = await agentKeys.load();
expect(first.publicKeyBase64).toBe(second.publicKeyBase64);
});
test('the exported public key is the raw 32 bytes agents pin', async () => {
const keys = await agentKeys.load();
expect(Buffer.from(keys.publicKeyBase64, 'base64')).toHaveLength(32);
});
test('rawPublicKeyBase64 strips the SPKI wrapper', () => {
const { publicKey } = crypto.generateKeyPairSync('ed25519', {
privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
publicKeyEncoding: { type: 'spki', format: 'pem' }
});
const raw = Buffer.from(agentKeys.rawPublicKeyBase64(publicKey), 'base64');
expect(raw).toHaveLength(32);
// and it is the tail of the DER encoding
const der = crypto.createPublicKey(publicKey).export({ type: 'spki', format: 'der' });
expect(raw.equals(der.subarray(der.length - 32))).toBe(true);
});
});
-72
View File
@@ -1,72 +0,0 @@
'use strict';
// Agent-facing ops (DESIGN.md §5): node-scoped secrets. OpenBao is not present
// in the test env, so @simpleworkjs/bao-conf is mocked.
jest.mock('@simpleworkjs/bao-conf', () => ({
request: jest.fn(async (method, path) => {
if (path.startsWith('secret/data/nodes/')) {
return {
ok: true,
status: 200,
json: async () => ({ data: { data: { username: 'alice', password: 's3cret' } } }),
};
}
return { ok: false, status: 404, json: async () => ({}) };
}),
}));
const { request, app } = require('./setup');
const { Agent } = require('../models/agent');
async function enrollAgent() {
const { agent, token } = await Agent.enroll({
name: `ops-test-${Date.now().toString(36)}`,
description: 'api_agent_ops test',
enrolledBy: 'test'
});
return { agent, token };
}
describe('Agent ops — POST /api/v1/agent/secrets', () => {
test('an agent can fetch its own node-scoped secrets', async () => {
const { agent, token } = await enrollAgent();
const path = `secret/data/nodes/${agent.id}/db`;
const res = await request(app)
.post('/api/v1/agent/secrets')
.set('Authorization', `Bearer ${token}`)
.send({ paths: [path] });
expect(res.status).toBe(200);
expect(res.body.status).toBe('ok');
expect(res.body.secrets[path]).toEqual({ username: 'alice', password: 's3cret' });
});
test('a path outside the node scope is rejected', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/agent/secrets')
.set('Authorization', `Bearer ${token}`)
.send({ paths: ['secret/data/nodes/other-node/db'] });
expect(res.status).toBe(403);
});
test('no bearer token returns 401', async () => {
const res = await request(app)
.post('/api/v1/agent/secrets')
.send({ paths: ['secret/data/nodes/x/db'] });
expect(res.status).toBe(401);
});
test('missing paths returns 400', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/agent/secrets')
.set('Authorization', `Bearer ${token}`)
.send({});
expect(res.status).toBe(400);
});
});
-126
View File
@@ -1,126 +0,0 @@
'use strict';
// LDAP-over-HTTPS API (DESIGN.md §3). Exercises caller auth (agent token vs
// PAT), the bind flow against the real test OpenLDAP, and the agent-only search
// restriction.
const { TEST_CREDS, request, app } = require('./setup');
const { Agent } = require('../models/agent');
const { ApiToken } = require('../models/api_token');
async function enrollAgent() {
const { agent, token } = await Agent.enroll({
name: `ldap-test-${Date.now().toString(36)}`,
description: 'api_ldap test agent',
enrolledBy: 'test'
});
return { agent, token };
}
async function makePat() {
const token = await ApiToken.add({
name: 'ldap-test-pat',
description: 'api_ldap test',
created_by: 'test'
});
return token._raw_token;
}
describe('LDAP-over-HTTPS — POST /api/v1/ldap/bind', () => {
test('valid credentials return the bound DN', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/bind')
.set('Authorization', `Bearer ${token}`)
.send({ username: TEST_CREDS.uid, password: TEST_CREDS.password });
expect(res.status).toBe(200);
expect(res.body.status).toBe('ok');
expect(res.body.uid).toBe(TEST_CREDS.uid);
expect(res.body.dn).toContain(TEST_CREDS.uid);
});
test('wrong password returns 401', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/bind')
.set('Authorization', `Bearer ${token}`)
.send({ username: TEST_CREDS.uid, password: 'wrong-password' });
expect(res.status).toBe(401);
});
test('unknown user returns 401 (no existence oracle)', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/bind')
.set('Authorization', `Bearer ${token}`)
.send({ username: 'no_such_user_xyz', password: 'whatever' });
expect(res.status).toBe(401);
});
test('a PAT caller can bind', async () => {
const pat = await makePat();
const res = await request(app)
.post('/api/v1/ldap/bind')
.set('Authorization', `Bearer ${pat}`)
.send({ username: TEST_CREDS.uid, password: TEST_CREDS.password });
expect(res.status).toBe(200);
});
test('no bearer token returns 401', async () => {
const res = await request(app)
.post('/api/v1/ldap/bind')
.send({ username: TEST_CREDS.uid, password: TEST_CREDS.password });
expect(res.status).toBe(401);
});
test('missing username/password returns 400', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/bind')
.set('Authorization', `Bearer ${token}`)
.send({ username: TEST_CREDS.uid });
expect(res.status).toBe(400);
});
});
describe('LDAP-over-HTTPS — POST /api/v1/ldap/search', () => {
test('an agent can search the user tree', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/search')
.set('Authorization', `Bearer ${token}`)
.send({ filter: `(uid=${TEST_CREDS.uid})`, attributes: ['uid', 'cn'] });
expect(res.status).toBe(200);
expect(res.body.status).toBe('ok');
expect(Array.isArray(res.body.entries)).toBe(true);
expect(res.body.entries.length).toBeGreaterThan(0);
expect(res.body.entries[0].uid).toBe(TEST_CREDS.uid);
});
test('a PAT caller is denied search (agent-only)', async () => {
const pat = await makePat();
const res = await request(app)
.post('/api/v1/ldap/search')
.set('Authorization', `Bearer ${pat}`)
.send({ filter: `(uid=${TEST_CREDS.uid})` });
expect(res.status).toBe(403);
});
test('missing filter returns 400', async () => {
const { token } = await enrollAgent();
const res = await request(app)
.post('/api/v1/ldap/search')
.set('Authorization', `Bearer ${token}`)
.send({});
expect(res.status).toBe(400);
});
});
-104
View File
@@ -1,104 +0,0 @@
'use strict';
// Pure-logic coverage for the two reconciler rules that real Proxmox + UniFi
// data broke. Both were found by running discovery against a live cluster:
// the directory came back listing MAC addresses as host names, and one
// resource ended up as its own parent.
// Mirrors the ranking in services/discovery_reconciler.js. Kept here (rather
// than exported) because it is a few lines of predicate that the reconciler
// applies inline while merging; if it grows, export it and drop this copy.
const isIp = (str) => /^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$/.test(str || '');
const isMac = (str) => /^([0-9a-f]{2}[:-]){5}[0-9a-f]{2}$/i.test((str || '').trim());
const nameRank = (str) => {
if (!str || !String(str).trim()) return 0;
if (isMac(str)) return 0;
if (isIp(str)) return 1;
return 2;
};
function bestNameOf(existingName, incomingName) {
let best = existingName;
if (incomingName) {
const a = nameRank(incomingName);
const b = nameRank(best);
if (a > b || (a === b && incomingName.length > (best || '').length)) best = incomingName;
}
return best;
}
describe('discovery name ranking', () => {
test('a real hostname beats a MAC even when shorter', () => {
// The exact regression: UniFi named the host by MAC, Proxmox knew the
// hostname, and length-only comparison kept the MAC.
expect(bestNameOf('ac:16:2d:b3:da:80', 'dl380-0')).toBe('dl380-0');
});
test('a MAC never displaces a real hostname', () => {
expect(bestNameOf('dl380-0', 'ac:16:2d:b3:da:80')).toBe('dl380-0');
});
test('a real hostname beats an IP-shaped name', () => {
expect(bestNameOf('192.168.1.27', 'hass.io')).toBe('hass.io');
});
test('an IP beats a MAC', () => {
expect(bestNameOf('bc:24:11:3f:cd:c8', '192.168.1.27')).toBe('192.168.1.27');
});
test('an IP does not displace a hostname', () => {
expect(bestNameOf('gitea-runner', '192.168.1.176')).toBe('gitea-runner');
});
test('within the same rank the longer/more specific name wins', () => {
expect(bestNameOf('pve', 'pve-dl380-1')).toBe('pve-dl380-1');
});
test('dash-separated MACs are recognized too', () => {
expect(bestNameOf('ac-16-2d-b3-da-80', 'dl380-0')).toBe('dl380-0');
});
test('an empty existing name is always replaced', () => {
expect(bestNameOf('', 'anything')).toBe('anything');
expect(bestNameOf(null, 'ac:16:2d:b3:da:80')).toBe('ac:16:2d:b3:da:80');
});
});
// Mirrors isDescendant() in the reconciler.
function isDescendant(candidateId, rootId, edges) {
const seen = new Set();
const stack = [rootId];
while (stack.length) {
const id = stack.pop();
if (id === candidateId) return true;
if (seen.has(id)) continue;
seen.add(id);
for (const e of edges) if (e.parentId === id) stack.push(e.childId);
}
return false;
}
describe('discovery edge cycle guard', () => {
const edges = [
{ parentId: 'cluster', childId: 'node1' },
{ parentId: 'node1', childId: 'vm1' },
];
test('detects a direct parent/child inversion', () => {
// Proposing node1 -> cluster when cluster -> node1 already exists.
expect(isDescendant('node1', 'cluster', edges)).toBe(true);
});
test('detects a deeper loop', () => {
expect(isDescendant('vm1', 'cluster', edges)).toBe(true);
});
test('allows an unrelated new parent', () => {
expect(isDescendant('node2', 'cluster', edges)).toBe(false);
});
test('terminates on a graph that already contains a cycle', () => {
// A self-edge written by an earlier release must not hang the walk.
const cyclic = [{ parentId: 'a', childId: 'a' }, { parentId: 'a', childId: 'b' }];
expect(isDescendant('zzz', 'a', cyclic)).toBe(false);
});
});
+20 -29
View File
@@ -12,13 +12,11 @@ const {
GOD_ADMIN,
} = require('../utils/groups');
// Resource fixtures mirror the directory: hosts carry a `host_` prefix, services
// are stored bare. The builders take the *name* slug (kind stripped) + a kind, so
// a host `host_web-01` gives `main-office_host_web-01_*` and a service `emby`
// gives `main-office_app_emby_*` -- matching docs/GROUPS.md §2.
// Resource fixtures mirror the directory's real slugs: hosts carry a `host_`
// prefix, services/apps are stored bare. The group-model builders use these
// verbatim (no re-slugifying, no kind insertion) -- see groups.js.
const HOST = { site: 'main-office', kind: 'host', slug: 'host_web-01' };
const APP = { site: 'main-office', kind: 'app', slug: 'emby' };
const SERVICE = { site: 'main-office', kind: 'service', slug: 'emby' };
const OTHER_SITE_HOST = { site: 'branch-office', kind: 'host', slug: 'host_db' };
describe('slugify', () => {
@@ -32,13 +30,9 @@ describe('slugify', () => {
});
describe('group cn builders', () => {
test('per-resource names the kind + name slug (docs §2)', () => {
expect(resourceGroupCns('main-office', 'host', 'web-01', 'admin')).toBe('main-office_host_web-01_admin');
expect(resourceGroupCns('main-office', 'app', 'emby', 'access')).toBe('main-office_app_emby_access');
});
test('a prefixed site slug is kept verbatim; the resource name slug is kind-stripped', () => {
expect(resourceGroupCns('site_local', 'host', 'theta-env', 'access')).toBe('site_local_host_theta-env_access');
expect(resourceGroupCns('site_local', 'app', 'sso-manager', 'access')).toBe('site_local_app_sso-manager_access');
test('per-resource uses the resource slug verbatim (kind is carried in the slug)', () => {
expect(resourceGroupCns('main-office', 'host_web-01', 'admin')).toBe('main-office_host_web-01_admin');
expect(resourceGroupCns('main-office', 'emby', 'access')).toBe('main-office_emby_access');
});
test('aggregate uses the plural kind', () => {
expect(aggregateGroupCns('main-office', 'host', 'admin')).toBe('main-office_hosts_admin');
@@ -48,13 +42,15 @@ describe('group cn builders', () => {
expect(siteSuperAdminCns('main-office')).toBe('main-office_super_admin');
expect(siteEveryoneCns('main-office')).toBe('main-office_everyone');
});
test('a directory site slug with a kind prefix is kept verbatim', () => {
test('a directory site slug with a kind prefix is kept verbatim, not re-slugified', () => {
// Resource slugs are `site_local` / `host_theta-env` -- re-slugifying the
// site (`site_local` -> `site-local`) would corrupt the delimiter.
expect(siteSuperAdminCns('site_local')).toBe('site_local_super_admin');
expect(siteEveryoneCns('site_local')).toBe('site_local_everyone');
expect(aggregateGroupCns('site_local', 'host', 'admin')).toBe('site_local_hosts_admin');
expect(resourceGroupCns('site_local', 'host_theta-env', 'access')).toBe('site_local_host_theta-env_access');
});
test('invalid kind throws', () => {
expect(() => resourceGroupCns('s', 'service', 'x', 'admin')).toThrow();
test('invalid kind throws (aggregates only — per-resource has no kind arg)', () => {
expect(() => aggregateGroupCns('s', 'service', 'admin')).toThrow();
});
});
@@ -93,37 +89,32 @@ describe('hasPermission — inheritance', () => {
});
test('specific host group grants only that host', () => {
const cn = resourceGroupCns('main-office', 'host', 'web-01', 'admin');
const cn = resourceGroupCns('main-office', 'host_web-01', 'admin');
expect(hasPermission([cn], HOST, 'admin')).toBe(true);
expect(hasPermission([cn], OTHER_SITE_HOST, 'admin')).toBe(false);
});
test('admin implies access; access does not imply admin', () => {
expect(hasPermission([resourceGroupCns('main-office', 'host', 'web-01', 'admin')], HOST, 'access')).toBe(true);
expect(hasPermission([resourceGroupCns('main-office', 'host', 'web-01', 'access')], HOST, 'admin')).toBe(false);
expect(hasPermission([resourceGroupCns('main-office', 'host_web-01', 'admin')], HOST, 'access')).toBe(true);
expect(hasPermission([resourceGroupCns('main-office', 'host_web-01', 'access')], HOST, 'admin')).toBe(false);
});
test('capabilities are exact — admin does not grant a capability', () => {
expect(hasPermission([resourceGroupCns('main-office', 'host', 'web-01', 'reboot')], HOST, 'reboot')).toBe(true);
expect(hasPermission([resourceGroupCns('main-office', 'host', 'web-01', 'admin')], HOST, 'reboot')).toBe(false);
expect(hasPermission([resourceGroupCns('main-office', 'host_web-01', 'reboot')], HOST, 'reboot')).toBe(true);
expect(hasPermission([resourceGroupCns('main-office', 'host_web-01', 'admin')], HOST, 'reboot')).toBe(false);
// aggregate capability
expect(hasPermission(['main-office_hosts_reboot'], HOST, 'reboot')).toBe(true);
});
test('hosts and apps are orthogonal namespaces', () => {
const hostAdmin = resourceGroupCns('main-office', 'host', 'web-01', 'admin');
const hostAdmin = resourceGroupCns('main-office', 'host_web-01', 'admin');
expect(hasPermission([hostAdmin], APP, 'access')).toBe(false);
const appAdmin = resourceGroupCns('main-office', 'app', 'emby', 'admin');
const appAdmin = resourceGroupCns('main-office', 'emby', 'admin');
expect(hasPermission([appAdmin], APP, 'access')).toBe(true);
});
test('a service maps to the app kind (docs §11)', () => {
// The directory `service` kind is the group model's `app`.
expect(hasPermission([resourceGroupCns('main-office', 'app', 'emby', 'admin')], SERVICE, 'admin')).toBe(true);
expect(hasPermission([resourceGroupCns('main-office', 'host', 'web-01', 'admin')], SERVICE, 'admin')).toBe(false);
});
test('cross-site isolation', () => {
const mainHostAdmin = resourceGroupCns('main-office', 'host', 'web-01', 'admin');
const mainHostAdmin = resourceGroupCns('main-office', 'host_web-01', 'admin');
expect(hasPermission([mainHostAdmin], OTHER_SITE_HOST, 'access')).toBe(false);
expect(hasPermission(['branch-office_hosts_admin'], OTHER_SITE_HOST, 'admin')).toBe(true);
});
-118
View File
@@ -1,118 +0,0 @@
'use strict';
const fs = require('fs');
const path = require('path');
// @simpleworkjs/orm models expose `list`/`get`/`count`/`create` -- there is no
// `find`, `findOne`, `findAll` or `where`. Calling one is not a syntax error and
// nothing catches it until the line actually runs, so it can sit in a rarely
// exercised path indefinitely.
//
// It did: `models/sms.js` called `PluginInstance.find({...})`, which threw
// "is not a function" on EVERY SMS send -- the test button, OTP-by-SMS and
// notifications alike -- before it could even reach the VoIP.ms fallback. SMS
// delivery had simply never worked.
const ORM_MODELS = [
'Resource', 'ResourceEdge', 'ResourceGroup', 'AccessRequest', 'Webhook',
'PluginInstance', 'SharedSecret', 'SharedSecretGrant', 'VaultAppToken',
'Agent', 'AgentJoinKey',
];
const MISSING_STATICS = ['find', 'findOne', 'findAll', 'findAndCountAll', 'where'];
const ROOT = path.join(__dirname, '..');
const SCAN_DIRS = ['models', 'routes', 'services', 'utils', 'plugins', 'controller', 'middleware'];
function walk(dir, out = []) {
let entries;
try { entries = fs.readdirSync(dir, { withFileTypes: true }); } catch (e) { return out; }
for (const entry of entries) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name === 'node_modules') continue;
walk(full, out);
} else if (entry.name.endsWith('.js')) {
out.push(full);
}
}
return out;
}
// Strip comments so a line *describing* the bug (like the one in models/sms.js)
// isn't reported as the bug.
function stripComments(src) {
return src
.replace(/\/\*[\s\S]*?\*\//g, '')
.replace(/(^|[^:])\/\/.*$/gm, '$1');
}
test('no source file calls an ORM static that does not exist', () => {
const pattern = new RegExp(
`\\b(${ORM_MODELS.join('|')})\\s*\\.\\s*(${MISSING_STATICS.join('|')})\\s*\\(`,
'g'
);
const offenders = [];
for (const dir of SCAN_DIRS) {
for (const file of walk(path.join(ROOT, dir))) {
const src = stripComments(fs.readFileSync(file, 'utf8'));
src.split('\n').forEach((line, i) => {
const m = line.match(pattern);
if (m) offenders.push(`${path.relative(ROOT, file)}:${i + 1}${m.join(', ')}`);
});
}
}
expect(offenders).toEqual([]);
});
// models/email.js exports `{Mail}`, not a bare sender. Requiring the module and
// calling `.send` on it -- as routes/api_conf.js's test-email did -- always
// threw "Email.send is not a function", so the Test Email button could never
// have worked.
test('the email module exports Mail.send and callers destructure it', () => {
const mod = require('../models/email');
expect(typeof mod.Mail).toBe('object');
expect(typeof mod.Mail.send).toBe('function');
// The bare module has no send() -- this is exactly the mistake to catch.
expect(mod.send).toBeUndefined();
const offenders = [];
for (const dir of SCAN_DIRS) {
for (const file of walk(path.join(ROOT, dir))) {
const src = stripComments(fs.readFileSync(file, 'utf8'));
// `X = require('...email')` followed by `X.send(` where X was not
// destructured.
const assigned = [...src.matchAll(/(?:const|let|var)\s+(\w+)\s*=\s*require\([^)]*models\/email[^)]*\)/g)]
.map(m => m[1]);
for (const name of assigned) {
if (new RegExp(`\\b${name}\\s*\\.\\s*send\\s*\\(`).test(src)) {
offenders.push(`${path.relative(ROOT, file)}${name}.send(), but the module exports {Mail}`);
}
}
}
}
expect(offenders).toEqual([]);
});
// The VoIP.ms REST API is a GET against voip.ms/api/v1/rest.php with
// api_username/api_password and method=sendSMS. `api.voip.ms/v1.0/sms/send`
// (which test-sms used to POST to with Basic auth) does not exist -- it
// returned an HTML page, so response.json() threw
// `Unexpected token '<', "<!DOCTYPE "...` and the button reported that.
test('nothing targets the non-existent api.voip.ms host', () => {
const offenders = [];
for (const dir of SCAN_DIRS) {
for (const file of walk(path.join(ROOT, dir))) {
// Comments stripped: the note in routes/api_conf.js explaining this
// very bug names the bad host, and describing a mistake is not
// making it.
const src = stripComments(fs.readFileSync(file, 'utf8'));
src.split('\n').forEach((line, i) => {
if (line.includes('api.voip.ms')) {
offenders.push(`${path.relative(ROOT, file)}:${i + 1}`);
}
});
}
}
expect(offenders).toEqual([]);
});
-78
View File
@@ -1,78 +0,0 @@
'use strict';
const { _Interfaces: Interfaces } = require('../plugins/discovery/proxmox');
// Regression coverage for the MAC/IP mismatch: the plugin used to collect MACs
// and IPs into two flat lists and zip them by index, so on a multi-NIC guest
// -- or any guest where one NIC had no address -- the directory recorded an IP
// against the wrong MAC. Interfaces keys by MAC so a pairing can only come from
// the source that observed both together.
describe('proxmox Interfaces', () => {
test('keeps each IP on the NIC it was observed on', () => {
const i = new Interfaces();
i.add('AA:BB:CC:00:00:01', ['10.0.0.5'], 'eth0');
i.add('AA:BB:CC:00:00:02', ['192.168.9.7'], 'eth1');
expect(i.toArray()).toEqual([
{ mac: 'aa:bb:cc:00:00:01', ip: '10.0.0.5', ips: ['10.0.0.5'], name: 'eth0' },
{ mac: 'aa:bb:cc:00:00:02', ip: '192.168.9.7', ips: ['192.168.9.7'], name: 'eth1' },
]);
});
test('a NIC with no address does not steal the next NIC\'s IP', () => {
const i = new Interfaces();
i.add('AA:BB:CC:00:00:01', [], 'eth0'); // stopped/unconfigured
i.add('AA:BB:CC:00:00:02', ['10.0.0.9'], 'eth1');
const byMac = Object.fromEntries(i.toArray().map(x => [x.mac, x.ip]));
expect(byMac['aa:bb:cc:00:00:01']).toBeNull();
expect(byMac['aa:bb:cc:00:00:02']).toBe('10.0.0.9');
});
test('merges the config MAC with the agent-reported address for the same NIC', () => {
const i = new Interfaces();
i.add('aa:bb:cc:00:00:01', ['10.0.0.5'], 'eth0'); // guest agent
i.add('AA:BB:CC:00:00:01', [], 'net0'); // VM config, same NIC
expect(i.toArray()).toHaveLength(1);
expect(i.toArray()[0]).toMatchObject({ mac: 'aa:bb:cc:00:00:01', ip: '10.0.0.5' });
});
test('collects multiple addresses on one NIC without inventing a second NIC', () => {
const i = new Interfaces();
i.add('aa:bb:cc:00:00:01', ['10.0.0.5', '10.0.0.6'], 'eth0');
expect(i.toArray()).toHaveLength(1);
expect(i.toArray()[0].ips).toEqual(['10.0.0.5', '10.0.0.6']);
expect(i.primaryIp()).toBe('10.0.0.5');
});
test('ignores placeholder and malformed MACs', () => {
const i = new Interfaces();
i.add('00:00:00:00:00:00', [], 'eth0');
i.add('not-a-mac', [], 'eth1');
i.add('', [], 'eth2');
expect(i.toArray()).toEqual([]);
expect(i.primaryMac()).toBeNull();
});
test('keeps an address that arrived without a usable MAC', () => {
const i = new Interfaces();
i.add(null, ['10.0.0.5'], 'eth0');
expect(i.primaryIp()).toBe('10.0.0.5');
expect(i.primaryMac()).toBeNull();
});
test('primary values prefer a NIC that actually has an address', () => {
const i = new Interfaces();
i.add('aa:bb:cc:00:00:01', [], 'eth0');
i.add('aa:bb:cc:00:00:02', ['10.0.0.9'], 'eth1');
expect(i.primaryIp()).toBe('10.0.0.9');
expect(i.primaryMac()).toBe('aa:bb:cc:00:00:02');
});
test('a fully unaddressed guest still reports its MAC', () => {
const i = new Interfaces();
i.add('aa:bb:cc:00:00:01', [], 'net0');
expect(i.primaryIp()).toBeNull();
expect(i.primaryMac()).toBe('aa:bb:cc:00:00:01');
});
});
-26
View File
@@ -1,26 +0,0 @@
'use strict';
// Authenticate an agent from a Bearer token (the same token the agent presents
// on its WSS channel). Used by agent-facing REST endpoints (secrets, IAM) that
// are NOT admin-gated — the caller is the agent itself, not an admin session.
const { Agent } = require('../models/agent');
// Resolve a Bearer token to its (non-revoked) Agent, or null. Every failure
// collapses to null so a probing caller learns nothing about which part was
// wrong.
async function authenticateAgent(req) {
const auth = req.headers['authorization'] || '';
const m = /^Bearer\s+(.+)$/i.exec(auth);
if (!m) return null;
const token = String(m[1]).trim();
if (!token) return null;
try {
const agent = await Agent.authenticate(token);
return agent || null;
} catch (_) {
return null;
}
}
module.exports = { authenticateAgent };
-99
View File
@@ -1,99 +0,0 @@
'use strict';
// The Ed25519 key pair the SSO signs high-risk agent commands with, stored in
// OpenBao at `secret/agent/signing-key`.
//
// This used to be generated in the AgentManager constructor and kept only in
// memory, which made the whole signing scheme decorative: every SSO restart
// produced a new key, so the `public_key` pinned in an agent's agent.yml stopped
// matching and the agent either rejected everything or (because it skips
// verification when no key is configured) executed everything unverified. A
// trust anchor that changes on restart is not a trust anchor.
//
// Requires the sso-broker OpenBao policy to grant `secret/agent/*`
// (theta-suite setup.sh). Without it the load fails and signing is reported as
// unavailable -- we deliberately do NOT fall back to an ephemeral key, because
// signing with a key no agent has ever seen is worse than refusing: it looks
// like it worked.
const crypto = require('crypto');
const baoConf = require('@simpleworkjs/bao-conf');
const PATH = 'agent/signing-key'; // baoConf adds the secret/data prefix
let cached = null; // { privateKeyPem, publicKeyPem, publicKeyBase64 }
let loadError = null;
// Agents pin the raw 32-byte Ed25519 public key, base64-encoded (see the Go
// client's verifySignature, which base64-decodes cfg.public_key and expects
// ed25519.PublicKeySize bytes). Node hands us SPKI PEM, so strip the 12-byte
// DER prefix to get the raw key the agent actually wants.
function rawPublicKeyBase64(publicKeyPem) {
const der = crypto.createPublicKey(publicKeyPem).export({ type: 'spki', format: 'der' });
return Buffer.from(der.subarray(der.length - 32)).toString('base64');
}
function generate() {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ed25519', {
privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
publicKeyEncoding: { type: 'spki', format: 'pem' }
});
return { privateKeyPem: privateKey, publicKeyPem: publicKey };
}
// Load the stored key pair, generating and persisting one on first run.
// Idempotent and safe to call repeatedly; the result is cached in-process.
async function load() {
if (cached) return cached;
let stored = null;
try {
stored = await baoConf.get(PATH);
} catch (err) {
loadError = `could not read ${PATH} from OpenBao: ${err.message}`;
console.error(`[agent_keys] ${loadError}`);
return null;
}
if (stored && stored.privateKeyPem && stored.publicKeyPem) {
cached = {
privateKeyPem: stored.privateKeyPem,
publicKeyPem: stored.publicKeyPem,
publicKeyBase64: rawPublicKeyBase64(stored.publicKeyPem)
};
loadError = null;
return cached;
}
// First run: mint one and persist it before use, so a crash between
// generating and storing can't leave agents pinned to a key we forgot.
const fresh = generate();
try {
await baoConf.set(PATH, fresh);
} catch (err) {
loadError = `could not persist a signing key to ${PATH}: ${err.message}. `
+ 'Re-run ./setup.sh so the sso-broker policy grants secret/agent/*.';
console.error(`[agent_keys] ${loadError}`);
return null;
}
cached = {
...fresh,
publicKeyBase64: rawPublicKeyBase64(fresh.publicKeyPem)
};
loadError = null;
console.log('[agent_keys] generated and stored a new agent signing key');
return cached;
}
function status() {
return { available: !!cached, error: loadError };
}
// Test seam: drop the in-process cache.
function _reset() {
cached = null;
loadError = null;
}
module.exports = { load, status, rawPublicKeyBase64, _reset, PATH };
+106 -227
View File
@@ -1,19 +1,26 @@
'use strict';
const crypto = require('crypto');
const agentKeys = require('./agent_keys');
const { Agent } = require('../models/agent');
// Tracks the live WebSocket for each enrolled agent and brokers commands to it.
//
// The durable facts about an agent (identity, host binding, last seen, last
// discovery/telemetry) live in the Agent table; this class holds only what
// cannot be persisted -- the open socket. That split is what makes an installed
// -but-offline agent visible, and what stops a restart from erasing the fleet.
class AgentManager {
constructor() {
// agentId -> { ws, ipAddress, connectedAt, lastResponse, pending }
this.live = new Map();
this.agents = new Map(); // token -> agentRecord
this.privateKeyPem = null;
this.publicKeyPem = null;
this.initKeyPair();
}
initKeyPair() {
try {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ed25519', {
privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
publicKeyEncoding: { type: 'spki', format: 'pem' }
});
this.privateKeyPem = privateKey;
this.publicKeyPem = publicKey;
} catch (err) {
console.error('[AgentManager] Failed to generate Ed25519 key pair:', err);
}
}
/**
@@ -21,103 +28,63 @@ class AgentManager {
* Sort keys alphabetically, remove whitespace, omit 'signature' key.
*/
canonicalize(payload) {
const sortObj = (val) => {
if (val === null || typeof val !== 'object') return val;
if (Array.isArray(val)) return val.map(sortObj);
const sorted = {};
const keys = Object.keys(val).filter(k => k !== 'signature').sort();
for (const k of keys) {
sorted[k] = sortObj(val[k]);
}
return sorted;
};
return JSON.stringify(sortObj(payload));
const cleanObj = {};
const sortedKeys = Object.keys(payload).filter(k => k !== 'signature').sort();
for (const key of sortedKeys) {
cleanObj[key] = payload[key];
}
return JSON.stringify(cleanObj);
}
/**
* Sign payload using the persisted Ed25519 private key. Throws when no key is
* available rather than minting a throwaway one -- an agent verifies against
* the key pinned in its agent.yml, so a signature from a key it has never
* seen is not a weaker signature, it is a broken command that looks fine from
* this side.
* Sign payload using Ed25519 private key.
* Returns base64 encoded signature.
*/
async signPayload(payload) {
const keys = await agentKeys.load();
if (!keys) {
const { error } = agentKeys.status();
throw new Error(`agent command signing is unavailable: ${error || 'no signing key'}`);
signPayload(payload) {
if (!this.privateKeyPem) {
throw new Error('Ed25519 private key is not initialized');
}
const canonicalBytes = Buffer.from(this.canonicalize(payload), 'utf8');
return crypto.sign(null, canonicalBytes, keys.privateKeyPem).toString('base64');
const signature = crypto.sign(null, canonicalBytes, this.privateKeyPem);
return signature.toString('base64');
}
async publicKeyBase64() {
const keys = await agentKeys.load();
return keys ? keys.publicKeyBase64 : null;
}
async publicKeyPem() {
const keys = await agentKeys.load();
return keys ? keys.publicKeyPem : null;
}
// Bind a freshly authenticated socket to an enrolled agent. `agent` is an
// Agent row that Agent.authenticate() has already vouched for -- this method
// never sees a raw token and must never be called with an unauthenticated one.
// Synchronous by design. The caller must attach its `message` listener in the
// same tick as the connection is accepted: `ws` drops events emitted before a
// listener exists, and the agent sends `discovery` immediately on open, so
// awaiting a database round-trip here silently lost every agent's first
// discovery frame. The connect timestamp is persisted in the background.
registerAgent(agent, ws, remoteAddress) {
const existing = this.live.get(agent.id);
registerAgent(token, ws, remoteAddress) {
const existing = this.agents.get(token);
if (existing && existing.ws && existing.ws !== ws) {
try { existing.ws.close(4002, 'Superseded by new connection'); } catch (e) {}
}
this.live.set(agent.id, {
const agentRecord = {
token,
ws,
ipAddress: remoteAddress,
hostname: 'unknown',
connectedAt: new Date().toISOString(),
lastResponse: null
});
lastSeen: new Date().toISOString(),
discovery: {},
telemetry: {},
pendingResponses: new Map()
};
agent.update({
last_seen: Math.floor(Date.now() / 1000),
last_ip: remoteAddress || null
}).catch(err => console.error(`[AgentManager] could not record connect for ${agent.id}:`, err.message));
this.agents.set(token, agentRecord);
return agentRecord;
}
unregisterAgent(agentId, ws) {
const state = this.live.get(agentId);
if (state && state.ws === ws) this.live.delete(agentId);
unregisterAgent(token, ws) {
const record = this.agents.get(token);
if (record && record.ws === ws) {
this.agents.delete(token);
}
}
// Drop an agent's live socket now. Revocation that only takes effect on the
// next reconnect is not revocation -- a connected agent would keep receiving
// commands indefinitely.
disconnect(agentId, code = 4003, reason = 'Disconnected by server') {
const state = this.live.get(agentId);
if (!state || !state.ws) return false;
try { state.ws.close(code, reason); } catch (e) {}
this.live.delete(agentId);
return true;
}
handleDiscovery(token, payload) {
const agent = this.agents.get(token);
if (!agent) return;
isConnected(agentId) {
const state = this.live.get(agentId);
return !!(state && state.ws && state.ws.readyState === 1);
}
async touch(agent, extra = {}) {
await agent.update({
last_seen: Math.floor(Date.now() / 1000),
...extra
}).catch(err => console.error(`[AgentManager] could not persist agent ${agent.id}:`, err.message));
}
async handleDiscovery(agent, payload) {
const discovery = {
agent.lastSeen = new Date().toISOString();
agent.hostname = payload.hostname || agent.hostname;
agent.discovery = {
hostname: payload.hostname || '',
ip_addresses: Array.isArray(payload.ip_addresses) ? payload.ip_addresses : [],
os: payload.os || '',
@@ -125,124 +92,30 @@ class AgentManager {
cpu: payload.cpu || '',
ram_total_gb: payload.ram_total_gb || 0,
disk_total_gb: payload.disk_total_gb || 0,
location: payload.location || 'default',
// The agent's enabled capabilities (from its local agent.yml). The agent
// is the authoritative source for what it will actually do.
capabilities: payload.capabilities || {}
location: payload.location || 'default'
};
await this.touch(agent, { lastDiscovery: discovery });
await this.applyDiscoveryToDirectory(agent, discovery);
}
// An agent runs ON the host it describes, which makes it the most
// authoritative source the directory has -- more so than a hypervisor API or
// a network scan. It previously updated nothing at all: the facts sat on an
// in-memory record and were lost on disconnect.
//
// When the agent is bound to a resource we write that row directly; guessing
// is only for an unbound agent, and then we let the shared reconciler do the
// matching (same MAC/IP/name rules every other source goes through) rather
// than inventing a second matcher here.
async applyDiscoveryToDirectory(agent, discovery) {
try {
const { Resource } = require('../models/resource');
const metadata = {
os: discovery.os || undefined,
kernel: discovery.kernel || undefined,
cpu: discovery.cpu || undefined,
ram_total_gb: discovery.ram_total_gb || undefined,
disk_total_gb: discovery.disk_total_gb || undefined,
ip: (discovery.ip_addresses || [])[0] || undefined,
public_ip: discovery.public_ip || undefined,
agentId: agent.id,
last_seen: Date.now()
};
// Drop undefined so a field the agent could not determine never
// overwrites a good value already in the directory.
for (const k of Object.keys(metadata)) if (metadata[k] === undefined) delete metadata[k];
handleTelemetry(token, payload) {
const agent = this.agents.get(token);
if (!agent) return;
if (agent.resourceId) {
const resource = await Resource.get(agent.resourceId);
if (!resource) return;
const merged = { ...(resource.metadata || {}), ...metadata };
const sources = new Set(merged.discovery_sources || []);
sources.add('theta-agent');
merged.discovery_sources = [...sources];
await resource.update({ metadata: merged, updated_on: Math.floor(Date.now() / 1000) });
return;
}
agent.lastSeen = new Date().toISOString();
agent.telemetry = {
cpu_usage_percent: payload.cpu_usage_percent || 0,
ram_usage_percent: payload.ram_usage_percent || 0,
disk_usage_percent: payload.disk_usage_percent || 0,
zfs_health: payload.zfs_health || 'N/A',
gpu_usage_percent: payload.gpu_usage_percent ?? -1,
timestamp: payload.timestamp || new Date().toISOString()
};
}
if (!discovery.hostname) return;
const { DiscoveryReconciler } = require('../services/discovery_reconciler');
const { ResourceEdge } = require('../models/resource');
const hostSlug = `host-${discovery.hostname.toLowerCase().replace(/[^a-z0-9_-]/g, '-')}`;
await DiscoveryReconciler.reconcile('theta-agent', {
resources: [{
kind: 'host',
name: discovery.hostname,
slug: hostSlug,
metadata: { ...metadata, subType: 'linux', managed: true }
}],
edges: []
});
// Find the matched or created host resource
const allHosts = await Resource.list({ where: { kind: 'host' } });
const hostRes = allHosts.find(r =>
r.name.toLowerCase() === discovery.hostname.toLowerCase() ||
r.slug === hostSlug ||
r.metadata?.agentId === agent.id
);
if (hostRes) {
// Bind the agent to its Host resource
await agent.update({ resourceId: hostRes.id }).catch(() => {});
// Attach host to matching Site by Public IP if not already parented
const existingEdges = await ResourceEdge.list({ where: { childId: hostRes.id } });
if (existingEdges.length === 0) {
const sites = await Resource.list({ where: { kind: 'site' } });
let targetSite = null;
if (discovery.public_ip) {
targetSite = sites.find(s => {
const siteIp = (s.metadata?.public_ip || s.metadata?.ip || s.metadata?.address || '').trim();
return siteIp && (siteIp === discovery.public_ip || siteIp.includes(discovery.public_ip));
});
}
if (!targetSite) targetSite = sites[0];
if (targetSite) {
await ResourceEdge.create({
id: crypto.randomUUID(),
parentId: targetSite.id,
childId: hostRes.id,
relation: 'hosts'
}).catch(() => {});
}
}
}
} catch (err) {
// Never let a directory write break the agent connection.
console.error(`[AgentManager] discovery -> directory failed for agent ${agent.id}:`, err.message);
handleHeartbeat(token, payload, ws) {
const agent = this.agents.get(token);
if (agent) {
agent.lastSeen = new Date().toISOString();
}
}
async handleTelemetry(agent, payload) {
await this.touch(agent, {
lastTelemetry: {
cpu_usage_percent: payload.cpu_usage_percent || 0,
ram_usage_percent: payload.ram_usage_percent || 0,
disk_usage_percent: payload.disk_usage_percent || 0,
zfs_health: payload.zfs_health || 'N/A',
gpu_usage_percent: payload.gpu_usage_percent ?? -1,
timestamp: payload.timestamp || new Date().toISOString()
}
});
}
async handleHeartbeat(agent, payload, ws) {
await this.touch(agent);
try {
ws.send(JSON.stringify({
type: 'heartbeat_ack',
@@ -251,51 +124,57 @@ class AgentManager {
} catch (e) {}
}
async handleResponse(agent, payload) {
const state = this.live.get(agent.id);
if (state) {
state.lastResponse = {
handleResponse(token, payload) {
const agent = this.agents.get(token);
if (agent) {
agent.lastSeen = new Date().toISOString();
agent.lastResponse = {
status: payload.status || 'ok',
message: payload.message || '',
output: payload.output || '',
timestamp: new Date().toISOString()
};
}
await this.touch(agent);
}
async sendCommand(agent, commandType, payload = {}, isHighRisk = false) {
const state = this.live.get(agent.id);
if (!state || !state.ws || state.ws.readyState !== 1) {
throw new Error(`Agent "${agent.name}" is not connected`);
sendCommand(token, commandType, payload = {}, isHighRisk = false) {
const agent = this.agents.get(token);
if (!agent || !agent.ws || agent.ws.readyState !== 1) {
throw new Error(`Agent with token "${token}" is not connected`);
}
const finalPayload = { ...payload };
if (isHighRisk) finalPayload.signature = await this.signPayload(finalPayload);
if (isHighRisk) {
finalPayload.signature = this.signPayload(finalPayload);
}
const message = { type: commandType, payload: finalPayload };
state.ws.send(JSON.stringify(message));
const message = {
type: commandType,
payload: finalPayload
};
agent.ws.send(JSON.stringify(message));
return message;
}
// Live view for one agent, for merging into its row.
liveState(agentId) {
const state = this.live.get(agentId);
if (!state) return { connected: false, lastResponse: null };
return {
connected: !!(state.ws && state.ws.readyState === 1),
ipAddress: state.ipAddress,
connectedAt: state.connectedAt,
lastResponse: state.lastResponse || null
};
}
// Every enrolled agent, connected or not.
async listAgents() {
const rows = await Agent.list();
return rows.map(a => a.toPublic(this.liveState(a.id)));
getConnectedAgents() {
const list = [];
const now = new Date();
for (const [token, agent] of this.agents.entries()) {
list.push({
token,
hostname: agent.hostname,
ipAddress: agent.ipAddress,
connectedAt: agent.connectedAt,
lastSeen: agent.lastSeen,
discovery: agent.discovery,
telemetry: agent.telemetry,
lastResponse: agent.lastResponse || null,
isOnline: (now - new Date(agent.lastSeen)) < 90000
});
}
return list;
}
}
module.exports = new AgentManager();
module.exports.AgentManager = AgentManager;
+17 -26
View File
@@ -41,23 +41,17 @@ function assertKind(kind) {
if (!KINDS.includes(kind)) throw new Error(`invalid resource kind: ${kind} (must be host or app)`);
}
// Strip the kind prefix a directory resource slug may carry (`host_theta-env` ->
// `theta-env`), leaving the resource's name slug. Services are stored bare
// (`sso-manager`), so this is a no-op for them.
function resourceNameSlug(slug) {
return String(slug || '').replace(/^(site|host|app)_/, '');
}
// {site}_{kind}_{nameSlug}_{level} — the per-resource group for ONE resource.
// Matches docs/GROUPS.md §2 (`S_host_<host>_<level>` / `S_app_<app>_<level>`):
// `site` is the site resource's slug verbatim (`site_local`), `kind` is the
// group-model kind (`host`/`app`), `nameSlug` is the resource's name (kind
// stripped, e.g. `theta-env` from `host_theta-env`). So a host `host_theta-env`
// yields `site_local_host_theta-env_access` and a service `sso-manager` yields
// `site_local_app_sso-manager_access`.
function resourceGroupCns(site, kind, nameSlug, level) {
assertKind(kind);
return `${site}_${kind}_${slugify(nameSlug)}_${level}`;
// {site}_{slug}_{level} — the per-resource group for one resource.
//
// Both `site` and `slug` are the resource slugs verbatim (e.g. `site_local`,
// `host_theta-env`), NOT slugified or kind-inserted: directory resource slugs
// carry their kind as a prefix (`host_theta-env`), so `site_local` + `host_theta-env`
// yields `site_local_host_theta-env_access`. Services are stored without a
// prefix (`sso-manager`), yielding `site_local_sso-manager_access`. This is the
// convention the auto-provisioner, the resolver, and the access-request tests
// all share -- re-slugifying or inserting a kind would double the delimiter.
function resourceGroupCns(site, slug, level) {
return `${site}_${slug}_${level}`;
}
// {site}_hosts_<level> / {site}_apps_<level> (plural kind — the aggregate).
@@ -100,13 +94,11 @@ function levelGrants(level, wanted) {
// granted groups (see permission.onResource). This keeps the function pure over
// the user's membership only.
function hasPermission(memberOf, resource, level) {
// `site` is used verbatim (`site_local`); `kind` maps the directory `service`
// kind onto the group model's `app` (docs/GROUPS.md §11 — consoles/services are
// apps); `nameSlug` is the resource name with any kind prefix stripped.
// `site` and `slug` are used verbatim (resource slugs may carry a kind prefix,
// e.g. `site_local` / `host_theta-env`) -- see resourceGroupCns.
const site = resource && resource.site;
const rawKind = resource && resource.kind;
const kind = rawKind === 'service' ? 'app' : rawKind;
const nameSlug = resourceNameSlug(resource && resource.slug);
const kind = resource && resource.kind;
const slug = resource && resource.slug;
const set = new Set(memberOf || []);
if (set.has(GOD_ADMIN)) return true;
@@ -115,13 +107,13 @@ function hasPermission(memberOf, resource, level) {
if (isKnownLevel(level)) {
// admin / access
if (set.has(aggregateGroupCns(site, kind, level))) return true;
if (set.has(resourceGroupCns(site, kind, nameSlug, level))) return true;
if (set.has(resourceGroupCns(site, slug, level))) return true;
if (level === 'access' && hasPermission(memberOf, resource, 'admin')) return true;
return false;
}
// Opaque capability — exact aggregate or specific grant only.
if (set.has(aggregateGroupCns(site, kind, level))) return true;
if (set.has(resourceGroupCns(site, kind, nameSlug, level))) return true;
if (set.has(resourceGroupCns(site, slug, level))) return true;
return false;
}
@@ -130,7 +122,6 @@ module.exports = {
KNOWN_LEVELS,
KINDS,
slugify,
resourceNameSlug,
resourceGroupCns,
aggregateGroupCns,
siteSuperAdminCns,
-84
View File
@@ -1,84 +0,0 @@
'use strict';
// LDAP byte-pump relay (DESIGN.md §4). The agent forwards raw LDAP bytes from a
// local socket (SSSD) over the WSS channel as `ldap_tunnel` messages; this
// module relays them into the SSO's real OpenLDAP and pipes the responses back.
// The SSO does not parse LDAP either — it is a transparent socket relay.
const net = require('net');
const conf = require('@simpleworkjs/conf').ldap;
// Parse host:port from an ldap:// or ldaps:// URL. The relay connects plaintext
// to the SSO's own slapd (which is plaintext on localhost); an ldaps:// URL
// would need TLS termination here and is not supported yet (DESIGN.md §9.5).
function ldapTarget() {
const url = conf.url || 'ldap://localhost:389';
const m = /^ldaps?:\/\/([^:/]+)(?::(\d+))?/.exec(url);
const host = m ? m[1] : 'localhost';
const port = m && m[2] ? Number(m[2]) : 389;
return { host, port };
}
// Per-agent relay state: agentId -> Map(conn_id -> LDAP socket).
const relays = new Map();
function relayFor(agentId) {
if (!relays.has(agentId)) relays.set(agentId, new Map());
return relays.get(agentId);
}
// Handle one ldap_tunnel message from an agent.
function handleTunnel(agentId, ws, payload) {
const connId = payload.conn_id;
if (!connId) return;
const conns = relayFor(agentId);
// End of connection: close the relay socket.
if (payload.close) {
const sock = conns.get(connId);
if (sock) { sock.destroy(); conns.delete(connId); }
return;
}
const data = Buffer.from(payload.data || '', 'base64');
if (data.length === 0) return;
let sock = conns.get(connId);
if (!sock) {
const { host, port } = ldapTarget();
sock = net.connect(port, host);
conns.set(connId, sock);
// Relay OpenLDAP's responses back to the agent.
sock.on('data', (chunk) => {
if (ws.readyState === 1) {
ws.send(JSON.stringify({
type: 'ldap_tunnel',
payload: { conn_id: connId, data: chunk.toString('base64') }
}));
}
});
sock.on('close', () => {
conns.delete(connId);
if (ws.readyState === 1) {
ws.send(JSON.stringify({
type: 'ldap_tunnel',
payload: { conn_id: connId, close: true }
}));
}
});
sock.on('error', () => { sock.destroy(); });
}
sock.write(data);
}
// Drop every relay socket for an agent (on WSS disconnect).
function cleanup(agentId) {
const conns = relays.get(agentId);
if (conns) {
for (const sock of conns.values()) sock.destroy();
relays.delete(agentId);
}
}
module.exports = { handleTunnel, cleanup };
-21
View File
@@ -410,27 +410,6 @@ mintAppRouter.post('/', async (req, res, next) => {
}
});
// List the minted external-app tokens (metadata only — the token itself is shown
// once at mint and never stored; the accessor is a renewal/revoke handle and is
// never exposed). Lets the Apps tab show what has been minted instead of a
// credential vanishing into the void.
mintAppRouter.get('/', async (req, res, next) => {
try {
await permission.byGroup(req.user, [ADMIN_GROUP]);
const rows = await VaultAppToken.list();
res.json({ apps: rows.map((r) => ({
name: r.name,
createdBy: r.created_by,
createdOn: r.created_on,
lastRenewedAt: r.lastRenewedAt || null,
lastError: r.lastError || null,
})) });
} catch (e) {
if (e.status === 401) return res.status(403).json({ error: 'admin only' });
next(e);
}
});
module.exports = {
getOrCreateUserToken,
getOrCreateAdminToken,
+70 -1068
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -206,8 +206,8 @@
return '<div class="card shadow-sm service-card ' + (accessible ? 'border-success' : '') + '">'
+ '<div class="card-body">'
+ '<h5 class="card-title d-flex align-items-start gap-2">'
+ '<span>' + esc(r.name) + '</span>'
+ iconHtml
+ '<span>' + esc(r.name) + '</span>'
+ '</h5>'
+ '<div class="mb-2"><span class="badge bg-secondary">' + esc(r.kind)
+ (md.subType ? ' · ' + esc(md.subType) : '') + '</span>' + badges + '</div>'
+1 -4
View File
@@ -656,10 +656,7 @@
}
</script>
<!-- Wrapped in the same `.container` as the profile/edit cards above (which
closes before this block): without it the API Tokens card renders
full-bleed and is visibly wider than every other card on the site. -->
<div id="own-api-tokens-section" class="container" style="display:none">
<div id="own-api-tokens-section" style="display:none">
<div class="row mt-3">
<div class="col-12">
<div class="card shadow-lg">
+2 -53
View File
@@ -4,13 +4,12 @@
<div class="row">
<div class="col-12">
<div class="card shadow">
<div class="card-header d-flex justify-content-between align-items-center flex-wrap gap-2">
<div class="card-header">
<ul class="nav nav-tabs card-header-tabs" id="vault-tabs" role="tablist">
<li class="nav-item"><button class="nav-link active" data-bs-toggle="tab" data-bs-target="#tab-secrets" type="button"><i class="fa-solid fa-lock"></i> Secrets</button></li>
<li class="nav-item" id="vault-apps-tab" style="display:none"><button class="nav-link" data-bs-toggle="tab" data-bs-target="#tab-apps" type="button"><i class="fa-solid fa-key"></i> Apps</button></li>
<li class="nav-item"><button class="nav-link" data-bs-toggle="tab" data-bs-target="#tab-shared" type="button"><i class="fa-solid fa-share-nodes"></i> Shared</button></li>
</ul>
<span class="small text-muted"><i class="fa-solid fa-database me-1"></i>Powered by <a href="https://openbao.org" target="_blank" rel="noopener">OpenBao</a></span>
</div>
<div class="card-body p-0">
<div class="tab-content">
@@ -18,10 +17,7 @@
<div class="tab-pane fade show active" id="tab-secrets">
<div class="card-header d-flex flex-wrap justify-content-between align-items-center gap-2">
<h5 class="mb-0" id="vault-title"><i class="fas fa-lock"></i> My Secrets <small class="text-muted">(personal namespace)</small></h5>
<div class="d-flex align-items-center gap-2">
<a href="/docs/vault" class="text-reset" title="Vault help &amp; documentation"><i class="fa-solid fa-circle-question"></i></a>
<button class="btn btn-primary btn-sm" onclick="showCreateModal()"><i class="fas fa-plus"></i> New Secret</button>
</div>
<button class="btn btn-primary btn-sm" onclick="showCreateModal()"><i class="fas fa-plus"></i> New Secret</button>
</div>
<div class="p-3">
<div class="row">
@@ -92,20 +88,6 @@ curl "$VAULT_ADDR/v1/secret/data/apps/<span id="app-name-display"></span>/conf"
</div>
</div>
</div>
<div class="row mt-3">
<div class="col-12">
<div class="card shadow-sm">
<div class="card-header d-flex justify-content-between align-items-center">
<h5 class="card-title mb-0"><i class="fa-solid fa-key me-1"></i> Minted apps</h5>
<button class="btn btn-sm btn-outline-primary" onclick="loadApps()"><i class="fas fa-rotate"></i> Refresh</button>
</div>
<div class="card-body">
<p class="text-muted small mb-2">Each entry is a scoped OpenBao credential an external service uses to read <code>secret/apps/&lt;name&gt;/*</code>. The token itself is shown <strong>once</strong> at mint — this list is metadata sso keeps so it can renew the token and so you can see what's been minted. If an app shows a renewal error, re-mint it here.</p>
<div id="apps-list"><div class="text-muted small">Loading…</div></div>
</div>
</div>
</div>
</div>
</div>
</div>
@@ -423,44 +405,12 @@ curl "$VAULT_ADDR/v1/secret/data/apps/<span id="app-name-display"></span>/conf"
document.getElementById('app-token').textContent = result.token;
document.getElementById('app-name-display').textContent = name;
document.getElementById('app-result-card').classList.remove('d-none');
loadApps();
} catch (err) {
errorEl.textContent = err.message;
errorEl.classList.remove('d-none');
}
}
// List the minted external-app tokens (metadata only). Makes the Apps tab show
// what's been minted instead of a credential that vanishes after the once-only
// token display.
async function loadApps() {
const $list = document.getElementById('apps-list');
if (!$list) return;
$list.textContent = 'Loading…';
try {
const res = await fetch('/api/vault/apps', {
headers: { 'auth-token': app.auth.getToken() }
});
if (!res.ok) { $list.innerHTML = '<div class="text-danger small">Failed to load apps.</div>'; return; }
const { apps = [] } = await res.json();
if (!apps.length) { $list.innerHTML = '<div class="text-muted small">No apps minted yet.</div>'; return; }
$list.innerHTML = '<div class="list-group shadow-sm">' + apps.map(a => {
const ok = !a.lastError;
const renewed = a.lastRenewedAt ? ' · renewed ' + moment(a.lastRenewedAt).fromNow() : ' · never renewed';
return `<div class="list-group-item d-flex justify-content-between align-items-center">
<div>
<strong class="font-monospace">${app.util.escapeHtml(a.name)}</strong>
${ok ? '<span class="badge bg-success ms-1">renewing</span>' : '<span class="badge bg-danger ms-1" title="' + app.util.escapeHtml(a.lastError) + '">renewal error</span>'}
<div class="small text-muted">minted ${moment(a.createdOn).format('YYYY-MM-DD HH:mm')}${renewed}</div>
</div>
<span class="font-monospace small text-muted">secret/apps/${app.util.escapeHtml(a.name)}/</span>
</div>`;
}).join('') + '</div>';
} catch (err) {
$list.innerHTML = '<div class="text-danger small">Failed to load apps: ' + app.util.escapeHtml(err.message) + '</div>';
}
}
function copyText(text) {
navigator.clipboard.writeText(text).then(() => app.messages.toast('Copied', 'success'));
}
@@ -622,7 +572,6 @@ curl "$VAULT_ADDR/v1/secret/data/apps/<span id="app-name-display"></span>/conf"
'<i class="fas fa-lock"></i> Vault Secrets <small class="text-muted">(admin — all of secret/)</small>';
document.getElementById('secret-path-label').textContent = 'Secret path (under secret/)';
document.getElementById('secret-path-input').placeholder = 'e.g. apps/my-service/conf';
loadApps();
}
loadSecrets();
loadShared();
-110
View File
@@ -1,110 +0,0 @@
'use strict';
// End-to-end test of the LDAP byte-pump tunnel (DESIGN.md §4).
//
// Simulates the agent: enrolls one, connects to the SSO WSS with its token,
// sends a real LDAP bind request as raw bytes in an `ldap_tunnel` message, and
// verifies the SSO relays it into OpenLDAP and pipes the bind response back.
// This proves the SSO side of the tunnel without needing the agent binary.
const WebSocket = require('ws');
const SSO_URL = process.env.SSO_URL || 'http://sso:3001';
const WS_URL = SSO_URL.replace(/^http/, 'ws') + '/api/agent/ws';
const TEST_CREDS = { uid: 'test', password: 'MyTestPassword!2' };
const USER_DN = 'cn=test,ou=people,dc=test,dc=local';
function fail(msg) { console.error('E2E FAIL:', msg); process.exit(1); }
async function waitForSso() {
for (let i = 0; i < 60; i++) {
try {
const r = await fetch(`${SSO_URL}/health`);
if (r.ok) return;
} catch (_) {}
await new Promise((res) => setTimeout(res, 1000));
}
fail('SSO never became ready');
}
async function login() {
const r = await fetch(`${SSO_URL}/api/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(TEST_CREDS),
});
if (!r.ok) fail(`login failed: ${r.status}`);
const body = await r.json();
return body.token;
}
async function enrollAgent(authToken) {
const r = await fetch(`${SSO_URL}/api/agent/enroll`, {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'auth-token': authToken },
body: JSON.stringify({ name: `e2e-${Date.now().toString(36)}` }),
});
if (!r.ok) fail(`enroll failed: ${r.status}`);
const body = await r.json();
return body.token;
}
// Build a simple LDAP bind request (version 3) as raw BER bytes.
function buildBindRequest(dn, password) {
const dnBuf = Buffer.from(dn, 'utf8');
const pwBuf = Buffer.from(password, 'utf8');
const version = Buffer.from([0x02, 0x01, 0x03]);
const name = Buffer.concat([Buffer.from([0x04, dnBuf.length]), dnBuf]);
const simple = Buffer.concat([Buffer.from([0x80, pwBuf.length]), pwBuf]);
const bindContent = Buffer.concat([version, name, simple]);
const bindReq = Buffer.concat([Buffer.from([0x60, bindContent.length]), bindContent]);
const msgId = Buffer.from([0x02, 0x01, 0x01]);
const msgContent = Buffer.concat([msgId, bindReq]);
return Buffer.concat([Buffer.from([0x30, msgContent.length]), msgContent]);
}
// A successful bind response is a BindResponse (0x61) with resultCode 0 (0x0a 01 00).
function isSuccessBindResponse(buf) {
return buf.includes(Buffer.from([0x61])) && buf.includes(Buffer.from([0x0a, 0x01, 0x00]));
}
async function main() {
await waitForSso();
const authToken = await login();
const agentToken = await enrollAgent(authToken);
console.log('E2E: enrolled agent, connecting WSS...');
const ws = new WebSocket(`${WS_URL}?token=${agentToken}`);
await new Promise((res, rej) => { ws.on('open', res); ws.on('error', rej); });
console.log('E2E: WSS connected');
const bindBytes = buildBindRequest(USER_DN, TEST_CREDS.password);
ws.send(JSON.stringify({
type: 'ldap_tunnel',
payload: { conn_id: 'e2e-1', data: bindBytes.toString('base64') },
}));
console.log('E2E: sent bind request bytes');
const result = await new Promise((res, rej) => {
const timeout = setTimeout(() => rej(new Error('timed out waiting for bind response')), 10000);
ws.on('message', (data) => {
let msg;
try { msg = JSON.parse(data); } catch (_) { return; }
if (msg.type !== 'ldap_tunnel') return;
if (msg.payload.close) return;
const buf = Buffer.from(msg.payload.data || '', 'base64');
if (isSuccessBindResponse(buf)) {
clearTimeout(timeout);
res({ ok: true, bytes: buf.length });
}
});
ws.on('error', (e) => { clearTimeout(timeout); rej(e); });
});
console.log(`E2E: got successful bind response (${result.bytes} bytes)`);
ws.close();
console.log('E2E PASS');
process.exit(0);
}
main().catch((e) => fail(e.message));