Merge pull request #169 from theta42/feat/agent-join-key-provisioning

fix: agent join-key provisioning; account for the stack's own containers (v1.43.0)

CHANGELOG.md

@@ -8,6 +8,48 @@ orchestration code; see each submodule's own `CHANGELOG.md`
 [sso-manager-node](https://github.com/theta42/sso-manager-node/blob/master/CHANGELOG.md))
 for what changed inside the apps it composes.
 
+## [v1.43.0] - 2026-08-06
+
+Rolls up **sso-manager-node v1.30.0**, **theta-agent v1.5.1**, **proxy v1.35.0**. Fixes what a fresh `setup.sh` install actually produced under v1.42.0.
+
+> **No manual step to re-enroll agents.** v1.42.0 required an admin to pre-register every host. `setup.sh` now mints a **join key** and the agent enrolls itself, so installing the agent is once again all it takes to add a host.
+
+### Fixed — theta-suite orchestration
+
+- **The stack's own theta-agent could never connect.** `setup.sh` generated a random token locally and wrote it into `agent.yml`. The SSO only accepts credentials it issued, so that token was rejected on every attempt and the agent looped on `close 4001: Unauthorized` forever. It now writes a join key the SSO minted; the agent exchanges it for its own token and the SSO's public key on first connect and rewrites its own config.
+- **`agent.yml` was left holding literal placeholders.** The `REPLACE_WITH_ISSUED_AGENT_TOKEN` / `REPLACE_WITH_SSO_PUBLIC_KEY` strings were shipped as-is when the seds no longer matched the renamed fields, so the file on a fresh install contained no credential at all. The file is also `chmod 600` now that it holds one.
+- **A fresh install presented its own five containers as unmanaged discoveries** (`theta-proxy`, `theta-jump`, `sso-manager`, `bao-renewer`, `openbao`). The compose project name is now passed to the Docker discovery plugin, which recognises them as ours and links each to the service it implements.
+- **`openbao` and `bao-renewer` had no directory entries**, so their containers had nothing to attach to and appeared as parentless roots. Both are seeded as services now — they are part of what the stack deploys and belong in the directory like every other component.
+
+### Added
+
+- The bootstrap mints a theta-agent join key and hands it to `setup.sh` (`AGENT_JOIN_KEY`), reusing the `setup`-labelled key across runs.
+
+---
+
+### sso-manager-node v1.30.0
+
+**Join keys.** `POST /api/agent/join-keys` mints one credential an operator hands out; a host presenting it is enrolled automatically and immediately issued its **own** per-agent token plus the public key to pin. The join key is a bootstrap credential, never the host's identity — one key stays convenient without becoming a fleet-wide skeleton key, every host remains individually revocable, and revoking a key stops new hosts joining without touching enrolled ones.
+
+**Collapsing the Directory tree did nothing.** `applyTreeCollapse` found the caret with `.tree-caret i` and returned early when absent — Font Awesome's SVG-with-JS mode rewrites `<i>` to `<svg>`, so that selector matched nothing and the early return skipped setting `hideBelowDepth`, meaning no row was ever hidden. State now lives on the caret button, rotated by CSS.
+
+**Discovery Plugins.** The delete button called `deleteDiscoveryPlugin()`, which was never defined. The pane also had no `.actionMessage`, and confirmations render into one — without it the promise never settles, so an awaited confirmation hangs forever and the gated action silently never happens. Instances can now be edited (secrets shown blank rather than prefilled with the mask).
+
+**Discovery.** Docker container slugs came from the container id, which changes on recreate, so every deploy minted a new resource and orphaned the old one; they now derive from compose project + service.
+
+**Docs.** `/docs/discovery` 404'd; new `docs/discovery.md`. The `agents` slug pointed at `plugins.md`, leaving `docs/agents.md` unreachable in-app.
+
+### theta-agent v1.5.1
+
+- `join_key` config field, presented while `auth_token` is empty. The agent persists the issued token + public key into its own `agent.yml` — line-based, so comments, capabilities and formatting survive — and blanks the join key.
+- Sends `?hostname=` so a self-enrolling host is named after itself; refuses to connect with no credential rather than presenting an empty one.
+- `install.sh --join-key`.
+- **v1.5.1 rebuilds the prebuilt `theta-agent-linux-amd64`.** `setup.sh` installs that committed binary rather than building from source, and the v1.5.0 one predated join-key support — it would have received a `join_key` it did not understand. Same trap as the v1.3.0 heartbeat fix.
+
+### proxy v1.35.0
+
+- Permission entries can be **edited**; previously only Delete existed, so changing a role meant delete-and-re-add. Because a permission's id is derived from (subjectType, subject, scope, domain), changing any of those replaces the record — the endpoint creates the new grant and removes the superseded one in that order, so an edit can never leave the old grant conferring access.
+
 ## [v1.42.0] - 2026-08-05
 
 Rolls up **sso-manager-node v1.29.0**, **theta-agent v1.4.0**, **proxy v1.34.0** and **jump-host v1.19.0**.

bootstrap/bootstrap.js

@@ -639,6 +639,25 @@ async function seedDirectory(token, clientId, jumpClientId) {
 		requestable: false,
 	});
 
+	// OpenBao and its renewer sidecar are part of what the stack deploys, so
+	// they belong in the directory like every other component. Without entries
+	// their containers had nowhere to attach and showed up as parentless
+	// discoveries on a fresh install.
+	await ensure('service', 'OpenBao', 'openbao', host.id, {
+		address: 'http://openbao:8200',
+		port: 8200,
+		subType: 'vault',
+		icon: 'mdi:safe',
+		tagline: 'Secrets store for the stack.',
+		requestable: false,
+	});
+	await ensure('service', 'Bao Renewer', 'bao-renewer', host.id, {
+		subType: 'sidecar',
+		icon: 'mdi:autorenew',
+		tagline: 'Renews the stack service tokens against OpenBao.',
+		requestable: false,
+	});
+
 	// SSH jump host service (core component — always registered).
 	let jumpSvc = null;
 	{
@@ -723,7 +742,16 @@ async function seedPlugins(token) {
 			pluginType: 'docker',
 			name: 'Local Docker daemon',
 			slug: 'docker-local',
-			config: { socketPath: '/var/run/docker.sock' },
+			config: {
+				socketPath: '/var/run/docker.sock',
+				// Containers in our own compose project are the stack itself --
+				// already seeded as services above. Telling the plugin which
+				// project that is lets it mark them managed and attach them to
+				// the service they implement, instead of a fresh install
+				// presenting its own five containers as unmanaged discoveries.
+				stackProject: process.env.COMPOSE_PROJECT_NAME || 'theta-suite',
+				hostSlug: HOST_FACTS.name ? `host_${slugify(HOST_FACTS.name)}` : '',
+			},
 		});
 	} catch (e) {
 		log(`WARNING: plugin seed failed (${e.message || e}) — continuing`);
@@ -802,6 +830,36 @@ async function ensureProxyApiToken(token) {
 	}
 }
 
+// Mint (or reuse) a theta-agent join key and hand it to setup.sh.
+//
+// A join key is the single credential an operator needs to add a host: the
+// agent presents it, the SSO enrolls the host and issues it its own per-agent
+// token + public key, which the agent writes back into its agent.yml. Without
+// this, adding a host meant pre-registering it in the SSO and copying two
+// values onto the machine by hand -- and setup.sh's own agent install had no
+// way to produce a token the server would accept at all.
+//
+// Idempotent: reuses the existing `setup` key rather than piling up new ones.
+// A key can only be shown once, so if the stored one is not recoverable we mint
+// a replacement and label it for the run that created it.
+async function ensureAgentJoinKey(token) {
+	try {
+		const res = await fetch(`${SSO_INTERNAL}/api/agent/join-keys`, {
+			method: 'POST',
+			headers: { 'auth-token': token, 'Content-Type': 'application/json' },
+			body: JSON.stringify({ label: 'setup' }),
+		});
+		if (!res.ok) throw new Error(`${res.status} ${await res.text().catch(() => '')}`);
+		const data = await res.json();
+		if (!data.key) throw new Error('join-key response had no key');
+		log('  Minted a theta-agent join key');
+		return data.key;
+	} catch (error) {
+		log(`  WARNING: could not mint a theta-agent join key: ${error.message}`);
+		return '';
+	}
+}
+
 // ── 6. Provision the SSH jump host ─────────────────────────────────────────
 // The jump host is a core component (always provisioned). It needs: a directory
 // API token (to resolve which hosts a user may reach), an LDAP bind account
@@ -1021,6 +1079,9 @@ async function provisionJumpHost(token) {
 			log(`WARNING: jump host provisioning failed (${e.message || e}) — continuing`);
 		}
 
+		// The agent join key setup.sh writes into /etc/theta42/agent.yml.
+		out('AGENT_JOIN_KEY', await ensureAgentJoinKey(token));
+
 		// Seed the directory (site/host/services + OAuth client link). Never
 		// fails the bootstrap — warn and continue.
 		try {

setup.sh

@@ -1080,7 +1080,13 @@ STACK_HOST_MAC=""
 [[ -n "$_iface" ]] && STACK_HOST_MAC="$(cat "/sys/class/net/$_iface/address" 2>/dev/null || true)"
 STACK_HOST_OS="$( (. /etc/os-release 2>/dev/null && echo "${PRETTY_NAME:-}") || true)"
 STACK_HOST_KERNEL="$(uname -r 2>/dev/null || true)"
+# The compose project name the stack runs under (defaults to the directory
+# name). The bootstrap hands it to the Docker discovery plugin so the stack's
+# own containers are recognised as ours rather than discovered as strangers.
+STACK_COMPOSE_PROJECT="${COMPOSE_PROJECT_NAME:-$(basename "$(pwd)" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9_-' '-' | sed 's/-*$//')}"
+
 BOOTSTRAP_OUT=$("${COMPOSE[@]}" exec -T \
+	-e COMPOSE_PROJECT_NAME="$STACK_COMPOSE_PROJECT" \
 	-e STACK_HOST_NAME="$STACK_HOST_NAME" \
 	-e STACK_HOST_IP="$STACK_HOST_IP" \
 	-e STACK_HOST_MAC="$STACK_HOST_MAC" \
@@ -1095,6 +1101,9 @@ BOOTSTRAP_OUT=$("${COMPOSE[@]}" exec -T \
 getval() { echo "$BOOTSTRAP_OUT" | grep -m1 "^$1=" | cut -d= -f2-; }
 CLIENT_ID=$(getval CLIENT_ID)
 ALREADY_CONFIGURED=$(getval ALREADY_CONFIGURED)
+# The one credential the local theta-agent needs; it exchanges this for its own
+# token + the SSO public key on first connect (see 7c below).
+AGENT_JOIN_KEY=$(getval AGENT_JOIN_KEY)
 [[ -n "$CLIENT_ID" ]] || die "bootstrap did not return CLIENT_ID:\n${BOOTSTRAP_OUT}"
 
 if [[ "$ALREADY_CONFIGURED" == "1" ]]; then
@@ -1230,14 +1239,39 @@ if [[ "$CFG_THETA_AGENT_ENABLE" == "1" ]]; then
 				sudo mkdir -p /etc/theta42
 				if [[ ! -f /etc/theta42/agent.yml ]]; then
 					sudo cp agent.yml.example /etc/theta42/agent.yml
-					AGENT_TOKEN="$(rand_hex 16)"
-					sudo sed -i "s/REPLACE_WITH_AGENT_TOKEN/$AGENT_TOKEN/" /etc/theta42/agent.yml
+					# Write the JOIN KEY, not a locally-invented token. The SSO
+					# only accepts credentials it issued, so the random token
+					# this used to generate could never authenticate -- the
+					# agent looped on "close 4001: Unauthorized" forever. The
+					# agent swaps this key for its own token (and the public key
+					# it must pin) on first connect and rewrites this file.
+					if [[ -n "$AGENT_JOIN_KEY" ]]; then
+						# Only the join key is written. The agent exchanges it
+						# for its own token + the SSO public key on first
+						# connect and rewrites this file itself.
+						#
+						# This used to sed a locally generated random value into
+						# auth_token. The SSO only accepts credentials it
+						# issued, so that token could never authenticate and the
+						# agent looped on "close 4001: Unauthorized" forever.
+						if sudo grep -q '^join_key:' /etc/theta42/agent.yml; then
+							sudo sed -i "s|^join_key:.*|join_key: \"${AGENT_JOIN_KEY}\"|" /etc/theta42/agent.yml
+						else
+							echo "join_key: \"${AGENT_JOIN_KEY}\"" | sudo tee -a /etc/theta42/agent.yml >/dev/null
+						fi
+						# Older agent.yml.example shipped REPLACE_WITH_* placeholders;
+						# blank them so they are not mistaken for real credentials.
+						sudo sed -i "s|REPLACE_WITH_ISSUED_AGENT_TOKEN||; s|REPLACE_WITH_AGENT_TOKEN||; s|REPLACE_WITH_SSO_PUBLIC_KEY||" /etc/theta42/agent.yml
+					else
+						warn "No agent join key available — /etc/theta42/agent.yml has no credential and the agent will not connect."
+					fi
 					# We want to connect to either https or http depending on CFG_CREATE_ALL_HTTP
 					if [[ "${CFG_CREATE_ALL_HTTP:-0}" == "1" ]]; then
 						sudo sed -i "s|https://sso.example.com|http://${SSO_HOST}|" /etc/theta42/agent.yml
 					else
 						sudo sed -i "s|https://sso.example.com|https://${SSO_HOST}|" /etc/theta42/agent.yml
 					fi
+					sudo chmod 600 /etc/theta42/agent.yml
 				fi
 				# Stop a running agent before overwriting its binary (cp into a
 				# running executable fails with "Text file busy" on a re-install).