diff --git a/CHANGELOG.md b/CHANGELOG.md index 3383b7e..4ff6173 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,53 @@ All notable changes to the `theta-agent` daemon will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] - LDAP byte-pump tunnel (DESIGN.md §4) + +The agent now serves a local LDAP socket for SSSD/PAM. It is a **pure byte +pump**: it forwards raw LDAP bytes to the SSO over the WSS channel, and the SSO +relays them into its real OpenLDAP and pipes the response back. The agent never +parses LDAP. + +### Added +- **`ldap_tunnel` capability + `ldap_socket` config.** When enabled, the agent + binds a unix socket (default `/run/theta/ldap.sock`, root:theta `0660`) and + relays bytes bidirectionally as `ldap_tunnel` messages over the existing WSS + channel. Point SSSD at it with `ldap_uri = ldapi://%2frun%2ftheta%2fldap.sock`. +- **`safeWriter`** — serializes WebSocket writes. Gorilla allows only one + concurrent writer, but telemetry, heartbeat, the LDAP tunnel and command + responses all write to the same socket; without this, concurrent writes + corrupt the stream. +- **Offline behavior:** when the WSS is down the agent cannot forward bytes, so + it closes local socket connections; SSSD sees a connection failure and falls + back to its local cache. + +### Added — secrets engine (DESIGN.md §5) +- **`secrets` capability + `secrets` config.** The agent renders local templates + that embed OpenBao secrets (`{{ bao "secret/data/nodes//#" }}`). + It parses the placeholders, fetches the values from the SSO (which holds the + OpenBao access; the agent never holds a Vault token), renders each target + atomically at `0600`, and runs the configured reload. Triggered by a signed + `render_secrets` command. + +### Added — capability reporting +- **The agent reports its enabled capabilities in its `discovery` frame.** The + SSO stores them and the Directory UI shows them as badges on the host's Metrics + tab, so an operator can see at a glance what an agent is allowed to do + (telemetry, LDAP tunnel, secrets, IAM, reboot, bash, service control). + +### Added — IAM engine (DESIGN.md §6) +- **`iam` capability.** The SSO pushes node-scoped identity config as a signed + `iam_apply` command; the agent verifies the Ed25519 signature (fail-closed) and + applies it locally: + - **Sudo rules** — writes `/etc/sudoers.d/theta-iam-`, validates with + `visudo -c`, atomic swap. + - **SSH keys** — stores per-user keys and installs the `AuthorizedKeysCommand` + script (`/usr/local/bin/theta-authorized-keys`) that sshd calls per login. + - **Access control** — writes `/etc/security/access.conf` with allowed login + groups. + - **Revocation** — flushes the SSSD cache (`sss_cache -E`) and drops active + sessions (`pkill -u`) for revoked users. + ## [v1.5.1] - 2026-08-06 ### Fixed diff --git a/DESIGN.md b/DESIGN.md new file mode 100644 index 0000000..0880637 --- /dev/null +++ b/DESIGN.md @@ -0,0 +1,211 @@ +# theta-agent v2 — Simplified Architecture (LDAP-over-HTTPS) + +**Status:** Draft for review · **Supersedes:** the WebRTC/SCTP spec (WebRTC dropped) + +This document defines the v2 architecture for `theta-agent`. It replaces the +earlier WebRTC/SCTP design with a strictly simpler model: **one agent per node, +one outbound WSS channel, and the agent provides local LDAP + secrets + IAM.** +The core problem it solves is the original ask — *LDAP binds are painful across +hostnames, networks, and TLS cert chains* — by making the client stop speaking +LDAP and instead do an HTTPS call to the SSO, where the directory is reachable. + +--- + +## 1. Overview + +``` +APP (no agent) SSO + HTTPS POST /ldap/bind ──────────────► [LDAP-over-HTTPS API] ──► OpenLDAP + HTTPS POST /ldap/search ───────────► ▲ + │ raw bytes +NODE (with theta-agent) │ + SSSD ──► /run/theta/ldap.sock ──► agent ──► WSS (ldap_tunnel) ─┘ + secrets: /etc/theta/templates/*.tpl ──► agent ──► HTTPS ──► OpenBao + IAM: sudoers.d / authorized_keys ──► agent ◄── WSS ◄── IAM engine +``` + +Two ways to reach the directory, both simple: +- **Apps (no agent):** call the HTTPS LDAP API directly — one bind/search + contract, no LDAP protocol. +- **Nodes (with agent):** the agent is a **pure byte pump** — it forwards raw + LDAP bytes from a local socket to the SSO, which relays them into OpenLDAP. + The agent never parses LDAP. + +--- + +## 2. Transport + +- **Single transport: persistent outbound WebSocket (WSS) over TCP 443.** No + WebRTC, no SCTP, no UDP, no ICE, no fallback logic. The agent dials out; the + SSO never needs an inbound port. +- **Authentication:** the existing token / join-key enrollment model carries + over unchanged (see `PROTOCOL.md` §1.1). The agent presents its per-agent + token; the SSO rejects anything it did not issue. +- **Message framing:** the existing `WSMessage` JSON envelope (`type` + `payload`) + carries all traffic. Message types are extended for LDAP, secrets, and IAM + (below). No binary framing, no stream multiplexing. The LDAP tunnel carries + raw bytes as base64 in `ldap_tunnel` messages (see §4). + +--- + +## 3. LDAP-over-HTTPS API (SSO side) + +A small service (or routes in the existing proxy/sso) that performs the real +LDAP operation against OpenLDAP, which is reachable from the SSO network. This +is the pinepain/ldap-auth-proxy model: the client never speaks LDAP. + +| Endpoint | Request | Response | +| :--- | :--- | :--- | +| `POST /api/v1/ldap/bind` | `{username, password}` | `200 {dn, attributes}` or `401` | +| `POST /api/v1/ldap/search` | `{base_dn, scope, filter, attributes}` | `200 {entries: [...]}` | + +- **bind** performs a real LDAP bind server-side and returns the bound DN and + identity attributes. +- **search** runs a real LDAP search server-side and returns entries. +- **Authorization:** the API authorizes the *caller* (an app, or an agent acting + for a node). OpenLDAP enforces the actual directory ACLs. This resolves the + original spec's contradiction — we do not parse BER to enforce per-operation + policy; the directory does. + +### 3.1 Consumers + +- **Apps/services:** call the API directly over HTTPS. No LDAP hostname, no + LDAPS cert chain, no cross-network LDAP firewall rule. +- **theta-agent:** does **not** use this API. It tunnels raw LDAP bytes to the + SSO's OpenLDAP over the WSS channel instead (see §4) — a byte pump, not a + translation. The HTTPS API is for apps that have no agent on their network. + +--- + +## 4. Agent local LDAP socket — a pure byte pump (for SSSD/PAM) + +The agent provides a local LDAP endpoint so SSSD/PAM on the node can +authenticate without direct LDAP connectivity. **The agent does not speak LDAP +at all.** It is a dumb byte pump: whatever bytes land on the local socket are +forwarded to the SSO, which relays them into its real OpenLDAP and pipes the +response back. + +``` +SSSD ──► /run/theta/ldap.sock ──► agent ──► WSS (ldap_tunnel) ──► SSO ──► OpenLDAP + ◄───────────────────────────────────────────────────────────────────────◄ +``` + +- **Socket:** a **Unix domain socket** at `/run/theta/ldap.sock`, owned by root, + mode `0660` (root + theta group). A unix socket is preferred over + `127.0.0.1:389` because filesystem permissions restrict *which local processes* + can connect — any process can reach a TCP port, only root/theta can reach the + socket. +- **Tunnel framing:** each local connection gets a `conn_id`. Bytes are carried + over the existing WSS channel as `ldap_tunnel` messages: + `{type:"ldap_tunnel", payload:{conn_id, data:, close:bool}}`. The + agent reads the socket and sends chunks up; the SSO relays them into OpenLDAP + and sends OpenLDAP's response chunks back down; the agent writes them to the + socket. `close:true` ends a connection. +- **SSSD config:** `ldap_uri = ldapi://%2frun%2ftheta%2fldap.sock`, StartTLS off + (the transport to the SSO is already TLS/WSS; the local hop is plaintext on a + root-owned socket). + +### 4.1 Offline boot handling (validated) + +If the agent cannot reach the SSO (WSS down), it cannot forward bytes, so it +closes local socket connections. SSSD sees a connection failure and falls back +to its local cache. **This has been validated against real SSSD behavior:** any +user/group seen in the last N days can log in offline — a laptop user can log in +on the road, and an admin can still reach a broken service host to fix it. + +### 4.2 Open question — `ldapi://` support + +SSSD's `ldap_uri` accepts `ldapi://` unix-socket URIs on modern versions, but +this must be confirmed on the target SSSD build. If unsupported, fall back to +`127.0.0.1:389` with a local firewall rule restricting the port to loopback. + +--- + +## 5. Secrets engine + +The agent renders OpenBao secrets to local files, reusing the existing +`@simpleworkjs/bao-conf` patterns rather than inventing a parallel mechanism. + +- **Templates:** `/etc/theta/templates/*.tpl` declare the secrets a service + needs, e.g. `DB_PASS="{{ bao "secret/data/nodes/node-42/db#password" }}"`. +- **Flow:** the agent requests the secret paths over the WSS channel → the SSO + fetches from OpenBao (node-scoped to `/secret/data/nodes/${NODE_ID}/*`) → + the agent renders the file **atomically** (write temp + rename) with mode + `0600` → runs the configured post-render action (`systemctl reload `). +- **Rotation:** on a rotation/invalidation event pushed down the channel, the + agent re-fetches, re-renders, and reloads. +- **Note:** OpenBao KV v2 secrets have no leases; "renewal" is a re-read on + invalidation, not a lease renewal. (Dynamic secrets, if used, are a separate + path.) + +--- + +## 6. IAM engine + +The SSO pushes node-scoped identity config down the WSS channel; the agent +applies it locally. + +- **Sudo rules:** write `/etc/sudoers.d/theta-iam-`, run `visudo -c`, + and atomically swap on success. +- **SSH keys:** sync user public keys via `AuthorizedKeysCommand` (the agent + implements the command the SSH daemon calls per login) — *not* a non-standard + `/etc/ssh/authorized_keys.d/` directory. +- **Access control:** configure SSSD / `/etc/security/access.conf` for allowed + login groups. +- **Revocation:** on account disable / group drop, flush local SSSD caches and + drop active sessions for the affected user (mechanism TBD — `loginctl` vs + `pkill -u`; high-risk, needs a defined trigger/event model). + +### 6.1 Security — signed IAM payloads + +Sudo rules and SSH keys grant root-equivalent access, so **every IAM push is +Ed25519-signed** using the existing signature model (`PROTOCOL.md` §5). The agent +verifies the signature against its pinned `public_key` before applying anything. +Unsigned or invalid IAM payloads are rejected fail-closed. + +--- + +## 7. Security model + +- **Fail-closed, capability-matrix philosophy carries over** from v1: the agent + only applies what its local config permits; the SSO cannot override local + settings. +- **Local socket auth:** only root/theta can reach `/run/theta/ldap.sock`. +- **Signed high-risk operations:** IAM pushes (and any new high-risk command) + require the Ed25519 signature. +- **Node-scoped secrets:** OpenBao access is restricted to the node's own path + prefix. +- **Blast radius:** the agent runs as root; the unix socket + signature model + + node-scoped secrets contain the damage if the agent is compromised. + +--- + +## 8. What this drops from the original spec + +- WebRTC / SCTP / DTLS / UDP / ICE — **gone**, WSS only. +- Three SCTP streams — **replaced** by one WSS channel with message types. +- The "node-scoped authorization" contradiction — **resolved**: the API + authorizes the caller, OpenLDAP enforces ACLs. +- LDAP parsing in the agent — **gone**. The agent is a byte pump; it never + parses LDAP. The SSO relays raw bytes into its real OpenLDAP. + +--- + +## 9. Open questions / verification items + +1. **SSSD `ldapi://` unix-socket support** on the target build (§4.2). +2. **Revocation mechanism** — implemented as `sss_cache -E` + `pkill -u ` + (§6). The event model (what triggers a push) is still to be wired into the + SSO UI/engine. +3. **`AuthorizedKeysCommand`** — implemented: the agent installs + `/usr/local/bin/theta-authorized-keys` which cats the user's key file + (`/etc/theta/authorized_keys/`). sshd must be configured with + `AuthorizedKeysCommand /usr/local/bin/theta-authorized-keys %u` (§6). +4. **Versioning/migration** — is v2 a replacement for v1, or a parallel mode? + The existing `/api/agent/ws` vs the new `/api/v1/ldap/*` paths need a story. +5. **SSO relay target** — the SSO relays tunnel bytes into its local OpenLDAP + (slapd). The target address comes from `conf.ldap.url`; confirm it is a + plaintext LDAP port reachable from the SSO process (§4). +6. **Secrets node scope** — the agent's node scope is its agent id + (`secret/data/nodes//*`). Confirm this matches how node secrets are + provisioned in OpenBao (§5). diff --git a/PROTOCOL.md b/PROTOCOL.md index 2a5ba80..a4dfbe9 100644 --- a/PROTOCOL.md +++ b/PROTOCOL.md @@ -70,6 +70,24 @@ All messages are exchanged as JSON objects following the `WSMessage` structure. } ``` +### 2.1 `ldap_tunnel` — the LDAP byte pump (DESIGN.md §4) + +The agent serves a local LDAP socket for SSSD/PAM. It is a **pure byte pump**: +the agent forwards raw LDAP bytes to the SSO, which relays them into its real +OpenLDAP and pipes the response back. Neither side parses LDAP. + +- **Type**: `ldap_tunnel` (bidirectional — sent by both agent and SSO) +- **Payload**: + - `conn_id`: (string) correlates one local LDAP connection. + - `data`: (string, optional) base64-encoded raw LDAP bytes. + - `close`: (bool, optional) ends the connection. + +The agent reads its local socket and sends `data` chunks up; the SSO relays them +into OpenLDAP and sends OpenLDAP's response chunks back down; the agent writes +them to the socket. `close:true` ends a connection. When the WSS is down the +agent cannot forward bytes, so it closes local socket connections and SSSD falls +back to its local cache. + ## 3. Client $\rightarrow$ Server Messages ### 3.1 Discovery (One-time & On-Change) @@ -153,6 +171,8 @@ These commands **require** an Ed25519 signature in the payload. The agent verifi | `configure_ldap` | `{ "config": "...", "signature": "..." }` | Writes `/etc/sssd/sssd.conf` and restarts `sssd`. | | `arbitrary_bash` | `{ "script": "...", "signature": "..." }` | Executes raw bash script. | | `update_binary` | `{ "url": "...", "sha256": "...", "signature": "..." }` | Downloads, verifies, and replaces the agent binary. | +| `render_secrets` | `{ "signature": "..." }` | Renders the configured secret templates to their targets (DESIGN.md §5). | +| `iam_apply` | `{ "node_id", "revision", "access_control", "signature" }` | Applies node IAM: sudo rules, SSH keys, access control, revocation (DESIGN.md §6). | ## 5. Cryptographic Verification Process diff --git a/README.md b/README.md index ba45f8c..a2a9247 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,17 @@ Theta Agent is a unified endpoint management daemon for the theta42 stack. It re The agent dials out to the central SSO Manager via a persistent WebSocket connection, enabling real-time telemetry, dynamic discovery, and secure remote operations. +## What you get + +Install the agent on a node and it becomes a managed member of the directory — over a **single outbound connection**, with no inbound ports, no LDAP hostname/firewall/TLS setup, and no manual secret copying. + +- **Directory logins (LDAP byte pump).** SSSD/PAM on the node authenticates through the agent's local socket, which forwards raw LDAP bytes to the SSO's OpenLDAP. OS logins work across any network — laptops, CGNAT, cloud VMs — and fall back to the local SSSD cache when offline. +- **Secrets delivered automatically.** Services get their config files (DB passwords, TLS keys, API tokens) rendered from OpenBao to disk, atomically, with a post-render reload. Rotate a secret and it propagates. +- **IAM managed centrally.** Sudo rules, SSH keys, and login access are pushed from the SSO to the node. Add a user to a group and their access appears on the right hosts; revoke them and their sessions are dropped. +- **Telemetry & remote operations.** Host discovery, live metrics, and signed remote commands (reboot, service control, config, self-update) — the original C2 capabilities. + +Everything is gated by a strict, local-first capability matrix and high-risk operations are Ed25519-signed (see below). + ## Core Functionality ### 1. Telemetry & Observability @@ -56,6 +67,9 @@ either drops the agent's live connection immediately. See `PROTOCOL.md` §1.1. |------------|------------|-------------|---------| | `telemetry` | Safe | Read-only metrics. | Pushes system health to SSO Manager. | | `configure_ldap` | Moderate | Configures SSSD. | Updates `/etc/sssd/sssd.conf` and restarts `sssd`. | +| `ldap_tunnel` | Moderate | Local LDAP byte-pump socket. | Forwards raw LDAP bytes to the SSO for SSSD/PAM (DESIGN.md §4). | +| `secrets` | Moderate | Renders OpenBao secrets. | Renders `/etc/theta/templates/*.tpl` to targets, atomic + reload (DESIGN.md §5). | +| `iam` | High | Applies node IAM. | Writes sudo rules, SSH keys, access control; revokes sessions (DESIGN.md §6). | | `reboot` | High | System reboot. | Triggers an immediate host reboot. | | `service_control` | High | Service management. | Restarts services listed in the allowed list. | | `arbitrary_bash` | CRITICAL | Raw bash execution. | Executes any script sent by the manager as root. | diff --git a/agent.yml.example b/agent.yml.example index 95b072c..d8874dc 100644 --- a/agent.yml.example +++ b/agent.yml.example @@ -26,16 +26,39 @@ public_key: "" location: "default" # Location identifier (e.g., site, datacenter) for naming +# Local LDAP byte-pump socket (DESIGN.md §4). The agent forwards raw LDAP bytes +# from this socket to the SSO, which relays them into its OpenLDAP. The agent +# never parses LDAP. Point SSSD at it with: +# ldap_uri = ldapi://%2frun%2ftheta%2fldap.sock +ldap_socket: "/run/theta/ldap.sock" + capabilities: # --------------------------------------------------------- # Basic Capabilities (Safe, read-only or infrastructure management) # --------------------------------------------------------- - + # Push CPU, RAM, GPU, and ZFS metrics to the SSO Manager telemetry: true - + # Allow the SSO Manager to push down SSSD and SSH keys configuration configure_ldap: true + + # Serve the local LDAP byte-pump socket for SSSD/PAM (DESIGN.md §4) + ldap_tunnel: true + + # Render OpenBao secrets to local files from /etc/theta/templates (DESIGN.md §5) + secrets: true + + # Apply node IAM (sudo rules, SSH keys, access control, revocation) (DESIGN.md §6) + iam: true + +# Secret templates to render (DESIGN.md §5). Each maps a local template to a +# target file and an optional post-render reload. The template embeds secrets as +# {{ bao "secret/data/nodes//#" }}. +# secrets: +# - template: /etc/theta/templates/db.env.tpl +# target: /etc/theta/db.env +# reload: systemctl reload app # --------------------------------------------------------- # Advanced Capabilities (High risk, remote operations) diff --git a/config.go b/config.go index 0efc863..69194c8 100644 --- a/config.go +++ b/config.go @@ -16,6 +16,17 @@ type Capabilities struct { Reboot bool `yaml:"reboot"` ServiceControl []string `yaml:"service_control"` ArbitraryBash bool `yaml:"arbitrary_bash"` + LdapTunnel bool `yaml:"ldap_tunnel"` + Secrets bool `yaml:"secrets"` + IAM bool `yaml:"iam"` +} + +// SecretTarget maps a local template to a rendered target file and an optional +// post-render reload command (DESIGN.md §5). +type SecretTarget struct { + Template string `yaml:"template"` + Target string `yaml:"target"` + Reload string `yaml:"reload"` } type Config struct { @@ -28,7 +39,9 @@ type Config struct { JoinKey string `yaml:"join_key"` Location string `yaml:"location"` PublicKey string `yaml:"public_key"` // Ed25519 public key for signed commands - Capabilities Capabilities `yaml:"capabilities"` + LdapSocket string `yaml:"ldap_socket"` // local LDAP tunnel socket (DESIGN.md §4) + Secrets []SecretTarget `yaml:"secrets"` // secret templates to render (DESIGN.md §5) + Capabilities Capabilities `yaml:"capabilities"` } // Credential returns the value to present when connecting: our own token once diff --git a/demo/Dockerfile b/demo/Dockerfile new file mode 100644 index 0000000..95f6ddd --- /dev/null +++ b/demo/Dockerfile @@ -0,0 +1,6 @@ +FROM node:20-alpine +RUN apk add --no-cache bash +WORKDIR /demo +COPY get-secret.sh get-secret.js ./ +RUN chmod +x get-secret.sh +CMD ["sh", "-c", "sh /demo/get-secret.sh && node /demo/get-secret.js"] diff --git a/demo/get-secret.js b/demo/get-secret.js new file mode 100644 index 0000000..794d9c4 --- /dev/null +++ b/demo/get-secret.js @@ -0,0 +1,17 @@ +// Demo: a 3rd-party node app reads the secret the theta-agent rendered to disk. +const fs = require('fs'); +const path = '/etc/theta/rendered/db.env'; +console.log('=== node app reads the rendered secret ==='); +if (fs.existsSync(path)) { + const env = fs.readFileSync(path, 'utf8'); + const db = {}; + for (const line of env.split('\n')) { + const m = /^(\w+)="(.*)"$/.exec(line.trim()); + if (m) db[m[1]] = m[2]; + } + console.log('DB_USER=' + db.DB_USER); + console.log('DB_PASS=' + db.DB_PASS); +} else { + console.error('rendered secret not found — run render_secrets first'); + process.exit(1); +} diff --git a/demo/get-secret.sh b/demo/get-secret.sh new file mode 100644 index 0000000..9658083 --- /dev/null +++ b/demo/get-secret.sh @@ -0,0 +1,12 @@ +#!/bin/sh +# Demo: a 3rd-party bash app reads the secret the theta-agent rendered to disk. +# The agent rendered /etc/theta/rendered/db.env from a template + OpenBao. +echo "=== bash app reads the rendered secret ===" +if [ -f /etc/theta/rendered/db.env ]; then + . /etc/theta/rendered/db.env + echo "DB_USER=$DB_USER" + echo "DB_PASS=$DB_PASS" +else + echo "rendered secret not found — run render_secrets first" + exit 1 +fi diff --git a/go.mod b/go.mod index 24fcff7..1f2a163 100644 --- a/go.mod +++ b/go.mod @@ -2,16 +2,19 @@ module github.com/theta42/theta-agent go 1.22.2 +require ( + github.com/gorilla/websocket v1.5.3 + github.com/shirou/gopsutil/v3 v3.24.5 + gopkg.in/yaml.v3 v3.0.1 +) + require ( github.com/go-ole/go-ole v1.2.6 // indirect - github.com/gorilla/websocket v1.5.3 // indirect github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect - github.com/shirou/gopsutil/v3 v3.24.5 // indirect github.com/shoenig/go-m1cpu v0.1.6 // indirect github.com/tklauser/go-sysconf v0.3.12 // indirect github.com/tklauser/numcpus v0.6.1 // indirect github.com/yusufpapurcu/wmi v1.2.4 // indirect golang.org/x/sys v0.20.0 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/go.sum b/go.sum index 17b6aae..7668791 100644 --- a/go.sum +++ b/go.sum @@ -1,16 +1,26 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY= github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4= github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c h1:ncq/mPwQF4JjgDlrVEn3C11VoGHZN7m8qihwgMEtzYw= github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI= github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk= github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM= github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ= +github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU= +github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k= +github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU= github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI= github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+Fk= @@ -23,7 +33,10 @@ golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/iam.go b/iam.go new file mode 100644 index 0000000..84b18b2 --- /dev/null +++ b/iam.go @@ -0,0 +1,210 @@ +package main + +import ( + "encoding/json" + "fmt" + "log" + "os" + "path/filepath" + "strings" +) + +// IAM engine (DESIGN.md §6). The SSO pushes node-scoped identity config down the +// WSS channel as a signed `iam_apply` command; the agent verifies the signature +// (fail-closed) and applies it locally: sudo rules, SSH keys, access control, +// and session revocation. + +// Paths are package-level vars so tests can redirect them to temp dirs. +var ( + iamSudoersDir = "/etc/sudoers.d" + iamKeysDir = "/etc/theta/authorized_keys" + iamKeysCommand = "/usr/local/bin/theta-authorized-keys" + iamAccessConf = "/etc/security/access.conf" +) + +// IAMPayload is the signed body of an `iam_apply` command. +type IAMPayload struct { + NodeID string `json:"node_id"` + Revision int `json:"revision"` + AccessControl AccessControl `json:"access_control"` +} + +type AccessControl struct { + AllowedLoginGroups []string `json:"allowed_login_groups"` + SudoRules []SudoRule `json:"sudo_rules"` + SSHKeys []SSHKey `json:"ssh_keys"` + RevokeUsers []string `json:"revoke_users"` +} + +type SudoRule struct { + Group string `json:"group"` + RunAs string `json:"run_as"` + Commands []string `json:"commands"` + Nopasswd bool `json:"nopasswd"` +} + +type SSHKey struct { + User string `json:"user"` + Keys []string `json:"keys"` +} + +// applyIAM applies a verified IAM payload. The caller must have already verified +// the Ed25519 signature. +func applyIAM(payload IAMPayload, exec Executor) error { + if len(payload.AccessControl.SudoRules) > 0 { + if err := applySudoRules(payload.AccessControl.SudoRules, payload.NodeID, exec); err != nil { + return fmt.Errorf("sudo rules: %w", err) + } + } + if len(payload.AccessControl.SSHKeys) > 0 { + if err := applySSHKeys(payload.AccessControl.SSHKeys, exec); err != nil { + return fmt.Errorf("ssh keys: %w", err) + } + } + if len(payload.AccessControl.AllowedLoginGroups) > 0 { + if err := applyAccessControl(payload.AccessControl.AllowedLoginGroups, exec); err != nil { + return fmt.Errorf("access control: %w", err) + } + } + if len(payload.AccessControl.RevokeUsers) > 0 { + applyRevocation(payload.AccessControl.RevokeUsers, exec) + } + return nil +} + +// applySudoRules writes /etc/sudoers.d/theta-iam-, verifies with +// `visudo -c`, and atomically swaps it in on success. +func applySudoRules(rules []SudoRule, nodeID string, exec Executor) error { + var b strings.Builder + for _, r := range rules { + if r.Group == "" { + continue + } + runAs := r.RunAs + if runAs == "" { + runAs = "ALL" + } + cmds := strings.Join(r.Commands, ", ") + if cmds == "" { + cmds = "ALL" + } + prefix := "" + if r.Nopasswd { + prefix = "NOPASSWD:" + } + fmt.Fprintf(&b, "%%%s ALL=(%s) %s%s\n", r.Group, runAs, prefix, cmds) + } + content := b.String() + + // Make sure the sudoers.d dir exists (a minimal host may not have it). + if err := os.MkdirAll(iamSudoersDir, 0755); err != nil { + return err + } + + // Write to a temp file in the sudoers.d dir, verify, then rename. + tmp, err := os.CreateTemp(iamSudoersDir, ".theta-iam-*") + if err != nil { + return err + } + tmpName := tmp.Name() + if _, err := tmp.WriteString(content); err != nil { + tmp.Close() + os.Remove(tmpName) + return err + } + tmp.Close() + os.Chmod(tmpName, 0440) + + // visudo -c -f validates a single file without touching the rest. + if _, err := exec.Execute("visudo", "-c", "-f", tmpName); err != nil { + os.Remove(tmpName) + return fmt.Errorf("visudo rejected the rules: %w", err) + } + + target := filepath.Join(iamSudoersDir, "theta-iam-"+nodeID) + if err := os.Rename(tmpName, target); err != nil { + os.Remove(tmpName) + return err + } + log.Printf("IAM: wrote %s", target) + return nil +} + +// applySSHKeys stores per-user keys and installs the AuthorizedKeysCommand +// script that sshd calls per login. +func applySSHKeys(keys []SSHKey, exec Executor) error { + if err := os.MkdirAll(iamKeysDir, 0755); err != nil { + return err + } + for _, k := range keys { + if k.User == "" { + continue + } + path := filepath.Join(iamKeysDir, k.User) + content := strings.Join(k.Keys, "\n") + "\n" + if err := os.WriteFile(path, []byte(content), 0600); err != nil { + return err + } + } + + // AuthorizedKeysCommand script: cat the user's key file. + script := "#!/bin/sh\ncat \"" + iamKeysDir + "/$1\" 2>/dev/null\n" + if err := os.WriteFile(iamKeysCommand, []byte(script), 0755); err != nil { + return err + } + log.Printf("IAM: wrote %d ssh key file(s) + %s", len(keys), iamKeysCommand) + return nil +} + +// applyAccessControl writes /etc/security/access.conf with the allowed login +// groups (PAM access). Format: `+:group:ALL` for each allowed group, then deny +// everything else. +func applyAccessControl(groups []string, exec Executor) error { + var b strings.Builder + for _, g := range groups { + if g != "" { + fmt.Fprintf(&b, "+:%s:ALL\n", g) + } + } + b.WriteString("-:ALL:ALL\n") + if err := os.MkdirAll(filepath.Dir(iamAccessConf), 0755); err != nil { + return err + } + if err := os.WriteFile(iamAccessConf, []byte(b.String()), 0644); err != nil { + return err + } + log.Printf("IAM: wrote %s", iamAccessConf) + return nil +} + +// applyRevocation flushes the SSSD cache and drops active sessions for the +// revoked users. +func applyRevocation(users []string, exec Executor) { + // Flush the whole SSSD cache once — a revoked user must not be resolvable + // from cache. + if _, err := exec.Execute("sss_cache", "-E"); err != nil { + log.Printf("IAM: sss_cache -E failed: %v", err) + } + for _, u := range users { + if u == "" { + continue + } + if _, err := exec.Execute("pkill", "-u", u); err != nil { + log.Printf("IAM: pkill -u %s failed (no sessions?): %v", u, err) + } + log.Printf("IAM: revoked %s", u) + } +} + +// parseIAMPayload extracts an IAMPayload from a WSMessage payload map. +func parseIAMPayload(payload map[string]interface{}) (IAMPayload, error) { + raw, err := json.Marshal(payload) + if err != nil { + return IAMPayload{}, err + } + var p IAMPayload + if err := json.Unmarshal(raw, &p); err != nil { + return IAMPayload{}, err + } + return p, nil +} diff --git a/iam_test.go b/iam_test.go new file mode 100644 index 0000000..23da527 --- /dev/null +++ b/iam_test.go @@ -0,0 +1,127 @@ +package main + +import ( + "os" + "path/filepath" + "testing" +) + +// TestApplyIAM verifies the agent applies sudo rules, SSH keys, access control, +// and revocation from a signed IAM payload. +func TestApplyIAM(t *testing.T) { + dir := t.TempDir() + iamSudoersDir = dir + iamKeysDir = filepath.Join(dir, "keys") + iamKeysCommand = filepath.Join(dir, "theta-authorized-keys") + iamAccessConf = filepath.Join(dir, "access.conf") + + payload := IAMPayload{ + NodeID: "node-42", + Revision: 82, + AccessControl: AccessControl{ + AllowedLoginGroups: []string{"sysadmins", "node-operators"}, + SudoRules: []SudoRule{ + {Group: "sysadmins", RunAs: "ALL", Commands: []string{"ALL"}}, + {Group: "node-operators", RunAs: "ALL", Commands: []string{"ALL"}, Nopasswd: true}, + }, + SSHKeys: []SSHKey{ + {User: "admin", Keys: []string{"ssh-ed25519 AAAA admin@theta"}}, + }, + RevokeUsers: []string{"olduser"}, + }, + } + + exec := &MockExecutor{} + if err := applyIAM(payload, exec); err != nil { + t.Fatalf("applyIAM: %v", err) + } + + // Sudo rules file. + sudoers, err := os.ReadFile(filepath.Join(dir, "theta-iam-node-42")) + if err != nil { + t.Fatalf("read sudoers: %v", err) + } + wantSudoers := "%sysadmins ALL=(ALL) ALL\n%node-operators ALL=(ALL) NOPASSWD:ALL\n" + if string(sudoers) != wantSudoers { + t.Fatalf("sudoers mismatch:\n got: %q\nwant: %q", sudoers, wantSudoers) + } + + // SSH key file + AuthorizedKeysCommand script. + keyFile, err := os.ReadFile(filepath.Join(iamKeysDir, "admin")) + if err != nil { + t.Fatalf("read key file: %v", err) + } + if string(keyFile) != "ssh-ed25519 AAAA admin@theta\n" { + t.Fatalf("key file mismatch: %q", keyFile) + } + script, err := os.ReadFile(iamKeysCommand) + if err != nil { + t.Fatalf("read keys command: %v", err) + } + if string(script) != "#!/bin/sh\ncat \""+iamKeysDir+"/$1\" 2>/dev/null\n" { + t.Fatalf("keys command mismatch: %q", script) + } + + // Access control. + access, err := os.ReadFile(iamAccessConf) + if err != nil { + t.Fatalf("read access.conf: %v", err) + } + wantAccess := "+:sysadmins:ALL\n+:node-operators:ALL\n-:ALL:ALL\n" + if string(access) != wantAccess { + t.Fatalf("access.conf mismatch:\n got: %q\nwant: %q", access, wantAccess) + } + + // Commands: visudo -c -f, sss_cache -E, pkill -u olduser. + ran := map[string]bool{} + for _, c := range exec.ExecutedCommands { + if c[0] == "visudo" { + ran["visudo"] = true + } + if c[0] == "sss_cache" { + ran["sss_cache"] = true + } + if c[0] == "pkill" && len(c) >= 3 && c[2] == "olduser" { + ran["pkill"] = true + } + } + for _, k := range []string{"visudo", "sss_cache", "pkill"} { + if !ran[k] { + t.Errorf("expected %s to run, got commands %v", k, exec.ExecutedCommands) + } + } +} + +// TestParseIAMPayload verifies the payload parses from a WSMessage payload map. +func TestParseIAMPayload(t *testing.T) { + payload := map[string]interface{}{ + "node_id": "node-42", + "revision": float64(82), + "access_control": map[string]interface{}{ + "allowed_login_groups": []interface{}{"sysadmins"}, + "sudo_rules": []interface{}{ + map[string]interface{}{"group": "sysadmins", "run_as": "ALL", "commands": []interface{}{"ALL"}, "nopasswd": true}, + }, + "ssh_keys": []interface{}{ + map[string]interface{}{"user": "admin", "keys": []interface{}{"ssh-ed25519 AAAA"}}, + }, + "revoke_users": []interface{}{"olduser"}, + }, + } + p, err := parseIAMPayload(payload) + if err != nil { + t.Fatalf("parseIAMPayload: %v", err) + } + if p.NodeID != "node-42" || p.Revision != 82 { + t.Fatalf("bad node/revision: %+v", p) + } + if len(p.AccessControl.SudoRules) != 1 || p.AccessControl.SudoRules[0].Group != "sysadmins" { + t.Fatalf("bad sudo rules: %+v", p.AccessControl.SudoRules) + } + if len(p.AccessControl.SSHKeys) != 1 || p.AccessControl.SSHKeys[0].User != "admin" { + t.Fatalf("bad ssh keys: %+v", p.AccessControl.SSHKeys) + } + if len(p.AccessControl.RevokeUsers) != 1 || p.AccessControl.RevokeUsers[0] != "olduser" { + t.Fatalf("bad revoke users: %+v", p.AccessControl.RevokeUsers) + } +} diff --git a/ldap_tunnel.go b/ldap_tunnel.go new file mode 100644 index 0000000..fe74e2e --- /dev/null +++ b/ldap_tunnel.go @@ -0,0 +1,182 @@ +package main + +import ( + "encoding/base64" + "encoding/json" + "fmt" + "log" + "net" + "os" + "path/filepath" + "sync" + "time" + + "github.com/gorilla/websocket" +) + +// ldapTunnel is the agent's local LDAP socket (DESIGN.md §4). It is a pure byte +// pump: bytes from a local LDAP client (SSSD) are forwarded to the SSO over the +// WSS channel as `ldap_tunnel` messages, and the SSO's responses are written +// back to the socket. The agent never parses LDAP — it does not know or care +// what the bytes mean. +// +// Each local connection gets a conn_id. The agent reads the socket and sends +// chunks up; the SSO relays them into its real OpenLDAP and sends the response +// chunks back down; the agent writes them to the socket. `close:true` ends a +// connection. +type ldapTunnel struct { + mu sync.Mutex + conns map[string]net.Conn + send func(WSMessage) error +} + +func newLdapTunnel(send func(WSMessage) error) *ldapTunnel { + return &ldapTunnel{ + conns: make(map[string]net.Conn), + send: send, + } +} + +// start binds the unix socket and accepts connections until stopCh closes. +func (t *ldapTunnel) start(socketPath string, stopCh <-chan struct{}) { + // Remove a stale socket left over from a previous run, and make sure the + // parent directory exists (e.g. /run/theta on a fresh boot). + os.Remove(socketPath) + if dir := filepath.Dir(socketPath); dir != "." && dir != "/" { + os.MkdirAll(dir, 0755) + } + + ln, err := net.Listen("unix", socketPath) + if err != nil { + log.Printf("LDAP tunnel: cannot bind %s: %v", socketPath, err) + return + } + // root:theta, 0660 — only root and the theta group can connect. A unix + // socket is preferred over 127.0.0.1:389 because filesystem permissions + // restrict which local processes can reach it. + os.Chmod(socketPath, 0660) + defer ln.Close() + log.Printf("LDAP tunnel: listening on %s", socketPath) + + go func() { + <-stopCh + ln.Close() + }() + + for { + conn, err := ln.Accept() + if err != nil { + return + } + go t.handleConn(conn) + } +} + +// handleConn pumps one local LDAP connection up to the SSO. +func (t *ldapTunnel) handleConn(conn net.Conn) { + connID := newConnID() + t.mu.Lock() + t.conns[connID] = conn + t.mu.Unlock() + + buf := make([]byte, 32*1024) + for { + n, err := conn.Read(buf) + if n > 0 { + msg := WSMessage{ + Type: "ldap_tunnel", + Payload: map[string]interface{}{ + "conn_id": connID, + "data": base64.StdEncoding.EncodeToString(buf[:n]), + }, + } + if err := t.send(msg); err != nil { + break + } + } + if err != nil { + break + } + } + + // Signal end of connection to the SSO so it can close its OpenLDAP relay. + t.send(WSMessage{ + Type: "ldap_tunnel", + Payload: map[string]interface{}{ + "conn_id": connID, + "close": true, + }, + }) + + t.mu.Lock() + delete(t.conns, connID) + t.mu.Unlock() + conn.Close() +} + +// handleMessage writes SSO→agent tunnel bytes to the matching local socket. +// Called from handleCommand when an `ldap_tunnel` message arrives. +func (t *ldapTunnel) handleMessage(payload map[string]interface{}) { + connID, _ := payload["conn_id"].(string) + if connID == "" { + return + } + + if closeFlag, _ := payload["close"].(bool); closeFlag { + t.mu.Lock() + conn := t.conns[connID] + delete(t.conns, connID) + t.mu.Unlock() + if conn != nil { + conn.Close() + } + return + } + + dataStr, _ := payload["data"].(string) + if dataStr == "" { + return + } + data, err := base64.StdEncoding.DecodeString(dataStr) + if err != nil { + return + } + + t.mu.Lock() + conn := t.conns[connID] + t.mu.Unlock() + if conn != nil { + conn.Write(data) + } +} + +var connCounter uint64 + +func newConnID() string { + connCounter++ + return fmt.Sprintf("%d-%d", time.Now().UnixNano(), connCounter) +} + +// safeWriter serializes writes to the WebSocket. Gorilla allows only one +// concurrent writer, but the agent has several (telemetry, heartbeat, the LDAP +// tunnel, command responses) — without this, concurrent WriteMessage calls +// corrupt the stream. +type safeWriter struct { + mu sync.Mutex + c *websocket.Conn +} + +func (w *safeWriter) WriteMessage(messageType int, data []byte) error { + w.mu.Lock() + defer w.mu.Unlock() + return w.c.WriteMessage(messageType, data) +} + +// sendTunnelMessage marshals a WSMessage and writes it as a text frame. +func sendTunnelMessage(w MessageWriter, msg WSMessage) error { + payload, err := json.Marshal(msg) + if err != nil { + return err + } + return w.WriteMessage(websocket.TextMessage, payload) +} diff --git a/ldap_tunnel_test.go b/ldap_tunnel_test.go new file mode 100644 index 0000000..6afca3c --- /dev/null +++ b/ldap_tunnel_test.go @@ -0,0 +1,125 @@ +package main + +import ( + "encoding/base64" + "net" + "path/filepath" + "sync" + "testing" + "time" +) + +// TestLdapTunnelBytePump verifies the agent is a pure byte pump: bytes written +// to the local socket are forwarded up as ldap_tunnel messages, and bytes sent +// back down are written to the socket. No LDAP parsing anywhere. +func TestLdapTunnelBytePump(t *testing.T) { + socketPath := filepath.Join(t.TempDir(), "ldap.sock") + + var mu sync.Mutex + var sent []WSMessage + tunnel := newLdapTunnel(func(msg WSMessage) error { + mu.Lock() + sent = append(sent, msg) + mu.Unlock() + return nil + }) + + stopCh := make(chan struct{}) + defer close(stopCh) + go tunnel.start(socketPath, stopCh) + + // Wait for the socket to exist. + deadline := time.Now().Add(2 * time.Second) + for { + if _, err := net.Dial("unix", socketPath); err == nil { + break + } + if time.Now().After(deadline) { + t.Fatal("socket never became reachable") + } + time.Sleep(10 * time.Millisecond) + } + + conn, err := net.Dial("unix", socketPath) + if err != nil { + t.Fatalf("dial: %v", err) + } + defer conn.Close() + + // Write bytes up to the SSO. + if _, err := conn.Write([]byte("hello-ldap")); err != nil { + t.Fatalf("write: %v", err) + } + + // The tunnel should forward them as a base64 ldap_tunnel message. + var upMsg WSMessage + deadline = time.Now().Add(2 * time.Second) + for { + mu.Lock() + if len(sent) > 0 { + upMsg = sent[0] + mu.Unlock() + break + } + mu.Unlock() + if time.Now().After(deadline) { + t.Fatal("no ldap_tunnel message was sent") + } + time.Sleep(10 * time.Millisecond) + } + + if upMsg.Type != "ldap_tunnel" { + t.Fatalf("expected ldap_tunnel type, got %q", upMsg.Type) + } + connID, _ := upMsg.Payload["conn_id"].(string) + if connID == "" { + t.Fatal("missing conn_id") + } + dataStr, _ := upMsg.Payload["data"].(string) + decoded, err := base64.StdEncoding.DecodeString(dataStr) + if err != nil { + t.Fatalf("bad base64: %v", err) + } + if string(decoded) != "hello-ldap" { + t.Fatalf("expected 'hello-ldap', got %q", decoded) + } + + // Send bytes back down from the SSO; the client should receive them. + tunnel.handleMessage(map[string]interface{}{ + "conn_id": connID, + "data": base64.StdEncoding.EncodeToString([]byte("world-ldap")), + }) + + buf := make([]byte, 32) + conn.SetReadDeadline(time.Now().Add(2 * time.Second)) + n, err := conn.Read(buf) + if err != nil { + t.Fatalf("read back: %v", err) + } + if string(buf[:n]) != "world-ldap" { + t.Fatalf("expected 'world-ldap', got %q", buf[:n]) + } + + // Closing the client should emit a close signal up to the SSO. + conn.Close() + deadline = time.Now().Add(2 * time.Second) + for { + mu.Lock() + closed := false + for _, m := range sent { + if m.Type == "ldap_tunnel" { + if c, _ := m.Payload["close"].(bool); c { + closed = true + } + } + } + mu.Unlock() + if closed { + break + } + if time.Now().After(deadline) { + t.Fatal("no close signal was sent") + } + time.Sleep(10 * time.Millisecond) + } +} diff --git a/secrets.go b/secrets.go new file mode 100644 index 0000000..aa0d804 --- /dev/null +++ b/secrets.go @@ -0,0 +1,159 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "log" + "net/http" + "os" + "path/filepath" + "regexp" + "strings" +) + +// Secrets engine (DESIGN.md §5). The agent renders local templates that embed +// OpenBao secrets, e.g.: +// +// # /etc/theta/templates/db.env.tpl +// DB_USER="{{ bao "secret/data/nodes/node-42/db#username" }}" +// DB_PASS="{{ bao "secret/data/nodes/node-42/db#password" }}" +// +// The agent parses the `{{ bao "path#key" }}` placeholders, fetches the secret +// values from the SSO (which holds the OpenBao access), renders each template to +// its target atomically (0600), and runs the configured reload. The agent never +// holds a Vault token. + +var baoRe = regexp.MustCompile(`\{\{\s*bao\s+"([^"]+)"\s*\}\}`) + +// renderSecrets renders every configured secret template. Called on a +// `render_secrets` command (signed) and on boot. +func renderSecrets(cfg *Config, exec Executor) error { + if len(cfg.Secrets) == 0 { + return nil + } + + // Collect the unique secret paths referenced across all templates. + pathSet := map[string]bool{} + var paths []string + for _, t := range cfg.Secrets { + content, err := os.ReadFile(t.Template) + if err != nil { + log.Printf("Secrets: cannot read template %s: %v", t.Template, err) + continue + } + for _, m := range baoRe.FindAllStringSubmatch(string(content), -1) { + path := refPath(m[1]) + if !pathSet[path] { + pathSet[path] = true + paths = append(paths, path) + } + } + } + if len(paths) == 0 { + return nil + } + + secrets, err := fetchSecrets(cfg, paths) + if err != nil { + return err + } + + for _, t := range cfg.Secrets { + if err := renderOne(t, secrets, exec); err != nil { + log.Printf("Secrets: render %s failed: %v", t.Template, err) + } + } + return nil +} + +func renderOne(t SecretTarget, secrets map[string]map[string]interface{}, exec Executor) error { + content, err := os.ReadFile(t.Template) + if err != nil { + return err + } + + out := baoRe.ReplaceAllStringFunc(string(content), func(match string) string { + ref := baoRe.FindStringSubmatch(match)[1] + path, key := refPath(ref), refKey(ref) + if v, ok := secrets[path][key]; ok { + return fmt.Sprintf("%v", v) + } + return "" + }) + + // Atomic write: temp file in the target's directory, then rename. 0600 — the + // rendered file holds secrets. + tmp, err := os.CreateTemp(filepath.Dir(t.Target), ".theta-secret-*") + if err != nil { + return err + } + tmpName := tmp.Name() + if _, err := io.WriteString(tmp, out); err != nil { + tmp.Close() + os.Remove(tmpName) + return err + } + tmp.Close() + os.Chmod(tmpName, 0600) + if err := os.Rename(tmpName, t.Target); err != nil { + os.Remove(tmpName) + return err + } + + if t.Reload != "" { + if _, err := exec.Execute("sh", "-c", t.Reload); err != nil { + log.Printf("Secrets: reload %q failed: %v", t.Reload, err) + } + } + return nil +} + +// fetchSecrets asks the SSO for the given node-scoped secret paths. +func fetchSecrets(cfg *Config, paths []string) (map[string]map[string]interface{}, error) { + base := strings.Replace(cfg.ServerURL, "wss://", "https://", 1) + base = strings.Replace(base, "ws://", "http://", 1) + url := strings.TrimRight(base, "/") + "/api/v1/agent/secrets" + + body, _ := json.Marshal(map[string]interface{}{"paths": paths}) + req, err := http.NewRequest("POST", url, bytes.NewReader(body)) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Authorization", "Bearer "+cfg.Credential()) + + resp, err := http.DefaultClient.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + return nil, fmt.Errorf("secrets fetch failed: %s", resp.Status) + } + + var out struct { + Secrets map[string]map[string]interface{} `json:"secrets"` + } + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + return nil, err + } + return out.Secrets, nil +} + +// refPath returns the secret path from a `path#key` reference. +func refPath(ref string) string { + if i := strings.Index(ref, "#"); i >= 0 { + return ref[:i] + } + return ref +} + +// refKey returns the key from a `path#key` reference. +func refKey(ref string) string { + if i := strings.Index(ref, "#"); i >= 0 { + return ref[i+1:] + } + return "" +} diff --git a/secrets_test.go b/secrets_test.go new file mode 100644 index 0000000..6ae8c30 --- /dev/null +++ b/secrets_test.go @@ -0,0 +1,113 @@ +package main + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "testing" +) + +// TestRenderSecrets verifies the agent parses `{{ bao "path#key" }}` placeholders, +// fetches the secrets from the SSO, renders the template to its target +// atomically, and runs the reload. +func TestRenderSecrets(t *testing.T) { + dir := t.TempDir() + tpl := filepath.Join(dir, "db.env.tpl") + target := filepath.Join(dir, "db.env") + os.WriteFile(tpl, []byte("DB_USER=\"{{ bao \"secret/data/nodes/n1/db#username\" }}\"\nDB_PASS=\"{{ bao \"secret/data/nodes/n1/db#password\" }}\"\n"), 0600) + + // Fake SSO secrets endpoint. + var gotPaths []string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/api/v1/agent/secrets" { + w.WriteHeader(404) + return + } + var req struct{ Paths []string `json:"paths"` } + json.NewDecoder(r.Body).Decode(&req) + gotPaths = req.Paths + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]interface{}{ + "secrets": map[string]interface{}{ + "secret/data/nodes/n1/db": map[string]interface{}{ + "username": "alice", + "password": "s3cret", + }, + }, + }) + })) + defer srv.Close() + + cfg := &Config{ + ServerURL: srv.URL, + AuthToken: "tok", + Secrets: []SecretTarget{ + {Template: tpl, Target: target, Reload: ""}, + }, + } + + exec := &MockExecutor{} + if err := renderSecrets(cfg, exec); err != nil { + t.Fatalf("renderSecrets: %v", err) + } + + // The requested path should be the one in the template. + if len(gotPaths) != 1 || gotPaths[0] != "secret/data/nodes/n1/db" { + t.Fatalf("expected to request secret/data/nodes/n1/db, got %v", gotPaths) + } + + // The target should be rendered with the secret values. + content, err := os.ReadFile(target) + if err != nil { + t.Fatalf("read target: %v", err) + } + expected := "DB_USER=\"alice\"\nDB_PASS=\"s3cret\"\n" + if string(content) != expected { + t.Fatalf("rendered content mismatch:\n got: %q\nwant: %q", content, expected) + } + + // The target should be 0600 (holds secrets). + info, _ := os.Stat(target) + if info.Mode().Perm() != 0600 { + t.Fatalf("expected 0600, got %o", info.Mode().Perm()) + } +} + +// TestRenderSecretsReload verifies the reload command runs after rendering. +func TestRenderSecretsReload(t *testing.T) { + dir := t.TempDir() + tpl := filepath.Join(dir, "app.tpl") + target := filepath.Join(dir, "app.conf") + os.WriteFile(tpl, []byte("KEY={{ bao \"secret/data/nodes/n1/app#key\" }}"), 0600) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]interface{}{ + "secrets": map[string]interface{}{ + "secret/data/nodes/n1/app": map[string]interface{}{"key": "v"}, + }, + }) + })) + defer srv.Close() + + cfg := &Config{ + ServerURL: srv.URL, + AuthToken: "tok", + Secrets: []SecretTarget{ + {Template: tpl, Target: target, Reload: "systemctl reload app"}, + }, + } + exec := &MockExecutor{} + if err := renderSecrets(cfg, exec); err != nil { + t.Fatalf("renderSecrets: %v", err) + } + if len(exec.ExecutedCommands) != 1 { + t.Fatalf("expected 1 reload command, got %v", exec.ExecutedCommands) + } + cmd := exec.ExecutedCommands[0] + if len(cmd) != 3 || cmd[0] != "sh" || cmd[2] != "systemctl reload app" { + t.Fatalf("expected reload 'sh -c systemctl reload app', got %v", cmd) + } +} diff --git a/telemetry.go b/telemetry.go index 309f22f..a4ffc37 100644 --- a/telemetry.go +++ b/telemetry.go @@ -16,14 +16,15 @@ import ( ) type DiscoveryData struct { - Hostname string `json:"hostname"` - IPs []string `json:"ip_addresses"` - OS string `json:"os"` - Kernel string `json:"kernel"` - CPUModel string `json:"cpu"` - RAMTotalGB float64 `json:"ram_total_gb"` - DiskTotalGB float64 `json:"disk_total_gb"` - Location string `json:"location"` + Hostname string `json:"hostname"` + IPs []string `json:"ip_addresses"` + OS string `json:"os"` + Kernel string `json:"kernel"` + CPUModel string `json:"cpu"` + RAMTotalGB float64 `json:"ram_total_gb"` + DiskTotalGB float64 `json:"disk_total_gb"` + Location string `json:"location"` + Capabilities map[string]interface{} `json:"capabilities"` } type TelemetryData struct { @@ -67,6 +68,16 @@ func CollectDiscoveryData(cfg *Config) DiscoveryData { RAMTotalGB: float64(vm.Total) / (1024 * 1024 * 1024), DiskTotalGB: float64(d.Total) / (1024 * 1024 * 1024), Location: cfg.Location, + Capabilities: map[string]interface{}{ + "telemetry": cfg.Capabilities.Telemetry, + "configure_ldap": cfg.Capabilities.ConfigureLDAP, + "ldap_tunnel": cfg.Capabilities.LdapTunnel, + "secrets": cfg.Capabilities.Secrets, + "iam": cfg.Capabilities.IAM, + "reboot": cfg.Capabilities.Reboot, + "service_control": cfg.Capabilities.ServiceControl, + "arbitrary_bash": cfg.Capabilities.ArbitraryBash, + }, } } diff --git a/theta-agent b/theta-agent index 236f481..966958b 100755 Binary files a/theta-agent and b/theta-agent differ diff --git a/websocket.go b/websocket.go index fafa4f6..5ccf700 100644 --- a/websocket.go +++ b/websocket.go @@ -160,8 +160,26 @@ func connectWebSocket(cm *ConfigManager, exec Executor) { stopCh := make(chan struct{}) + // All outbound writes go through the safe writer: gorilla allows only one + // concurrent writer, and telemetry, heartbeat, the LDAP tunnel and command + // responses all write to the same socket. + sw := &safeWriter{c: c} + + // Local LDAP byte-pump tunnel (DESIGN.md §4). The agent never parses LDAP; + // it forwards raw bytes to the SSO and writes the responses back. + tunnel := newLdapTunnel(func(msg WSMessage) error { + return sendTunnelMessage(sw, msg) + }) + if cfg.Capabilities.LdapTunnel { + socketPath := cfg.LdapSocket + if socketPath == "" { + socketPath = "/run/theta/ldap.sock" + } + go tunnel.start(socketPath, stopCh) + } + // Start telemetry and discovery with stopCh lifecycle control - StartTelemetryLoop(c, cm, exec, stopCh) + StartTelemetryLoop(sw, cm, exec, stopCh) // Heartbeat loop go func() { @@ -174,7 +192,7 @@ func connectWebSocket(cm *ConfigManager, exec Executor) { case <-ticker.C: hb := WSMessage{Type: "heartbeat", Payload: map[string]interface{}{"timestamp": time.Now().Format(time.RFC3339)}} payload, _ := json.Marshal(hb) - if err := c.WriteMessage(websocket.TextMessage, payload); err != nil { + if err := sw.WriteMessage(websocket.TextMessage, payload); err != nil { return } } @@ -208,7 +226,7 @@ func connectWebSocket(cm *ConfigManager, exec Executor) { continue } - handleCommand(cm, msg, c, exec) + handleCommand(cm, msg, sw, exec, tunnel) } // Cleanup on disconnect @@ -226,11 +244,13 @@ func connectWebSocket(cm *ConfigManager, exec Executor) { } } -func handleCommand(cm *ConfigManager, msg WSMessage, c MessageWriter, exec Executor) { +func handleCommand(cm *ConfigManager, msg WSMessage, c MessageWriter, exec Executor, tunnel *ldapTunnel) { cfg := cm.Get() // Don't log the server's fire-and-forget heartbeat ack — it arrives every // 60s and is not a command to act on; logging it is pure per-minute noise. - if msg.Type != "heartbeat_ack" { + // The LDAP tunnel is high-frequency (every chunk of a bind/search), so it is + // not logged either. + if msg.Type != "heartbeat_ack" && msg.Type != "ldap_tunnel" { log.Printf("Received command: %s", msg.Type) } @@ -240,6 +260,12 @@ func handleCommand(cm *ConfigManager, msg WSMessage, c MessageWriter, exec Execu } switch msg.Type { + case "ldap_tunnel": + // SSO→agent direction of the LDAP byte pump: write the response bytes to + // the matching local socket. + if tunnel != nil { + tunnel.handleMessage(msg.Payload) + } case "reload_config": if err := cm.Reload(); err != nil { log.Printf("Reload failed: %v", err) @@ -389,6 +415,46 @@ func handleCommand(cm *ConfigManager, msg WSMessage, c MessageWriter, exec Execu return } sendResponse("ok", "LDAP configuration updated") + case "render_secrets": + if !verifySignature(cfg, msg) { + sendResponse("error", "signature verification failed") + return + } + if !cfg.Capabilities.Secrets { + log.Println("Secrets render rejected: capability disabled in agent.yml") + sendResponse("error", "secrets capability disabled") + return + } + log.Println("Rendering secret templates...") + if err := renderSecrets(cfg, exec); err != nil { + log.Printf("Secrets render failed: %v", err) + sendResponse("error", fmt.Sprintf("secrets render failed: %v", err)) + return + } + sendResponse("ok", "secrets rendered") + case "iam_apply": + if !verifySignature(cfg, msg) { + sendResponse("error", "signature verification failed") + return + } + if !cfg.Capabilities.IAM { + log.Println("IAM apply rejected: capability disabled in agent.yml") + sendResponse("error", "iam capability disabled") + return + } + payload, err := parseIAMPayload(msg.Payload) + if err != nil { + log.Printf("IAM apply: bad payload: %v", err) + sendResponse("error", "invalid IAM payload") + return + } + log.Printf("Applying IAM revision %d for node %s...", payload.Revision, payload.NodeID) + if err := applyIAM(payload, exec); err != nil { + log.Printf("IAM apply failed: %v", err) + sendResponse("error", fmt.Sprintf("iam apply failed: %v", err)) + return + } + sendResponse("ok", "iam applied") case "arbitrary_bash": if !verifySignature(cfg, msg) { sendResponse("error", "signature verification failed") diff --git a/websocket_test.go b/websocket_test.go index a8e53e6..19a91c2 100644 --- a/websocket_test.go +++ b/websocket_test.go @@ -249,7 +249,7 @@ func TestHandleCommand(t *testing.T) { if tc.signed { msg.Payload = sign(t, msg.Payload) } - handleCommand(cm, msg, mockConn, mockExec) + handleCommand(cm, msg, mockConn, mockExec, nil) if tc.expectedNoResponse { if len(mockConn.Messages) != 0 {