# Home-Lab Directory / Inventory — Design Spec Status: **Implemented** (v1.2.1+: model, admin API, UI; v1.3.x: automatic registration from theta-env + ldap-client). §9 adds the planned-consumer readiness review. Owner: wmantly Last updated: 2026-07-23 --- ## 1. Overview & goals The SSO app manages LDAP groups that gate access across the home lab: - `app_*` — applications (Home Assistant, Gitea, Emby, …) - `host_*` — machines (Proxmox nodes, LXC containers, VMs, bare metal) A bootstrap script joins each Debian container to LDAP and grants SSH by group membership. Access control works — but the groups are **bare**. In LDAP a group (`models/group_ldap.js`) only carries `cn`, `description`, `member`, `owner`. There is: - **No metadata** — no URL, IP, FQDN, port, icon, or notes per app/host. - **No relationships** — nothing links a **Proxmox host → the container running on it → the service the container provides → the LDAP group that gates access**. Consequences: - **Users can't see what they have access to or how to reach it.** Today `views/profile.ejs` hardcodes a static "Services" list (Emby, Git, Proxmox…) with fixed URLs that are unrelated to the viewer's actual group membership. - **CI/CD and service discovery have no source of truth.** Pipelines and tooling can't answer "which host runs service X", "what's the IP of `host_ct101`", or "which group grants access to this". ### Goals 1. A machine-readable **discovery API** (build this first) that exposes apps/hosts, their metadata, and the infra graph — for CI/CD, dynamic inventory, and dashboards. 2. Answer, per user, **"what can I access and how do I reach it"** (later dashboard, powered by the same API). 3. Model the **full graph**: Proxmox host ← container/VM ← service/app. ### Non-goals (v1) - No write/admin UI yet (management comes in a later phase). - No auto-provisioning/sync from Proxmox yet (manual population first). - No change to how access is granted — **LDAP remains the access-control truth.** --- ## 2. Architecture Two stores, one join key. ``` ┌──────────────────────────────────────────────┐ │ SSO app (this repo) │ │ │ Identity │ LDAP ──────────────► access-control truth │ & access │ (users, app_*/host_* groups, membership) │ │ │ │ │ │ join on group cn │ │ ▼ │ Inventory │ SQL ──────────────► metadata + graph │ │ (resources, edges, resource↔group links) │ │ │ │ │ ▼ │ │ /api/discovery/* (read-first API) │ └──────────────────────────────────────────────┘ ``` - **LDAP = access truth.** Who is in `app_homeassistant` / `host_ct101` stays in LDAP, unchanged. Read via `Group.list(user.dn)` (`models/group_ldap.js`). - **SQL = inventory truth.** Metadata and the host→container→service graph. - **Join key = group `cn`.** A SQL resource references the LDAP group(s) that gate it by name (`app_*` / `host_*`). "Who can access resource X" is a join: LDAP membership ∩ `resource_group`. ### Tech choice (to confirm) - **PostgreSQL** recommended (JSONB for flexible metadata, real relational edges). **SQLite** is a lighter alternative acceptable for a single-node home lab. - Thin query layer: `node-postgres` (`pg`) directly, or `knex` for migrations + query building. This is the app's **first SQL dependency** (today it uses LDAP + Redis via `model-redis`); the directory is a self-contained module and should not disturb the existing stores. --- ## 3. Data model (full graph) Three tables. Metadata is a JSONB bag so fields can evolve without migrations; common query fields can be promoted to columns later. ### `resource` — a node in the graph | column | type | notes | |--------------|-------------|-------| | `id` | uuid / pk | | | `kind` | enum | `site` \| `host` \| `service` | | `name` | text | display name ("Home Assistant", "ct101") | | `slug` | text unique | url-safe id used by the API | | `description`| text | free text | | `metadata` | jsonb | `{ subType, ip, macAddress, address, vmid, port, externalPort, gitRepo, installPath, systemdService, os, kernel, isProduction, isExternalReachable, isPublic }` | | `created_at` / `updated_at` | timestamptz | | **Parent Enforcement Rules:** - A **Host** MUST have a parent **Site** or **Host**. - A **Service** MUST have a parent **Host**. - An **OAuth Integration** MUST have a parent **Service**. **LDAP Group Auto-Creation:** When a Host or Service is created, the system will automatically create two LDAP groups in the directory (if they do not already exist): - `_access` (for standard user access) - `_admin` (for administrative access) Additional groups can still be linked manually. ### `resource_edge` — directed relationships (the graph) | column | type | notes | |--------------|--------|-------| | `parent_id` | fk → resource | | | `child_id` | fk → resource | | | `relation` | enum | `runs_on` \| `hosts` \| `exposes` \| `depends_on` | Represents site←host←service (`hosts`/`runs_on`) and service→service (`depends_on`). Directed edges (not a single `parent_id` column) so a node can have multiple parents/children and multiple relation types. ### `resource_group` — link a resource to the LDAP group(s) that gate it | column | type | notes | |---------------|--------|-------| | `resource_id` | fk → resource | | | `group_cn` | text | LDAP group name, `app_*` / `host_*` | | `access_level`| enum | `user` \| `admin` \| `owner` | This is the bridge to auth. It lets one resource be gated by several groups (e.g. `app_gitea` for users, `host_ct_gitea` for shell/admin). **Naming:** the `app_*` / `host_*` prefix convention is retained in LDAP; in SQL the distinction is captured explicitly by `resource.kind` + `resource_group`, so the API never has to parse group-name prefixes. ### Example ``` resource: pve1 (proxmox_node) ── hosts ──▶ ct101 (container) ── exposes ──▶ gitea (service) │ resource_group: gitea ↔ app_gitea (user), ct101 ↔ host_ct101 (user), pve1 ↔ host_pve1 (admin) ``` A user in `app_gitea` sees the Gitea service + how to reach it; a user in `host_ct101` additionally sees SSH to the container; `host_pve1` sees the node. --- ## 4. Discovery API (v1 — read-first) JSON. Mounted at `/api/discovery` behind `middleware.auth` (see §5). | method & path | purpose | |---------------|---------| | `GET /api/discovery/resources?kind=&tag=&group=&parent=` | Filtered list of nodes + metadata. | | `GET /api/discovery/resources/:slug` | One node with its edges (parents + children). | | `GET /api/discovery/graph[?root=]` | Whole graph, or the subtree under a root. | | `GET /api/discovery/me` | Only the resources the **caller** is entitled to — LDAP membership (`Group.list(user.dn)`) ∩ `resource_group`. Powers the future dashboard and the "what can I access" question. | ### CI/CD-friendly output (optional formats, same data) - **Ansible dynamic inventory** shape (`?format=ansible`): groups of hosts with `ansible_host`/vars pulled from `metadata`. - **Dashboard** shape (Homepage/Glance): flat list of services with `href`, `icon`, `description`. Write endpoints (POST/PUT/DELETE) are **out of scope for v1**; population is manual (SQL seed / migration) until the admin UI phase. --- ## 5. AuthN / AuthZ - **Interactive users:** existing session auth — `middleware.auth` validating the `auth-token` header (an `AuthToken`, `models/token.js`). No change. - **CI/CD (machine) access:** scripts and external integrations (like jump hosts) will use the existing `ApiToken` system (Personal Access Tokens) passed in the `Authorization: Bearer sso_...` header. The `ApiToken` inherits the exact LDAP group permissions of the user who created it, seamlessly mapping to existing access controls. - **Read visibility (decision to confirm):** either (a) any authenticated user may read all resource metadata and only `/me` is filtered, or (b) list endpoints are themselves filtered to entitlement. Recommend **(a)** for a home lab — simpler, and infra metadata isn't secret — with `/me` as the personalized view. - **Management (later):** gate write/admin endpoints behind a new `app_sso_directory_admin` LDAP group, mirroring the existing `app_sso_oauth_admin` pattern (`routes/oauth_client.js` + `utils/permission.js`). --- ## 6. Integration points with the existing app - **Replace the hardcoded Services list** in `views/profile.ejs` (lines ~82–126) with a render of `GET /api/discovery/me`. - **Reuse the group selector** `app.ui.groupSelect` (`public/js/app.js`) for linking resources ↔ LDAP groups in the future admin UI — no new group-picker needed. - **Join convention:** `resource_group.group_cn` must equal an LDAP group `cn` exactly; the discovery layer never invents groups, it only references existing ones. - **Reuse `utils/permission.byGroup`** for the admin gate in the management phase. --- ## 7. Roadmap 1. **v1 — Discovery API** (this spec's focus): SQL schema + migrations, read models, `/api/discovery/*` endpoints, `ApiToken` for CI/CD. 2. **v2 — "My Access" dashboard**: swap `profile.ejs`'s static list for `/me`. 3. **v3 — Admin CRUD UI**: manage resources/edges/group links (reusing `app.ui` widgets and the `oauth_clients.ejs` card+modal pattern); gated by `app_sso_directory_admin`. 4. **v4 — Sync from Proxmox** (optional): auto-populate nodes/containers/VMs from the Proxmox API so inventory stays current without manual entry. --- ## 8. Open questions 1. **DB engine:** PostgreSQL (recommended) vs SQLite for a single-node lab. 2. **Population:** manual seed vs Proxmox pull for v1 (spec assumes manual). 3. **Metadata mirroring:** should any metadata be written back to the LDAP group `description` so LDAP-only external consumers see it? **Default: no** — keep LDAP for auth, SQL for inventory. 4. **Read-visibility policy:** confirm option (a) vs (b) in §5. 5. **Service token scope:** Currently `ApiToken` shares the creator's full permissions. A future enhancement could scope tokens specifically to the Directory API. --- ## 9. Planned consumers — data-model & API readiness Five consumers the directory data should be able to power. None are being built yet; this section records what each needs, what already exists, and the gaps to close so the model/API never paints us into a corner. The recurring theme: **the graph model itself (Resource / ResourceEdge / ResourceGroup + LDAP groups) is sufficient for all five.** The gaps are (a) one new model (access requests), (b) machine-to-machine auth for the read API, (c) documented metadata conventions instead of new columns, and (d) change detection for the drift/sync consumers. ### 9.1 End-user exploration ("Netflix-style" catalog + request access) A user browses everything that exists — part advertisement, part documentation — sees what they already have, and requests access to the rest. Already there: - `/api/discovery/me` (`getMyAccess`) — the "My Services" half. - `Resource.owner` + `_access` / `_admin` ResourceGroup links — who approves, and which group an approval means joining. - The Notification model — the approval-request delivery mechanism. Gaps: 1. **Catalog projection with metadata privacy.** `/api/discovery/resources` returns full `metadata` to any authenticated user — including the OAuth kind's `client_secret_hash`, and operator notes that may name internal IPs. Needed: a per-kind public projection (name, description, kind, subType, icon, address, hasAccess, requestable) and a private-key convention for the rest (e.g. only `app_sso_directory_admin` sees full metadata). This is a **fix worth doing before any catalog UI exists**. 2. **`AccessRequest` model** — the one genuinely new model: `{id, uid, resourceId, groupCn, status: pending|approved|denied, note, requestedOn, decidedBy, decidedOn}`. Approval = LDAP group add + notify. Endpoints: user POST/GET own; resource owner / directory admin list/approve/deny. 3. **Catalog metadata conventions**: `icon`, `tagline` (card-length blurb), `requestable: false` for resources that shouldn't be advertised. ### 9.2 SSH jump host (`username_-_{hostname-or-ip}@publicHost`) A public jump host parses the target out of the SSH username, checks the user may reach that host, and proxies the connection (WinSCP-friendly: one username string, no interactive menu needed — though an interactive picker on plain `username@` login is the same query). Already there: - Hosts carry `ip` (and `host_` slugs to resolve by name). - Access is already group-based (`_access`), checkable via LDAP alone — the jump host can run entirely off LDAP (SSSD) + one directory query. - User SSH keys are in LDAP (openssh-lpk) — the jump host authenticates the real user without local accounts. Gaps: 1. **Machine auth for the access query.** The jump host must ask "may user X reach host Y" / "list hosts user X may reach" *about another user*. `getMyAccess` only answers for the calling user. Needed: a service-token-authenticated endpoint (`GET /api/discovery/access/:uid[/:slug]`). `ServiceToken` already exists and is even linked to a resource (`resource_id`) — what's missing is an auth middleware that accepts it and a permission rule ("service tokens may read access info, scoped read-only"). 2. **Connection metadata conventions** on hosts: `sshPort` (default 22), optional `fqdn` (when IP is dynamic), optional `jumpVia` edge relation if multi-hop topologies ever appear. 3. Document the username grammar (`{uid}_-_{host-slug-or-ip}`) here so the seed/ldap-client keep host slugs DNS-safe (they already are: slugify strips everything but `[a-z0-9-]`). ### 9.3 Firewall port-forward rules (build / update / drift-test) An automation renders the public firewall's forwarding table from the directory, applies it, and alerts on drift in either direction. Already there: - `metadata.port` / `metadata.externalPort` / `metadata.ip` / `metadata.isExternalReachable` — the core mapping data, already seeded for the stack's own services. Gaps: 1. **Port-mapping convention is too thin for real rules**: no protocol, no multi-port services. Adopt `metadata.portMappings: [{proto: "tcp"|"udp", external: n, internal: n, comment}]` as the authoritative form (`port`/`externalPort` stay as the simple single-mapping case). 2. **Drift detection needs cheap change polling**: an `updated_on` timestamp on resources surfaced in the graph API, or a graph-level etag/hash, so the runner can poll without diffing full payloads. (The ORM already publishes create/update events internally — a future push feed can ride that; polling comes first.) 3. Same **service-token read auth** as 9.2 — automation must not run on a human's session token. ### 9.4 Local DNS / mDNS A DNS (or mDNS advertiser) zone is generated from the directory: hosts get A records from `metadata.ip`, services get CNAMEs/records from their addresses, sites map to zones. Already there: - `host_` + `ip` covers A records; `site_` is a natural zone boundary; service `address` yields names. Gaps: 1. **Name conventions**: `metadata.dnsNames: []` for extra aliases, and a documented rule for which name wins (slug vs `address` hostname). TTL only if someone actually needs per-record TTLs — default is fine. 2. Same **change detection** as 9.3 (poll `updated_on` / etag; push later). 3. Nothing else — this consumer is nearly free once 9.3's conventions land. ### 9.5 Access control for hosts Who may log in to / sudo on which machine, driven by the directory. Already there — this is the original point of the system: - `_access` / `_admin` groups are auto-provisioned per host; ldap-client configures SSSD/PAM against the directory; `sudoRole` and openssh-lpk schemas cover sudo and SSH keys. Gaps: 1. **Close the loop in ldap-client**: joined hosts should set an SSSD access filter (`access_provider = ldap`, filter on `host__access` membership) so directory group membership *is* login permission, not just identity. Today the registration exists but enforcement is host-side convention. 2. **`accessLevel` granularity**: ResourceGroup's `member`/`owner` maps to login/admin today; if finer roles emerge (e.g. `login` vs `sudo` vs `admin`), extend the enum — the join-table shape already supports it. ### 9.6 Consolidated work list (model/API only, no consumers) Ordered by how much they unblock: 1. **Metadata privacy projection** on the read API (blocks 9.1; fixes the `client_secret_hash` exposure regardless of any consumer). 2. **Service-token auth for `/api/discovery/*`** + `access/:uid` endpoint (blocks 9.2, 9.3; ServiceToken model already exists). 3. **`AccessRequest` model + endpoints** (blocks 9.1's request half). 4. **Metadata conventions doc entries** (`sshPort`, `portMappings`, `dnsNames`, `icon`, `tagline`, `requestable`) in `docs/directory.md` — conventions, not schema changes; the json column already holds them. 5. **`updated_on` in graph output / graph etag** (blocks drift/DNS freshness; trivial once surfaced).