Merge pull request #113 from theta42/release/1.19.0

Release 1.19.0: bump submodules, add ldap-test-host fixture

.gitmodules

@@ -8,3 +8,6 @@
 	path = jump-host
 	url = https://github.com/theta42/jump-host.git
 	branch = master
+[submodule "ldap-client"]
+	path = ldap-client
+	url = https://github.com/theta42/ldap-client.git

CHANGELOG.md

@@ -10,6 +10,171 @@ for what changed inside the apps it composes.
 
 ## [Unreleased]
 
+## [1.19.0] - 2026-07-30
+
+### Added
+- **New `ldap-client` submodule + `ldap-test-host` service** (`jump-host` compose profile): a real SSSD + AuthorizedKeysCommand LDAP-joined downstream host for testing jump-host's actual key-injection -> upstream-connect flow end-to-end against this stack's own local LDAP, instead of a container with a manually-dropped public key in `authorized_keys`. Verified live (SSH CLI and WinSCP) through jump-host's `uid_-_target` grammar.
+
+#### ldap-client — [v1.0.0](https://github.com/theta42/ldap-client/releases/tag/v1.0.0) (first tagged release)
+
+##### Added
+- Docker test fixture (`Dockerfile` + `entrypoint.sh`): Ubuntu 22.04 + sssd + sshd, no systemd required.
+
+##### Fixed
+Building that fixture surfaced three real bugs that would break login on any deployment, not just the test fixture:
+- `sssd.conf.mo` used `ldap_bind_dn`/`ldap_bind_pw`, which aren't real SSSD options — corrected to `ldap_default_bind_dn` / `ldap_default_authtok(_type)`.
+- `sssd.conf.mo` had no explicit `services =` list, so SSSD started only its backend, never the nss/pam responders — `getent passwd <ldap-user>` silently failed even with the domain reachable.
+- `ldap-ssh-key.sh`'s `memberof` filter was missing the `cn=` prefix on the group name, so the AuthorizedKeysCommand script always returned zero keys for a correctly-provisioned user — no error, just silently nothing.
+
+#### sso-manager-node — [v1.9.0](https://github.com/theta42/sso-manager-node/releases/tag/v1.9.0)
+
+##### Added
+- Directory modal's Associated LDAP Groups tab now supports full membership management: view, add, and remove members/owners of each associated group directly from the tab.
+- `app.util.revealItem()` (shared `app-base.js`): scrolls a just-added/-edited element into view and flashes its background.
+
+##### Changed
+- Groups page's search/sort bar is now sticky while scrolling.
+- Directory table: Kind/Name/Env/Host merged into a single "Resource" column.
+
+#### proxy — [v1.8.0](https://github.com/theta42/proxy/releases/tag/v1.8.0)
+
+##### Added
+- Users backed by SSO/OIDC login are now marked "External (SSO)" and read-only (password-change hidden client-side, `PUT /password/:username` rejects with 403 server-side). Redis user-backend only.
+
+##### Changed
+- All pages now wrap their content in a standard-width container, matching sso-manager-node instead of rendering full-bleed.
+- Users and Permissions pages converted from bare `<table>`s to the card-grid convention already used on the Groups page.
+
+#### jump-host — [v1.10.2](https://github.com/theta42/jump-host/releases/tag/v1.10.2)
+
+##### Changed
+- Dashboard, Sessions, and Audit pages now match sso-manager-node/proxy's page width.
+- Audit's nav entry is now admin-gated (`groups: ['admin']`).
+
+### Bumped
+- sso-manager-node -> [v1.9.0](https://github.com/theta42/sso-manager-node/releases/tag/v1.9.0)
+- proxy -> [v1.8.0](https://github.com/theta42/proxy/releases/tag/v1.8.0)
+- jump-host -> [v1.10.2](https://github.com/theta42/jump-host/releases/tag/v1.10.2)
+- ldap-client -> [v1.0.0](https://github.com/theta42/ldap-client/releases/tag/v1.0.0) (new submodule)
+
+## [1.18.0] - 2026-07-28
+
+### Changed
+Cross-app API-token self-service UI unification: all 3 apps now share the
+same card-grid list, "+ New Token" modal-based create flow, `app.modal`-based
+secret reveal, and Edit modal (with real created-by/on audit metadata).
+
+#### sso-manager-node — [v1.8.2](https://github.com/theta42/sso-manager-node/releases/tag/v1.8.2), [v1.8.3](https://github.com/theta42/sso-manager-node/releases/tag/v1.8.3)
+
+**v1.8.2**
+
+##### Fixed
+- **Creating a new OAuth integration didn't reliably show the "save this client secret now" reveal modal** — `saveResource()` called `app.modal.close()` immediately before conditionally showing the secret via `app.modal.open()`. `app.modal` is a singleton, and `close()` immediately followed by `open()` collides with Bootstrap's hide-transition guard. An intervening `await loadResources()` made this race unlikely to lose in practice, but not guaranteed to — found while fixing the same, guaranteed-to-lose bug in jump-host and proxy's API-token create flows.
+
+**v1.8.3**
+
+##### Changed
+- **`profile.ejs`'s self-service API-token UI unified onto `app.modal`**, matching the pattern already shipped this round in `directory.ejs`, proxy, and jump-host: the static `#secretModal`/`#editModal` elements are retired in favor of the shared `app.modal` singleton, the always-visible inline create-form card becomes a "+ New Token" button + modal, and badge classes switch from `bg-*` to `text-bg-*`.
+- Checkmark-flash copy feedback (silently broken by FontAwesome's `<i>`→`<svg>` replacement) replaced with toast-based `copyFieldValue`, matching proxy and jump-host.
+
+#### proxy — [v1.7.0](https://github.com/theta42/proxy/releases/tag/v1.7.0)
+
+##### Added
+- **API tokens: "+ New Token" modal button (replacing the always-visible inline create-form card) and a new Edit modal** — continues the cross-app API-token UI unification started in jump-host. The Edit modal's footer shows real created-by/on data; the `PUT /api-token/:id` route already fully supported editing, so no backend change was needed.
+
+##### Fixed
+- **Creating an API token didn't show the "save this secret now" reveal modal** — the create flow called `app.modal.close()` immediately before `app.modal.open()` (to show the secret) in the same tick; since `app.modal` is a singleton, that collided with Bootstrap's hide-transition guard and the reveal modal silently never appeared.
+
+#### jump-host — [v1.10.0](https://github.com/theta42/jump-host/releases/tag/v1.10.0), [v1.10.1](https://github.com/theta42/jump-host/releases/tag/v1.10.1)
+
+**v1.10.0**
+
+##### Added
+- **API-token UI unified with sso-manager-node/proxy**: card grid replacing the bare table, a new Edit modal (footer shows real created-by/on data), and a Description field on both the create and edit flows — the model and API already fully supported all of this, it just wasn't exposed anywhere in the dashboard.
+
+##### Changed
+- `@simpleworkjs/frontend` bumped to `^0.2.6` (this app was still on `^0.2.5`).
+
+**v1.10.1**
+
+##### Fixed
+- **The API-token reveal modal silently didn't show after creating a token** — `submitApiToken()` called `app.modal.close()` immediately before `showToken()`'s `app.modal.open()` in the same tick, colliding with Bootstrap's hide-transition guard on the singleton modal. Same root cause as the OAuth-secret-reveal race fixed in sso-manager-node (v1.8.2) and the create-token race fixed in proxy (v1.7.0).
+
+### Bumped
+- sso-manager-node -> [v1.8.3](https://github.com/theta42/sso-manager-node/releases/tag/v1.8.3)
+- proxy -> [v1.7.0](https://github.com/theta42/proxy/releases/tag/v1.7.0)
+- jump-host -> [v1.10.1](https://github.com/theta42/jump-host/releases/tag/v1.10.1)
+
+## [1.17.0] - 2026-07-28
+
+### Added
+- **proxy's host modal now has a footer (created/updated-by/on metadata) and a linkable `/hosts/{host}` URL**, migrated onto the same shared `app.modal` component as sso-manager-node's resource modal — continuing the entity-modal standardization across the stack.
+
+### Fixed
+- **proxy: the Let's-Encrypt challenge-type/wildcard-matching visibility logic could stop reacting to the hostname field after the first Add/Edit host**, and **the SSO allow-list autocomplete could go empty starting on the second Add/Edit** — both were DOM-rebuild timing bugs in the same class as the resource-modal fixes already shipped.
+- **sso-manager-node: the resource modal's "Associated LDAP Groups" autocomplete went empty after the first Add/Edit** — same DOM-rebuild timing bug, now fixed.
+
+### Bumped
+- sso-manager-node -> [v1.8.1](https://github.com/theta42/sso-manager-node/releases/tag/v1.8.1)
+- proxy -> [v1.6.0](https://github.com/theta42/proxy/releases/tag/v1.6.0)
+
+## [1.16.0] - 2026-07-28
+
+### Added
+- **"Quick Jump" copy-to-clipboard section on the jump-host dashboard** — one-click-copy SSH commands (interactive-picker mode, plus a per-host `uid_-_target` grammar-mode command) instead of having to remember/reconstruct the format by hand.
+
+### Fixed
+- **jump-host audit records for a failed downstream connection only ever said `upstream-unreachable`**, with no way to tell a network-layer failure from an auth failure — the real error (ECONNREFUSED, ETIMEDOUT, an ssh2 auth-failure message, etc.) is now captured and shown as a tooltip on the audit table's fail badge.
+
+### Bumped
+- jump-host -> [v1.9.0](https://github.com/theta42/jump-host/releases/tag/v1.9.0)
+
+## [1.15.0] - 2026-07-28
+
+### Fixed
+- **sso-manager's Directory data (every site/host/service/oauth-client resource and their relationships/LDAP-group associations) had no persistent volume** — `@simpleworkjs/orm` fell back to `./config/inventory.sqlite` (relative to the app's `/app` cwd) whenever `conf.orm` wasn't set, which sits in the container's ephemeral writable layer, not any mounted volume. Every container recreate (`docker compose up --build`, `down`/`up`, an image rebuild) silently wiped the entire Directory Management page. `setup.sh`'s generated `sso-secrets.js` (and the example template) now set `orm: { dialect: 'sqlite', storage: '/data/inventory.sqlite' }`, co-locating it with the already-persisted `sso-data` volume (where Redis lives). **Existing deployments**: this repo doesn't rewrite an operator's existing `config/sso-secrets.js` (re-running `setup.sh` leaves it untouched by design) — add the `orm` block above manually, and copy the container's current `/app/config/inventory.sqlite` to `/data/inventory.sqlite` *before* recreating the container, or the existing Directory data will be lost on the next recreate instead of migrated.
+
+### Bumped
+- sso-manager-node -> [v1.8.0](https://github.com/theta42/sso-manager-node/releases/tag/v1.8.0)
+
+## [1.14.0] - 2026-07-28
+
+### Fixed
+- **jump-host's Redis had zero persistence** (`--save '' --appendonly no`, no data-dir volume) — every container rebuild/recreation (including a `setup.sh` re-run) silently wiped all sessions, in-flight OAuth logins, and any admin-created API token. This is the root cause of the reported "re-running setup.sh breaks OAuth with jump" — the jump-host container gets recreated, and any token or in-flight login vanished with it, while proxy was unaffected because its Redis was already persisted. Now jump-host's Redis persists (AOF + periodic RDB) to `/data`, mounted as a new named volume, `jump-redis-data`. Verified live: minted a PAT, force-recreated the container, confirmed the same PAT still authenticated afterward.
+
+### Changed
+- `docker-compose.yml`: added the `jump-redis-data` volume, mounted at `/data` on the `jump-host` service.
+
+### Bumped
+- jump-host -> [v1.8.1](https://github.com/theta42/jump-host/releases/tag/v1.8.1)
+
+## [1.13.0] - 2026-07-28
+
+### Fixed
+Found via feedback on a fresh install:
+- **jump-host's OAuth client had no parent in the Directory.** `seedDirectory()` only ever linked the proxy's OAuth client; jump-host's own (minted by `provisionJumpHost`) was created but never passed through, so it never got a `ResourceEdge`. Existing deployments self-heal on the next `setup.sh` run.
+- **TUI-mode SSH connections (bare `ssh user@host`) could drop** with "PTY allocation request failed" / "shell request failed" — a session-listener race in jump-host, same class of bug `runGrammar` already had a fix for.
+- **Every form submit briefly showed literal HTML** instead of a loading spinner, across all three apps.
+- **`POST /api/user/` and `PUT /api/user/password` had no success message** — a green notification with nothing in it right after adding a user.
+- **The login page gave no explanation for why the user landed there** when redirected mid-OAuth-flow.
+
+### Changed
+- **Directory: tree view is now the only view; clicking a resource's name opens its detail modal.**
+
+### Bumped
+- sso-manager-node -> [v1.7.0](https://github.com/theta42/sso-manager-node/releases/tag/v1.7.0)
+- proxy -> [v1.5.3](https://github.com/theta42/proxy/releases/tag/v1.5.3)
+- jump-host -> [v1.8.0](https://github.com/theta42/jump-host/releases/tag/v1.8.0)
+
+No `setup.sh` or compose change. Also confirmed (no fix needed): the Let's Encrypt ACME account key persists correctly across container rebuilds — `lua-resty-auto-ssl`'s Redis storage adapter writes through the bundled Redis, which is started with `--appendonly yes` into `/data`, mapped to the persisted `proxy-data` volume. Only an explicit `docker-compose down -v` / volume removal would lose it (which is also what's required, and expected, on a domain change).
+
+## [1.12.0] - 2026-07-28
+
+### Bumped
+- sso-manager-node -> [v1.6.3](https://github.com/theta42/sso-manager-node/releases/tag/v1.6.3) — fixes the root cause of a real "lost user" report: `routes/group.js` never invalidated the User cache on membership changes, so an account added to the `app_sso_service_account` marker group (which hides accounts from the Users page's People tab) could look like it had vanished for up to 5 minutes — and, separately, could be added to that group with no warning at all. Both fixed; see the linked release for detail.
+
+No `setup.sh` or compose change.
+
 ## [1.11.0] - 2026-07-28
 
 ### Added

bootstrap/bootstrap.js

@@ -334,7 +334,7 @@ const HOST_FACTS = {
 	kernel: process.env.STACK_HOST_KERNEL || '',
 };
 
-async function seedDirectory(token, clientId) {
+async function seedDirectory(token, clientId, jumpClientId) {
 	let resources = ((await dirGet(token, 'resources')).results) || [];
 
 	// Create a resource unless its slug (or a legacy alternate from an earlier
@@ -420,9 +420,10 @@ async function seedDirectory(token, clientId) {
 	});
 
 	// Optional SSH jump host service.
+	let jumpSvc = null;
 	if (/^(1|true|yes)$/i.test(process.env.CFG_JUMP_HOST_ENABLED || '')) {
 		const jumpHost = process.env.CFG_JUMP_HOST || (DOMAIN ? `jump.${DOMAIN}` : '');
-		await ensure('service', 'SSH Jump Host', 'jump-host', host.id, {
+		jumpSvc = await ensure('service', 'SSH Jump Host', 'jump-host', host.id, {
 			address: jumpHost ? `https://${jumpHost}` : '',
 			port: 3002,
 			gitRepo: 'https://github.com/theta42/jump-host',
@@ -430,19 +431,21 @@ async function seedDirectory(token, clientId) {
 		});
 	}
 
-	// Link the proxy's OAuth client (Resource-backed since sso-manager 1.3.0)
-	// under its service, if it appears in the directory and isn't linked yet.
-	if (clientId) {
-		const oauthRes = resources.find((r) => r.id === clientId);
-		if (oauthRes) {
-			const edges = ((await dirGet(token, 'edges')).results) || [];
-			const linked = edges.some((e) => e.childId === clientId);
-			if (!linked) {
-				await dirPost(token, 'edges', { parentId: psvc.id, childId: clientId, relation: 'oauth' });
-				log(`  directory: linked OAuth client under 'proxy'`);
-			}
+	// Link an OAuth client (Resource-backed since sso-manager 1.3.0) under its
+	// owning service, if it appears in the directory and isn't linked yet.
+	async function linkOauthClient(id, parent, label) {
+		if (!id || !parent) return;
+		const oauthRes = resources.find((r) => r.id === id);
+		if (!oauthRes) return;
+		const edges = ((await dirGet(token, 'edges')).results) || [];
+		const linked = edges.some((e) => e.childId === id);
+		if (!linked) {
+			await dirPost(token, 'edges', { parentId: parent.id, childId: id, relation: 'oauth' });
+			log(`  directory: linked OAuth client under '${label}'`);
 		}
 	}
+	await linkOauthClient(clientId, psvc, 'proxy');
+	await linkOauthClient(jumpClientId, jumpSvc, 'jump-host');
 }
 
 // Write the OAuth client creds back into /config/proxy-secrets.js so the proxy
@@ -582,10 +585,17 @@ module.exports = {
 	fs.writeFileSync(JUMP_SECRETS, body, { mode: 0o600 });
 }
 
+// Returns the jump host's OAuth client id (so seedDirectory can link it under
+// the SSH Jump Host service), whether or not this run actually wrote a fresh
+// jump-secrets.js -- otherwise re-runs on an already-configured deployment
+// never get a chance to self-heal a missing directory link (see the "no
+// parent" bug this was written for).
 async function provisionJumpHost(token) {
 	if (jumpFileComplete()) {
 		log('Jump host: /config/jump-secrets.js already has API token + OIDC client — keeping.');
-		return;
+		const clients = await listClients(token);
+		const existing = clients.find((c) => c.name === JUMP_CLIENT_NAME);
+		return existing ? existing.client_id : null;
 	}
 	const apiToken = await mintApiToken(token, JUMP_TOKEN_NAME);
 
@@ -607,6 +617,7 @@ async function provisionJumpHost(token) {
 	writeJumpSecrets(apiToken, oidc, localAdminPass);
 	log(`Jump host: wrote /config/jump-secrets.js (API token + OAuth client ${oidc.id}).`);
 	log(`Jump host: local admin 'jumpadmin' password: ${localAdminPass}`);
+	return oidc.id;
 }
 
 (async function main() {
@@ -654,9 +665,10 @@ async function provisionJumpHost(token) {
 
 		// Provision the jump host (mint token + write config) when enabled.
 		// Warn-only — never fail the whole bring-up over the optional service.
+		let jumpClientId = null;
 		if (JUMP_ENABLED) {
 			try {
-				await provisionJumpHost(token);
+				jumpClientId = await provisionJumpHost(token);
 				out('JUMP_HOST_CONFIGURED', '1');
 			} catch (e) {
 				log(`WARNING: jump host provisioning failed (${e.message || e}) — continuing`);
@@ -667,7 +679,7 @@ async function provisionJumpHost(token) {
 		// fails the bootstrap — warn and continue.
 		try {
 			log('Seeding directory resources...');
-			await seedDirectory(token, resolvedClientId);
+			await seedDirectory(token, resolvedClientId, jumpClientId);
 		} catch (e) {
 			log(`WARNING: directory seed failed (${e.message || e}) — continuing`);
 		}

config.example/ldap-test-host.vars.example

@@ -0,0 +1,26 @@
+# ldap-client config for the optional local jump-host test fixture
+# (ldap-test-host service in docker-compose.yml, jump-host compose profile).
+# Copy to ./config/ldap-test-host.vars and fill in the bind password from
+# your own ./config/sso-secrets.js's `serviceAccountPass` (the
+# cn=ldapclient,ou=people,<base> service account bootstrap/bootstrap.js
+# creates specifically for this kind of 3rd-party/container LDAP bind).
+#
+# This is what lets ldap-test-host be a REAL SSSD+AuthorizedKeysCommand-joined
+# downstream host, so jump-host's key-injection -> upstream-connect flow can
+# be exercised end-to-end against something more than a container with a
+# manually-dropped public key in authorized_keys.
+export ldap_host="sso-manager"
+export ldap_base_dn="dc=localtest,dc=me"
+
+export ldap_bind_dn="cn=ldapclient,ou=People,$ldap_base_dn"
+export ldap_bind_password="REPLACE_WITH_serviceAccountPass_FROM_sso-secrets.js"
+
+# sso_url/sso_token deliberately left unset -- register the host + access
+# group manually via the Directory admin API instead (index.sh's optional
+# auto-registration also wants a parent site Resource to exist first).
+# index.sh gates that block on `[[ -v sso_token ]]`, which is true even for
+# an empty string, so leave these genuinely absent, not "".
+
+export ldap_location="jumptest"
+
+ldap_access_groups=( "${ldap_location}_access" "${ldap_location}_host_$(hostname)_access" )

config.example/sso-secrets.js.example

@@ -30,6 +30,16 @@ module.exports = {
 		jwtSecret: 'CHANGE-ME',                 // signs all tokens — keep secret
 		token_lifetime: { access_token: 3600, refresh_token: 2592000 },
 	},
+	// Without this, @simpleworkjs/orm falls back to './config/inventory.sqlite'
+	// (relative to the app's /app cwd) -- inside the container's ephemeral
+	// layer, not any mounted volume, so every Resource/site/host/service/oauth
+	// row (the whole Directory Management page) would be silently wiped on
+	// every container recreate. /data is already a persisted volume (Redis
+	// lives there too), so this just co-locates the sqlite file with it.
+	orm: {
+		dialect: 'sqlite',
+		storage: '/data/inventory.sqlite',
+	},
 
 	// ── Orchestrator-only (ignored by the app; read by setup.sh + bootstrap) ──
 	stack: {

docker-compose.yml

@@ -187,6 +187,32 @@ services:
     volumes:
       - ./config:/config:ro         # jump-secrets.js (written by ensure_config/bootstrap)
       - jump-data:/var/lib/jump-host  # generated host keys persist here
+      - jump-redis-data:/data         # Redis (sessions, OAuth state, API tokens) persists here
+
+  # A real, LDAP-joined (SSSD + AuthorizedKeysCommand) downstream host for
+  # testing jump-host's actual key-injection -> upstream-connect flow --
+  # a container with a manually-dropped public key in authorized_keys never
+  # exercises the LDAP-key-serving path a real production host does. Built
+  # from the theta42/ldap-client submodule -- see ./config/ldap-test-host.vars
+  # for setup notes. Same jump-host profile, so
+  # `docker compose --profile jump-host up` brings up jump-host and a host it
+  # can actually reach together.
+  ldap-test-host:
+    profiles: ["jump-host"]
+    build:
+      context: ./ldap-client
+      dockerfile: Dockerfile
+    container_name: ldap-test-host
+    hostname: ldap-test-host
+    restart: unless-stopped
+    networks: [theta-net]
+    depends_on:
+      sso-manager:
+        condition: service_healthy
+    privileged: false
+    volumes:
+      - ./config/ldap-test-host.vars:/config/ldap.vars:ro
+      - ./config/ldap-ca.crt:/config/ldap-ca.crt:ro
 
 networks:
   theta-net:
@@ -199,4 +225,5 @@ volumes:
   proxy-data:
   proxy-cache:
   proxy-logs:
-  jump-data:
+  jump-data:
+  jump-redis-data:

setup.sh

@@ -292,6 +292,16 @@ module.exports = {
 		jwtSecret: $(js_str "$CFG_JWT_SECRET"),
 		token_lifetime: { access_token: 3600, refresh_token: 2592000 },
 	},
+	// Without this, @simpleworkjs/orm falls back to './config/inventory.sqlite'
+	// relative to the app's /app cwd -- inside the container's ephemeral layer,
+	// not any mounted volume -- so every Resource/site/host/service/oauth row
+	// (the whole Directory Management page) would be silently wiped on every
+	// container recreate. /data is already a persisted volume (Redis lives
+	// there too), so this just co-locates the sqlite file with it.
+	orm: {
+		dialect: 'sqlite',
+		storage: '/data/inventory.sqlite',
+	},
 
 	// ── Orchestrator-only (ignored by the app) ───────────────────────────────
 	stack: {