theta42/sso-manager-node
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: real plugin system with loadable instances + OpenBao secrets (v1.17.0)

Browse Source 
Generalize the half-built discovery plugins into a real plugin system: plugin
TYPES (the plugins/<category>/<type>.js modules with manifests) and loadable,
configurable, multi-copy plugin INSTANCES (PluginInstance ORM model) managed
from a dedicated /plugins page and /api/plugins API, with per-instance secrets
in OpenBao at secret/plugins/<id>/conf.

- plugin_registry.js: getTypes/getModule/splitConfig/mask + required-field helpers
- PluginInstance model (Sequelize): id/pluginType/category/name/slug(unique)/
  enabled/cron/config(json, non-secret)/lastRun*; registered in models/index.js
- plugin_secrets.js: read/write/remove/mergeForRun over @simpleworkjs/bao-conf
- scheduler.js: schedules from the DB registry; per-instance stable BullMQ
  JobScheduler ids (plugin:<id>) for load/unload; legacy migration from
  conf.discovery.plugins on first boot (idempotent, empty-table-guarded)
- api_plugins.js (replaces routes/plugins.js): types/list/get/create/update/
  secrets/test/load/unload/run/delete/runs; admin-gated; secrets always masked
- /plugins page (plugins.ejs) + nav; Agents & Scheduler tab removed from
  /directory; /docs/agents aliased to /docs/plugins
- proxmox/unifi/nmap gained manifests (configSchema/validate/run alias)
- tests/plugins.test.js: registry unit + plugin_secrets (mocked bao-conf) +
  PluginInstance model round-trip/unique-slug
- docs (plugins.md, vault.md, _config.yml, API.md) + 1.16.1 -> 1.17.0

Requires theta-suite >= v1.30.1 for the sso-broker secret/plugins/* grant;
fails-soft with a clear error if absent.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
William Mantly
2026-08-01 20:33:57 -04:00
parent 21a56dce50
commit cec0d92c25
23 changed files with 1679 additions and 326 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nodejs/docs/plugins.md
+110 -42
View File
@@ -1,64 +1,132 @@
# Plugins & Scheduler
# Plugins
The SSO Manager includes a flexible background task runner and discovery system. Plugins are defined statically in your deployment configuration (`sso-secrets.js`) and run based on their defined `cron` schedule.
The SSO Manager runs **plugins** as scheduled background tasks. A plugin
**type** is an installed module; a plugin **instance** is a configured, loadable
copy of a type. You can create, edit, load/unload, run, and delete instances
from the **Plugins** page (or the `/api/plugins` API), and you can run several
instances of the same type — e.g. two Proxmox endpoints, each with its own URL
and token on its own schedule.
## Writing Custom Plugins
Per-instance **secrets** are stored in [OpenBao](https://openbao.org/) at
`secret/plugins/<instance-id>/conf`, not in `sso-secrets.js`. The admin UI only
ever shows them masked (`********`); the plugin reads them at run time. This
needs theta-suite ≥ v1.30.1 (which grants the `sso-broker` OpenBao policy
`secret/plugins/*`); re-run `./setup.sh` after upgrading.
You can write custom plugins to discover resources, manage internal state, or run automated scripts. Plugins must be placed in the `plugins/discovery/` directory of the SSO Manager node codebase.
## Plugin types
A plugin file must export a `discover` method.
A plugin type is a module under `nodejs/plugins/<category>/<type>.js`. The
filename basename (without `.js`) is the `type`; the parent directory is the
`category`. The built-ins ship under `plugins/discovery/`:
**Example Plugin (`plugins/discovery/my_plugin.js`):**
- `proxmox` — Proxmox VE (URL + API token)
- `unifi` — UniFi Network controller (URL + username/password)
- `nmap` — nmap OS + port scan (a target range; no credentials)
A module exports a **manifest**:
```javascript
module.exports = {
discover: async function(config) {
// The config object contains any keys passed in sso-secrets.js for this plugin.
// Perform discovery logic, hit external APIs, etc.
const resources = [
{
slug: 'my-custom-resource-1',
name: 'My Resource 1',
kind: 'Host',
metadata: {
ip: '10.0.0.100',
source: 'My Custom Plugin'
}
}
];
// Return the discovered resources array. The discovery reconciler will
// automatically save these to the Network Discovery database.
return resources;
}
// Identity — `type`/`category` default to the file/dir name but can be set
// explicitly. `name`/`description` show up in the UI.
type: 'proxmox',
category: 'discovery',
name: 'Proxmox VE',
description: 'Discover VMs, containers, and nodes from a PVE endpoint.',
// Drives the admin UI form, API validation, and secret masking. Fields with
// `secret: true` are stored in OpenBao; the rest live in the DB row.
configSchema: [
{ key: 'url', label: 'API URL', type: 'url', required: true },
{ key: 'tokenId', label: 'Token ID', type: 'text', required: true },
{ key: 'tokenSecret', label: 'Token Secret', type: 'password', required: true, secret: true }
],
// "Test" button: validate the config (don't do the work). Return
// { ok: true } or { ok: false, error: '...' }. Optional.
validate: async (config) => { … },
// The work. `run` is the generalized contract name; the discovery plugins
// also keep `discover` as an alias for back-compat. For `category:
// 'discovery'`, the scheduler passes the result to the discovery reconciler.
run: async (config) => { return { resources, edges }; },
discover: async (config) => { return { resources, edges }; }
};
```
## Configuring Plugins
`run(config)` receives the merged non-secret config + secret values as one flat
object (e.g. `{ url, tokenId, tokenSecret }`). For a discovery plugin it
returns `{ resources, edges }`; the reconciler upserts them into the resource
graph attributed to the instance's **slug** (the `discovery_sources` name).
In your `sso-secrets.js` file, add your plugin to the `discovery.plugins` object:
### Writing a custom plugin type
Drop a `.js` file under `nodejs/plugins/discovery/` (or a new category directory)
following the manifest above. New types are picked up at boot, so restart the
SSO Manager after adding one. Runtime load/unload is per-**instance** only —
adding a new type still needs a restart.
## The Plugins page
Under **Plugins** (nav, admin-only — `app_sso_admin` / `app_sso_directory_admin`
/ `app_super_admin`):
- **New Plugin** — pick a type, name it, choose a unique slug (the discovery
source name + the URL the resource graph attributes results to), set a cron
schedule, and fill in the config form (secret fields are password inputs).
Creating it schedules it and kicks one immediate run.
- **Edit** — name, cron, and non-secret config.
- **Edit Secrets** (key icon) — password fields, prefilled masked. Leave a
field blank to keep its current value.
- **Test** (vial icon) — runs the plugin's `validate`.
- **Run now** (play icon) — enqueues one immediate run regardless of state.
- **Load / Unload** — enable/disable the schedule without deleting the instance.
- **Delete** — removes the schedule, the OpenBao secret namespace, and the row.
## API
All endpoints are mounted at `/api/plugins`, require an authenticated admin
(`app_sso_admin` / `app_sso_directory_admin` / `app_super_admin`), and return
secret values masked.
| Method + path | Purpose |
|---|---|
| `GET /api/plugins/types` | list installed plugin types + their `configSchema` |
| `GET /api/plugins` | list instances (with masked secrets + last-run state) |
| `GET /api/plugins/:id` | one instance |
| `POST /api/plugins` | create — body `{ pluginType, name, slug, cron, config }` where `config` is a flat object of all field values; secret fields are split into OpenBao |
| `PUT /api/plugins/:id` | update name/cron/enabled + non-secret config |
| `PUT /api/plugins/:id/secrets` | update secret fields (blank = keep) |
| `POST /api/plugins/:id/test` | run `validate` → `{ ok }` or `{ ok:false, error }` |
| `POST /api/plugins/:id/load` | enable + schedule + run now |
| `POST /api/plugins/:id/unload` | unschedule + disable |
| `POST /api/plugins/:id/run` | enqueue one immediate run |
| `DELETE /api/plugins/:id` | unschedule + remove OpenBao secrets + delete row |
| `GET /api/plugins/:id/runs` | `{ lastRunAt, lastStatus, lastError }` |
## Scheduler internals
The scheduler ([BullMQ](https://docs.bullmq.io/) over Redis) gives each instance
a stable JobScheduler id (`plugin:<instanceId>`); load/unload upsert/remove
that one schedule without disturbing the others. A daily `garbage_collect` job
prunes discovery resources not seen in > 7 days.
### Legacy migration
Before this system, plugins were configured statically in `sso-secrets.js`:
```javascript
module.exports = {
// ...
discovery: {
plugins: {
my_plugin: {
enabled: true,
cron: "0 * * * *", // Run every hour
my_custom_key: "my_custom_value" // Passed to the config argument in discover()
}
proxmox: { enabled: true, cron: '0 * * * *', url: '…', tokenId: '…', tokenSecret: '…' }
}
}
// ...
};
```
### Overriding Timing and Enable/Disable
From the **Plugins & Scheduler** tab in the Directory Dashboard, you can override the schedule and enable/disable state for each plugin. These overrides take precedence over `sso-secrets.js` and are stored internally.
## Scheduler Internals
The scheduler uses BullMQ backed by Redis to manage execution. It automatically performs garbage collection on stale network resources (resources not updated in > 7 days) and triggers your plugins at the defined intervals.
On the first boot of SSO Manager ≥ v1.17.0, if the `PluginInstance` table is
empty **and** `conf.discovery.plugins` has entries, one instance per configured
type is seeded automatically (secret fields copied into OpenBao). After that the
table is non-empty and the static config is ignored — manage plugins from the
UI/API instead. The migration is idempotent (guarded by the empty-table check).