feat(multi-site): live catalog replication + identical-directory signing key

The shipped join flow (v2.2.0-v2.3.0) was a one-time snapshot: a spoke's
catalog never updated after joining. This adds the two pieces that were
explicitly designed but missing:

- Live replication: a spoke registers its own endpoint with the master
  right after joining (POST /api/site/spokes, Bearer join-key), receiving
  a pushToken. Every successful catalog write on the master now fires a
  fire-and-forget resync ping (utils/site_replicate.js) at every known
  spoke, concurrently -- one unreachable spoke never blocks or delays
  another (wired into the existing write-gate middleware in
  api_directory_admin.js). The spoke's POST /api/site/resync handler
  reuses the already-tested export+import path rather than applying a
  partial diff.

- Identical directories: POST /api/site/export now best-effort includes
  the master's agent-signing key; a spoke adopts it via agent_keys.adopt()
  on both join and every resync, so every site's sso-manager can validly
  sign a command for any agent enrolled anywhere -- the accepted tradeoff
  discussed for this deployment's scale (blast radius for simplicity).

New SiteSpoke model tracks registered spokes (endpoint + pushToken);
registered it in models/index.js (a real bug the e2e test below caught --
SiteSpoke.list() 500'd with "Cannot read properties of null (reading
'adapter')" until the model was added to initORM's model list).

Verified end-to-end against docker-compose.multisite-e2e.yml: mint join
key -> join with selfUrl -> write a NEW resource on master post-join ->
poll the spoke -> it shows up within a few seconds via the resync push,
no manual re-join needed. MULTISITE E2E PASS.

Unit tests: nodejs/tests/site_replicate.test.js (concurrent fan-out, one
failing spoke doesn't block another, empty-registry and list()-throws
edge cases).
This commit is contained in:
2026-08-10 16:34:38 -04:00
parent e5167729a8
commit d27763e556
9 changed files with 451 additions and 57 deletions
+42
View File
@@ -0,0 +1,42 @@
'use strict';
const crypto = require('crypto');
const { Model } = require('@simpleworkjs/orm');
// A spoke known to THIS node while it's acting as master — the registry that
// makes live replication possible. A spoke registers itself here (POST
// /api/site/spokes, authenticated by the same join key it used to join)
// right after adopting the master's export, handing over its own reachable
// endpoint. In return it's issued a `pushToken`: a shared secret the master
// then presents on every future POST <spoke endpoint>/api/site/resync call.
//
// This is a DIFFERENT credential direction than SiteJoinKey: a join key is
// presented TO the master and only ever needs to be verified (so it's stored
// hashed, like a password). pushToken is presented BY the master, repeatedly,
// so it has to be retrievable here -- there is no getting around storing it
// in plaintext on the master, the same way Webhook.secret is (see
// services/webhook_emitter.js) for the same reason (an HMAC/bearer credential
// the sender must keep re-presenting, not a one-time secret only ever
// verified).
class SiteSpoke extends Model {
static generatePushToken() {
return crypto.randomBytes(24).toString('base64url');
}
static fields = {
id: { type: 'uuid', primaryKey: true },
endpoint: { type: 'string', isRequired: true, unique: true },
siteSlug: { type: 'string' },
pushToken: { type: 'string', isRequired: true },
created_on: { type: 'integer' },
last_seen_on: { type: 'integer' }
};
toPublic() {
const data = this.toJSON ? this.toJSON() : { ...this };
delete data.pushToken;
return data;
}
}
module.exports = { SiteSpoke };