theta42/sso-manager-node
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(multi-site): coordinated master promotion + a dead-on-arrival authz bug

Browse Source 
Two real bugs, both only surfaced by the live two-container e2e test
(docker-compose.multisite-e2e.yml), not by inspection:

1. POST /site-promote's god_admin check read req.user.groups -- a field
   nothing in the codebase ever populates (Auth.checkToken returns
   User.get(), which has no .groups; every other admin gate resolves
   membership live via permission.byGroup()/Group.list(user.dn), which
   also handles nested-group membership). The check silently evaluated to
   an empty array on every request, so site-promote returned 403 for
   every user, including a real god_admin -- unusable since it shipped in
   v2.0.0. Fixed to use permission.byGroup(), the same pattern used
   elsewhere in this file and in api_site.js.

2. The read-only write-gate middleware (api_directory_admin.js) is
   registered before router.post('/site-promote', ...) later in the same
   file, so on a spoke it 403'd every promotion attempt before the
   handler ever ran -- the one mutating request a spoke must be able to
   make to itself. Exempted /site-promote from the gate.

Added coordinated demotion (MULTI_SITE_SPEC.md §3.2 -- promotion as ONE
action, never a two-step gap with two masters): site-promote now calls
the previous master's new POST /api/site/demote (Bearer the join key it
already holds, handing over a freshly-minted key for the demoted node's
own future use) before flipping itself to master. Best-effort: an
unreachable old master never blocks a god_admin's local promotion (the
WAN-outage scenario is the entire reason this control exists), it's
just reported in the response for manual reconciliation.

e2e test extended to promote the spoke, verify the old master was
actually demoted (isMaster:false, masterUrl pointing at the new master),
and verify writes now succeed on the new master and 403 on the old one.
Full chain verified passing: join -> live replication -> promotion ->
demotion -> write authority follows the promotion.
This commit is contained in:
William Mantly
2026-08-10 16:48:33 -04:00
parent d27763e556
commit 9c604f0258
3 changed files with 146 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/multisite_join_e2e.js
+53 -3
View File
@@ -92,11 +92,24 @@ userPassword: ${hash}
await execFileAsync('ldapadd', ['-x', '-H', `ldap://${ldapHost}:389`, '-D', bindDn, '-w', LDAP_ADMIN_PASS], { input: ldif })
.catch((e) => { if (!/Already exists/.test(e.stderr || '')) throw e; });
for (const group of ['app_sso_admin']) {
// god_admin is needed for site-promote (SUPER_ADMIN_GROUP, utils/permission.js).
for (const group of ['app_sso_admin', 'god_admin']) {
const modLdif = `dn: cn=${group},ou=groups,${baseDn}\nchangetype: modify\nadd: member\nmember: cn=${ADMIN_UID},ou=people,${baseDn}\n`;
await execFileAsync('ldapmodify', ['-x', '-H', `ldap://${ldapHost}:389`, '-D', bindDn, '-w', LDAP_ADMIN_PASS], { input: modLdif })
.catch((e) => { if (!/[Tt]ype or value exists/.test(e.stderr || '')) throw e; });
try {
await execFileAsync('ldapmodify', ['-x', '-H', `ldap://${ldapHost}:389`, '-D', bindDn, '-w', LDAP_ADMIN_PASS], { input: modLdif });
console.log(` (added ${ADMIN_UID} to ${group} on ${ldapHost})`);
} catch (e) {
if (!/[Tt]ype or value exists/.test(e.stderr || '')) {
console.error(` FAILED adding ${ADMIN_UID} to ${group} on ${ldapHost}: ${e.stderr || e.message}`);
throw e;
}
console.log(` (${ADMIN_UID} already in ${group} on ${ldapHost})`);
}
}
const verify = await execFileAsync('ldapsearch', ['-x', '-H', `ldap://${ldapHost}:389`, '-D', bindDn, '-w', LDAP_ADMIN_PASS,
'-b', `cn=god_admin,ou=groups,${baseDn}`, 'member']);
console.log(` god_admin members on ${ldapHost}:\n${verify.stdout}`);
}
async function login(url) {
@@ -234,6 +247,43 @@ async function main() {
const { body: masterCfg } = await api(MASTER_URL, '/api/site/config', { token: masterToken });
if (masterCfg.config.isMaster !== true) fail('master flipped away from isMaster:true unexpectedly');
step('Promoting the spoke to master (coordinated handoff -- must demote the old master too)');
const promoteRes = await api(SPOKE_URL, '/api/directory-admin/site-promote', {
method: 'POST',
token: spokeToken,
body: { selfUrl: 'http://spoke:3001' }
});
if (promoteRes.status !== 200) fail(`promotion failed: ${promoteRes.status} ${JSON.stringify(promoteRes.body)}`);
if (promoteRes.body.handoff !== 'previous master demoted') {
fail(`expected the old master to be demoted as part of promotion, got handoff=${JSON.stringify(promoteRes.body.handoff)}`);
}
step('Verifying the newly-promoted node is master');
const { body: newMasterCfg } = await api(SPOKE_URL, '/api/site/config', { token: spokeToken });
if (newMasterCfg.config.isMaster !== true) fail(`newly-promoted node should be isMaster:true, got ${JSON.stringify(newMasterCfg.config)}`);
step('Verifying the old master was actually demoted to a spoke of the new master');
const { body: oldMasterCfg } = await api(MASTER_URL, '/api/site/config', { token: masterToken });
if (oldMasterCfg.config.isMaster !== false) fail(`old master should be isMaster:false after being demoted, got ${JSON.stringify(oldMasterCfg.config)}`);
if (oldMasterCfg.config.masterUrl !== 'http://spoke:3001') {
fail(`old master's masterUrl should now point at the new master, got ${JSON.stringify(oldMasterCfg.config.masterUrl)}`);
}
step('Verifying the (now-demoted) old master rejects writes, and the new master accepts them');
const oldMasterWrite = await api(MASTER_URL, '/api/directory-admin/resources', {
method: 'POST',
token: masterToken,
body: { name: 'Should Be Rejected Post-Demotion', slug: 'host_e2e_should_reject_2', kind: 'host' }
});
if (oldMasterWrite.status !== 403) fail(`expected 403 writing to the demoted old master, got ${oldMasterWrite.status} ${JSON.stringify(oldMasterWrite.body)}`);
const newMasterWrite = await api(SPOKE_URL, '/api/directory-admin/resources', {
method: 'POST',
token: spokeToken,
body: { name: 'E2E Post-Promotion Host', slug: 'host_e2e_postpromotion', kind: 'host', parentSlug: 'site_e2e' }
});
if (newMasterWrite.status !== 200) fail(`expected the newly-promoted master to accept writes, got ${newMasterWrite.status} ${JSON.stringify(newMasterWrite.body)}`);
if (failed) {
console.error('MULTISITE E2E: one or more checks failed (see above)');
process.exit(1);