mirror of
https://github.com/aaronsb/obsidian-mcp-plugin.git
synced 2026-07-22 06:45:14 +00:00
f1cc9d63ba
Standalone weekly watch (+ workflow_dispatch) that FAILS only on a real
portal regression vs scripts/scorecard-baseline.json:
- Health/Review score-ratio downgrade
- automated issue count increase
- a new behaviour/permission ("**Title**:") finding
Never inspects advisory wording — structured deltas only. Scraper drift
→ exit 3, reported distinctly (fix scorecard.mjs, not a regression).
Portal unreachable → exit 0 (inconclusive, never a false fail). Gates
nothing in the release path; no PR/release triggers; no dependency
install (Node builtins only).
`make scorecard-gate` runs it; `make scorecard-baseline` re-snapshots
deliberately (refuses on drift) — a named act, since an accidental
re-baseline silently disarms the gate. Baseline captured for 0.11.25
(Health Excellent 4/4, Review Satisfactory 3/4, 5 issues, 12 findings)
— still shows Dynamic Code Execution; it clears (an improvement, not a
fail) when a post-ADR-201 release is scanned.
Validated: happy path exit 0, simulated regression exit 1 (all 3
classes), drift refusal, idempotent re-baseline.
Closes #165
53 lines
2.2 KiB
JavaScript
53 lines
2.2 KiB
JavaScript
#!/usr/bin/env node
|
|
// Regenerate scripts/scorecard-baseline.json from the live portal (#165).
|
|
//
|
|
// This is a DELIBERATE act: run it only when the scorecard genuinely
|
|
// changed in an ACCEPTED way (e.g. a finding cleared by a shipped fix, or
|
|
// a new finding analysed and accepted), then commit the new baseline. The
|
|
// scorecard-watch workflow fails against whatever this file records, so an
|
|
// accidental re-baseline silently disarms the gate — hence a separate,
|
|
// named command, not an automatic refresh.
|
|
//
|
|
// Refuses to write on scraper drift (the live read is untrustworthy then).
|
|
|
|
import { writeFileSync } from 'node:fs';
|
|
import { spawnSync } from 'node:child_process';
|
|
|
|
const run = spawnSync(process.execPath, ['scripts/scorecard.mjs', '--json'], {
|
|
encoding: 'utf8',
|
|
maxBuffer: 8 * 1024 * 1024,
|
|
});
|
|
const jsonLine = (run.stdout || '')
|
|
.split('\n')
|
|
.reverse()
|
|
.find((l) => l.trim().startsWith('{'));
|
|
if (!jsonLine) {
|
|
console.error('scorecard-baseline: portal unreachable / no JSON — not writing.');
|
|
process.exit(1);
|
|
}
|
|
const j = JSON.parse(jsonLine);
|
|
if (j.integrity === 'DRIFT' || run.status === 2) {
|
|
console.error(
|
|
'scorecard-baseline: SCRAPER DRIFT — refusing to baseline an untrustworthy read. Fix scripts/scorecard.mjs first.',
|
|
);
|
|
process.exit(3);
|
|
}
|
|
|
|
const baseline = {
|
|
note: 'Regenerate intentionally via `make scorecard-baseline` after an ACCEPTED scorecard change. scripts/scorecard-gate.mjs fails on regressions vs this snapshot.',
|
|
capturedFor: j.portal.currentVersion,
|
|
capturedAt: new Date().toISOString().slice(0, 10),
|
|
health: j.portal.health,
|
|
healthScore: j.portal.healthScore,
|
|
review: j.portal.review,
|
|
reviewScore: j.portal.reviewScore,
|
|
issuesFound: Number(j.portal.issuesFound),
|
|
findings: j.findings.slice().sort(),
|
|
};
|
|
writeFileSync(
|
|
'scripts/scorecard-baseline.json',
|
|
JSON.stringify(baseline, null, 2) + '\n',
|
|
);
|
|
console.log(
|
|
`scorecard-baseline: wrote baseline for ${baseline.capturedFor} — Health ${baseline.health} ${baseline.healthScore} | Review ${baseline.review} ${baseline.reviewScore} | ${baseline.issuesFound} issues | ${baseline.findings.length} findings.\nReview the diff and commit scripts/scorecard-baseline.json.`,
|
|
);
|