aaronsb_obsidian-mcp-plugin/scripts/scorecard-baseline.json
Aaron Bockelie f1cc9d63ba feat(scorecard): scheduled regression gate vs committed baseline (#165)
Standalone weekly watch (+ workflow_dispatch) that FAILS only on a real
portal regression vs scripts/scorecard-baseline.json:
- Health/Review score-ratio downgrade
- automated issue count increase
- a new behaviour/permission ("**Title**:") finding

Never inspects advisory wording — structured deltas only. Scraper drift
→ exit 3, reported distinctly (fix scorecard.mjs, not a regression).
Portal unreachable → exit 0 (inconclusive, never a false fail). Gates
nothing in the release path; no PR/release triggers; no dependency
install (Node builtins only).

`make scorecard-gate` runs it; `make scorecard-baseline` re-snapshots
deliberately (refuses on drift) — a named act, since an accidental
re-baseline silently disarms the gate. Baseline captured for 0.11.25
(Health Excellent 4/4, Review Satisfactory 3/4, 5 issues, 12 findings)
— still shows Dynamic Code Execution; it clears (an improvement, not a
fail) when a post-ADR-201 release is scanned.

Validated: happy path exit 0, simulated regression exit 1 (all 3
classes), drift refusal, idempotent re-baseline.

Closes #165
2026-05-16 23:42:05 -05:00

24 lines
1.6 KiB
JSON

{
"note": "Regenerate intentionally via `make scorecard-baseline` after an ACCEPTED scorecard change. scripts/scorecard-gate.mjs fails on regressions vs this snapshot.",
"capturedFor": "0.11.25",
"capturedAt": "2026-05-17",
"health": "Excellent",
"healthScore": "4/4",
"review": "Satisfactory",
"reviewScore": "3/4",
"issuesFound": 5,
"findings": [
"**Clipboard Access**: Reads or writes the system clipboard. May expose content copied from outside Obsidian.",
"**Direct Filesystem Access**: Uses the Node.js `fs` module to access the filesystem outside of the Obsidian vault API. Can read and write any file on the system.",
"**Dynamic Code Execution**: Executes dynamically generated code via `eval()` or `new Function()`. Prevents full static analysis of plugin behavior.",
"**Vault Enumeration**: Enumerates all files in the vault (`vault.getFiles`, `getMarkdownFiles`, etc.). Gives the plugin access to every file path in the vault.",
"**Vault Read**: Reads individual vault files via the Obsidian API (`vault.read`, `vault.cachedRead`)",
"**Vault Write**: Creates or modifies vault files via the Obsidian API (`vault.modify`, `vault.create`, etc.)",
"Malware scan not available.",
"Network requests scan not available.",
"Obfuscation scan not available.",
"The `main.js` release asset has a verified GitHub artifact attestation.",
"The `styles.css` release asset has a verified GitHub artifact attestation.",
"The release contains additional files: `obsidian-mcp-0.11.25.mcpb`, `obsidian-mcp.mcpb`. Only `main.js`, `manifest.json`, and `styles.css` are supported."
]
}