mirror of
https://github.com/sotashimozono/obsidian-remote-ssh.git
synced 2026-07-22 17:10:32 +00:00
6476372550
Closes the second half of the Phase C MVP, but with a structurally different cadence than the original M10 design (PR #106 v1). The bench (`plugin/tests/integration/perf.sync.bench.test.ts`) takes ~6 minutes per run — 96 % of the integration job's wall time, all of it in the SSH-tunneled RPC RTT (~41 ms × ~3000 calls). Running it on every PR was wrong on two counts: 1. **Feedback latency**: every PR adds 6 minutes to CI for a comment-only gate. Souta correctly called out that the bench is overkill for the per-PR feedback loop. 2. **Bench purpose**: the bench is a *trend tracker* and an *on-demand perf microscope*. Trends want a stable cadence (nightly), not noise from every PR; a microscope wants to be pulled out when there's a hypothesis to test, not run on every keystroke. So this PR splits the workflow: - **integration.yml** (PR + push to main): runs everything EXCEPT the bench. Drops back to ~30 s wall time. - **bench.yml** (new, nightly cron + workflow_dispatch + push-to-main): runs ONLY the bench. Updates `perf-baseline` branch on nightly + main-push. workflow_dispatch with an optional `pr_number` input posts the diff comment to the named PR — preserves the on-demand "hey what does this branch look like perf-wise" capability without paying the 6-minute tax on every push. What ships: - `plugin/scripts/perf/compare.mjs` (new, pure) Comparison core. Reads two NDJSON datasets, joins by `(span, transport, sizeBytes)`, computes p95 deltas, classifies each bucket as regressed / improved / unchanged / new / removed against per-span tolerance (default 25 %; S.fs widened to 40 % because shared-CI disk variance is high; S.note to 50 % for fsnotify debounce noise). Renders a Markdown table for the PR comment with arrows (🔴↑/🟢↓/−/🆕/➖), bolds regressed rows, appends a pass/fail summary + a footer with the baseline / head SHA pair + run URL. Status-ordered output (regressions first) so reviewers see the bad news without scrolling. - `plugin/scripts/perf/run-compare.mjs` (new, CLI) Picks the latest NDJSON in `plugin/perf-results/`, fetches the baseline at `<workspace>/perf-baseline/baseline.ndjson`, calls `compare()`, posts via `gh pr comment`. **Upserts** the comment by sentinel header so a long-running PR with multiple workflow_dispatch invocations gets one rolling comment instead of a per-trigger spam stack. Exits 0 unless `PERF_GATE=1` AND there's a regression past tolerance. - `plugin/scripts/perf/run-update-baseline.mjs` (new, CLI) Bootstraps the orphan `perf-baseline` branch on first run; on subsequent triggers, copies the latest NDJSON onto it as `baseline.ndjson` + writes a sidecar `baseline.sha`. Skips the commit when the baseline bytes haven't changed (no-op pushes on idle main). Uses `git worktree` to isolate the baseline checkout from the main checkout. - `.github/workflows/bench.yml` (new) Triggers: nightly cron (UTC 19:00 = JST 04:00) + manual `workflow_dispatch` + push to main. Manual triggers can pass an optional `pr_number` input to comment the diff to a PR (the on-demand spot-check path). Permissions: `contents: write` (baseline branch push) + `pull-requests: write` (PR comment). - `plugin/package.json` - `test:integration` — now excludes the bench file (`--exclude **/perf.sync.bench.test.ts`) so PR runs of the integration job stay fast. - `test:integration:bench` (new) — runs ONLY the bench file (used by bench.yml). - `plugin/tests/PerfCompare.test.ts` (new, 19 tests) Pure-module unit coverage for the comparison logic — runs in the fast unit suite via vitest's ESM loader (no .mjs build step needed; ~280 ms). Covered: parseNDJSON: empty, well-formed, malformed (line number in error), missing required fields. compare status semantics: new / removed / unchanged (within ±5 % band) / regressed (past tolerance) / improved (clear drop). Per-span tolerance override (S.fs 40 %). Drops to 'new' when baseline p95 is 0 (can't compute %). ordering: regressions surface first (status-grouped), then improvements, unchanged, new, removed, each lex-sorted. isolation: buckets across (span, transport, sizeBytes) tuples stay separate. formatMarkdown: header / table / pass-summary on no- regressions, ❌ + bolded rows on regressions, "no buckets" message on empty input, gate-enabled hint surface. round-trip: NDJSON in → table out, end-to-end smoke. - `.github/workflows/integration.yml` Unchanged from main — this PR deliberately does NOT add perf- related steps to the integration workflow. integration.yml stays the home of functional integration tests; bench.yml is the home of perf bench + baseline. The split keeps each workflow's purpose discoverable from its name. Architecture summary: PR push → integration.yml (no bench, ~30 s) push to main → integration.yml (no bench) → bench.yml (writes perf-baseline) nightly cron → bench.yml (refreshes perf-baseline) workflow_dispatch→ bench.yml (no baseline write; optional PR comment) Verification: - `npx vitest run` — 37 files / 501 tests green (was 482, +19 new) - `npx tsc --noEmit -p tsconfig.json` — clean - `npm run test:integration` (locally if Docker present) — bench excluded via `--exclude` flag; only the functional integration tests run. - workflow YAML: validated against the existing repo's structural patterns. manifest/package/versions bumped 0.4.46 → 0.4.48 (one ahead of the merged PR #105 / M9 at 0.4.47). Roadmap context: - M0 → M9 (Phase C MVP) ✓ - **M10 (this)** wires the gate but off the PR critical path. - Issue #110 (PERF_GATE flip) needs an update to track "flip on schedule + workflow_dispatch in bench.yml" rather than the old "flip in integration.yml" wording. - Issue #109 (E2E expansion) and #108 (modify case revival) are unaffected; they unlock more bench cells, not bench cadence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| compare.mjs | ||
| run-compare.mjs | ||
| run-update-baseline.mjs | ||