mirror of
https://github.com/firstsun-dev/git-files-sync.git
synced 2026-07-22 17:20:30 +00:00
perf(push): batch-commit push-all files + SHA-based diffing
Push-all was up to 3N sequential HTTP calls and N separate commits for N files (getFile check -> pushFile PUT -> optional extra getFile), plus two redundant full remote-tree fetches per run (one discarded in main.ts, one inside GitignoreManager). - Add optional GitServiceInterface.pushBatch. GitHub/Gitea implement it via the git blob->tree->commit->ref Data API, generalizing pushSymlink's existing pattern into shared resolveGitHubStyleBaseTree/commitGitHubStyleTree helpers in git-service-base.ts. GitLab implements it via its native multi-file Commits API (actions array), with a follow-up listFilesDetailed call to recover each file's new blob sha since that endpoint doesn't return them. Symlinks stay on the existing per-file pushSymlink path. - sync-manager.ts's push-all flow now classifies each file by comparing a locally-computed git blob sha (utils/git-blob-sha.ts, already used by the feat-006 status refresh) against a pre-fetched remote tree's per-entry sha, eliminating the per-file getFile call for the common case. Queued files are committed in one grouped pushBatch call, chunked at MAX_BATCH_PUSH_SIZE=200; a failed chunk marks every file in it as failed rather than dropping results silently. Providers without pushBatch fall back to the original sequential path. - main.ts fetches the remote tree once per push/pull-all run and threads it into both GitignoreManager.loadGitignores(tree) and SyncManager.pushAllFiles(files, onProgress, tree), replacing the previously-discarded listFiles() call and gitignore-manager's separate fetch with one shared call. Rename detection and the pull-all path are unchanged (out of scope). Evidence: npx eslint . -> 0 errors; npm run build -> clean; npx vitest run -> 330/330 passed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
f1420f0b44
commit
c28e0ec09a
17 changed files with 753 additions and 110 deletions
|
|
@ -81,6 +81,14 @@
|
||||||
"dependencies": ["feat-009"],
|
"dependencies": ["feat-009"],
|
||||||
"status": "not-started",
|
"status": "not-started",
|
||||||
"evidence": ""
|
"evidence": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "feat-011",
|
||||||
|
"name": "perf(push): batch-commit push-all + SHA-based diffing",
|
||||||
|
"description": "Push-all was up to 3N sequential HTTP calls and N separate commits for N files, plus two redundant full-tree fetches per run. Added optional GitServiceInterface.pushBatch (GitHub/Gitea via blob->tree->commit->ref, generalizing pushSymlink's pattern; GitLab via the native multi-action Commits API + a follow-up tree read for blob shas); sync-manager now classifies each file via a locally-computed git blob sha against one pre-fetched remote tree (no getFile per file) and commits all queued files in one grouped call (chunked at MAX_BATCH_PUSH_SIZE=200), falling back to the old per-file path when a provider has no pushBatch; main.ts/GitignoreManager now share one tree fetch instead of two",
|
||||||
|
"dependencies": [],
|
||||||
|
"status": "done",
|
||||||
|
"evidence": "npx eslint . -> 0 errors; npm run build -> clean; npx vitest run -> 330/330 passed; consolidated onto PR #51"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"_evidenceStyle": "Keep evidence to one line: commit hash + short pointer (e.g. 'Commit abc1234 - added X, tests pass'). Debugging narrative and design discussion belong in the commit message, not this file."
|
"_evidenceStyle": "Keep evidence to one line: commit hash + short pointer (e.g. 'Commit abc1234 - added X, tests pass'). Debugging narrative and design discussion belong in the commit message, not this file."
|
||||||
|
|
|
||||||
24
progress.md
24
progress.md
|
|
@ -12,9 +12,9 @@ Completed work is archived in [archive/](./archive/), one file per calendar mont
|
||||||
|
|
||||||
## Current State
|
## Current State
|
||||||
|
|
||||||
**Last Updated:** 2026-07-14 09:35
|
**Last Updated:** 2026-07-14 10:10
|
||||||
**Session ID:** current
|
**Session ID:** current
|
||||||
**Active Feature:** issue #78 (blog repo, misfiled — actually a git-files-sync bug: delete remote-only file fails) — three root causes found and fixed, awaiting user re-test after rebuild
|
**Active Feature:** feat-011 (perf: batch-commit push-all + SHA-based diffing) — done, lint/build/test all green. Previous active item (issue #78 delete-remote-only-file fix) still awaiting user re-test after rebuild.
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
|
|
@ -33,16 +33,28 @@ Completed work is archived in [archive/](./archive/), one file per calendar mont
|
||||||
- All three root causes committed (`896d77b`, merge `563a28e`, `fa42fea`) and pushed to `claude/fix-directory-symlink-pull-260713`. Test coverage for #2/#3 added after that — pending commit as of this note.
|
- All three root causes committed (`896d77b`, merge `563a28e`, `fa42fea`) and pushed to `claude/fix-directory-symlink-pull-260713`. Test coverage for #2/#3 added after that — pending commit as of this note.
|
||||||
- Not yet done: comment on/reference issue #78 (in the `blog` repo) noting the fix landed in `git-files-sync` instead — flag to user before doing so since it crosses repos.
|
- Not yet done: comment on/reference issue #78 (in the `blog` repo) noting the fix landed in `git-files-sync` instead — flag to user before doing so since it crosses repos.
|
||||||
|
|
||||||
|
- [x] feat-011: perf(push): batch-commit push-all + SHA-based diffing. User asked "現在外掛的 push功能很慢,可能是什麼原因" then approved doing all three identified fixes (a/b/c) via a full plan-mode design pass. Implementation:
|
||||||
|
1. **Batched commit instead of one-commit-per-file**: new optional `GitServiceInterface.pushBatch?(items, branch, message)`. `GitHubService`/`GiteaService` implement it via the git blob→tree→commit→ref Data API (generalizing `pushSymlink`'s existing pattern, factored into shared `resolveGitHubStyleBaseTree`/`commitGitHubStyleTree` helpers in `git-service-base.ts`); `GitLabService` implements it via the native multi-file Commits API (`POST .../repository/commits` with an `actions` array), with one follow-up `listFilesDetailed` call afterward to recover each file's new blob sha (that API doesn't return them). Symlinks stay on the existing per-file `pushSymlink` path, not folded into batches.
|
||||||
|
2. **SHA-based diffing instead of per-file `getFile`**: `sync-manager.ts`'s push-all flow (`processPushBatch`/`classifyPushCandidate`/`classifyAgainstTreeEntry`) now compares a locally-computed git blob sha (`utils/git-blob-sha.ts`'s `gitBlobSha`, already used by feat-006's status refresh) against a pre-fetched remote tree's per-entry sha — no network round trip needed to detect unchanged/new/modified/conflicting files. Rename detection and symlink handling are untouched (still per-file, immediate).
|
||||||
|
3. **Dedup the remote-tree fetch**: `main.ts:runAllFiles` now fetches the tree once (`listFilesDetailed(branch, false)`) and threads it into both `GitignoreManager.loadGitignores(tree)` (new optional param, replacing its own `getRepoGitignores` call when a tree is supplied) and `SyncManager.pushAllFiles(files, onProgress, tree)` — replacing the old discarded `listFiles()` call plus gitignore-manager's separate fetch with a single shared one.
|
||||||
|
- Batches are chunked at `MAX_BATCH_PUSH_SIZE = 200` files per commit call (`git-service-base.ts`); a failed chunk marks every file in that chunk as failed (not silently dropped), earlier successful chunks stay committed.
|
||||||
|
- Providers without `pushBatch` (future Bitbucket) fall back to the original sequential per-file push path, unchanged.
|
||||||
|
- Added `pushBatch` tests to `github-service.test.ts`/`gitea-service.test.ts`/`gitlab-service.test.ts` (happy path, empty-batch short-circuit, base64 encoding, GitLab's create-vs-update action + sha-recovery-miss case), a `MAX_BATCH_PUSH_SIZE` sanity test, new `sync-manager-batch.test.ts` cases (grouped pushBatch call, mixed binary+text batch, whole-chunk-failure, sha-match skips both `getFile` and `pushBatch`), and a `gitignore-manager.test.ts` case for the pre-fetched-tree path. Existing conflict/rename/symlink batch tests rewritten to mock `listFilesDetailed` instead of `getFile` for the equality/conflict check (rename detection's own `getFile` calls are untouched).
|
||||||
|
- Evidence: `npx eslint .` → 0 errors; `npm run build` → clean; `npx vitest run` → 330/330 passed (313 pre-existing + wording/mock updates + 17 new cases).
|
||||||
|
- Pull path (`pullAllFiles`) intentionally untouched — out of scope, still needs full remote content regardless of sha comparison.
|
||||||
|
- Not yet committed as of this note — pending commit onto `claude/fix-directory-symlink-pull-260713`.
|
||||||
|
|
||||||
### What's In Progress
|
### What's In Progress
|
||||||
|
|
||||||
- Nothing else actively in progress.
|
- Nothing else actively in progress.
|
||||||
|
|
||||||
### What's Next
|
### What's Next
|
||||||
|
|
||||||
1. Get user confirmation that delete now works (or a new detailed error message) after they rebuild/reload with the latest push.
|
1. Get user confirmation that delete now works (or a new detailed error message) after they rebuild/reload with the latest push (issue #78).
|
||||||
2. Issue #37 (Bitbucket provider support, feat-010) — large, was deferred until #38 (i18n) landed. Now unblocked.
|
2. Manually verify feat-011 in Obsidian if possible: push-all on a mixed vault should produce one new commit containing only changed/new files.
|
||||||
3. Re-sync against `gh issue list --repo firstsun-dev/git-files-sync --state open`. Remaining genuinely-unstarted issues as of this session: #47 (regex ignore lists), #45 (SonarQube findings), #37 (Bitbucket), #28 (non-engineering: community visibility).
|
3. Issue #37 (Bitbucket provider support, feat-010) — large, was deferred until #38 (i18n) landed. Now unblocked. Its `GitServiceInterface` implementation should simply omit `pushBatch` and use the existing per-file fallback.
|
||||||
4. PR #51 is large (7+ issues' worth of changes now). If the user wants to review/merge it before more work piles on, flag this rather than continuing to add commits indefinitely.
|
4. Re-sync against `gh issue list --repo firstsun-dev/git-files-sync --state open`. Remaining genuinely-unstarted issues as of this session: #47 (regex ignore lists), #45 (SonarQube findings), #37 (Bitbucket), #28 (non-engineering: community visibility).
|
||||||
|
5. PR #51 is large (8+ issues' worth of changes now). If the user wants to review/merge it before more work piles on, flag this rather than continuing to add commits indefinitely.
|
||||||
|
|
||||||
## Blockers / Risks
|
## Blockers / Risks
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
import ignore, { Ignore } from 'ignore';
|
import ignore, { Ignore } from 'ignore';
|
||||||
import { App } from 'obsidian';
|
import { App } from 'obsidian';
|
||||||
import { GitServiceInterface } from '../services/git-service-interface';
|
import { GitServiceInterface, GitTreeEntry } from '../services/git-service-interface';
|
||||||
import { logger } from '../utils/logger';
|
import { logger } from '../utils/logger';
|
||||||
import { readLocalSymlinkTarget } from '../utils/symlink';
|
import { readLocalSymlinkTarget } from '../utils/symlink';
|
||||||
|
|
||||||
|
|
@ -41,11 +41,34 @@ export class GitignoreManager {
|
||||||
/**
|
/**
|
||||||
* Discovers and parses .gitignore files from the local filesystem and remote repository.
|
* Discovers and parses .gitignore files from the local filesystem and remote repository.
|
||||||
* Local files take priority; remote supplements anything not found locally.
|
* Local files take priority; remote supplements anything not found locally.
|
||||||
|
*
|
||||||
|
* @param remoteTree Optional pre-fetched, unfiltered remote tree (e.g. from
|
||||||
|
* `gitService.listFilesDetailed(branch, false)`). When supplied, it's scanned
|
||||||
|
* directly for `.gitignore` paths instead of making another remote fetch via
|
||||||
|
* `getRepoGitignores`. Falls back to that fetch when omitted, so this method
|
||||||
|
* still works standalone (e.g. in tests).
|
||||||
*/
|
*/
|
||||||
async loadGitignores(): Promise<void> {
|
async loadGitignores(remoteTree?: GitTreeEntry[]): Promise<void> {
|
||||||
this.ignoreMap.clear();
|
this.ignoreMap.clear();
|
||||||
|
|
||||||
// 1. Collect all potential gitignore paths
|
const gitignorePaths = await this.collectGitignorePaths(remoteTree);
|
||||||
|
|
||||||
|
// Load content and build ignore instances
|
||||||
|
for (const fullGitignorePath of gitignorePaths) {
|
||||||
|
const dirPath = fullGitignorePath === '.gitignore'
|
||||||
|
? ''
|
||||||
|
: fullGitignorePath.slice(0, -(('.gitignore'.length) + 1));
|
||||||
|
const content = await this.getGitignoreContent(fullGitignorePath);
|
||||||
|
if (content) {
|
||||||
|
this.ignoreMap.set(dirPath, ignore().add(content));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Collects every candidate .gitignore path: repo root, rootPath ancestors,
|
||||||
|
* local vault scan, and the remote listing (from a pre-fetched tree when
|
||||||
|
* supplied, else a dedicated remote fetch). */
|
||||||
|
private async collectGitignorePaths(remoteTree?: GitTreeEntry[]): Promise<Set<string>> {
|
||||||
const gitignorePaths = new Set<string>();
|
const gitignorePaths = new Set<string>();
|
||||||
|
|
||||||
// a. Repo root
|
// a. Repo root
|
||||||
|
|
@ -66,23 +89,26 @@ export class GitignoreManager {
|
||||||
await this.scanLocalGitignores(gitignorePaths);
|
await this.scanLocalGitignores(gitignorePaths);
|
||||||
|
|
||||||
// d. Supplement with remote repo's gitignore listing (filtered to rootPath)
|
// d. Supplement with remote repo's gitignore listing (filtered to rootPath)
|
||||||
|
await this.addRemoteGitignorePaths(gitignorePaths, remoteTree);
|
||||||
|
|
||||||
|
return gitignorePaths;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Adds remote .gitignore paths to `out`: scanned directly from a
|
||||||
|
* pre-fetched tree when supplied, else via a dedicated remote fetch. */
|
||||||
|
private async addRemoteGitignorePaths(out: Set<string>, remoteTree?: GitTreeEntry[]): Promise<void> {
|
||||||
|
if (remoteTree) {
|
||||||
|
for (const entry of remoteTree) {
|
||||||
|
if (entry.path.endsWith('.gitignore')) out.add(entry.path);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
const remotePaths = await this.gitService.getRepoGitignores(this.branch);
|
const remotePaths = await this.gitService.getRepoGitignores(this.branch);
|
||||||
for (const p of remotePaths) gitignorePaths.add(p);
|
for (const p of remotePaths) out.add(p);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
logger.warn('Failed to fetch repo gitignores', e);
|
logger.warn('Failed to fetch repo gitignores', e);
|
||||||
}
|
}
|
||||||
|
|
||||||
// 2. Load content and build ignore instances
|
|
||||||
for (const fullGitignorePath of gitignorePaths) {
|
|
||||||
const dirPath = fullGitignorePath === '.gitignore'
|
|
||||||
? ''
|
|
||||||
: fullGitignorePath.slice(0, -(('.gitignore'.length) + 1));
|
|
||||||
const content = await this.getGitignoreContent(fullGitignorePath);
|
|
||||||
if (content) {
|
|
||||||
this.ignoreMap.set(dirPath, ignore().add(content));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private async scanLocalGitignores(out: Set<string>): Promise<void> {
|
private async scanLocalGitignores(out: Set<string>): Promise<void> {
|
||||||
|
|
|
||||||
|
|
@ -1,14 +1,19 @@
|
||||||
import { TFile, App, Notice } from 'obsidian';
|
import { TFile, App, Notice } from 'obsidian';
|
||||||
import { GitServiceInterface } from '../services/git-service-interface';
|
import { GitServiceInterface, GitTreeEntry } from '../services/git-service-interface';
|
||||||
|
import { MAX_BATCH_PUSH_SIZE } from '../services/git-service-base';
|
||||||
import { GitLabFilesPushSettings, getServiceName, getEffectiveSymlinkHandling } from '../settings';
|
import { GitLabFilesPushSettings, getServiceName, getEffectiveSymlinkHandling } from '../settings';
|
||||||
import { SyncConflictModal } from '../ui/SyncConflictModal';
|
import { SyncConflictModal } from '../ui/SyncConflictModal';
|
||||||
import { logger } from '../utils/logger';
|
import { logger } from '../utils/logger';
|
||||||
import { isBinaryPath, contentsEqual } from '../utils/path';
|
import { isBinaryPath, contentsEqual } from '../utils/path';
|
||||||
import { readLocalSymlinkTarget, createLocalSymlink } from '../utils/symlink';
|
import { readLocalSymlinkTarget, createLocalSymlink } from '../utils/symlink';
|
||||||
|
import { gitBlobSha } from '../utils/git-blob-sha';
|
||||||
|
|
||||||
/** Result of syncing one file within a batch push/pull. */
|
/** Result of syncing one file within a batch push/pull. */
|
||||||
type BatchOutcome = 'done' | 'unchanged' | 'conflict';
|
type BatchOutcome = 'done' | 'unchanged' | 'conflict';
|
||||||
|
|
||||||
|
/** A file classified as needing a push, queued for the grouped batch-commit call. */
|
||||||
|
type ToPushEntry = { path: string; name: string; repoPath: string; content: string | ArrayBuffer; existingSha?: string };
|
||||||
|
|
||||||
export class SyncManager {
|
export class SyncManager {
|
||||||
private readonly app: App;
|
private readonly app: App;
|
||||||
private gitService: GitServiceInterface;
|
private gitService: GitServiceInterface;
|
||||||
|
|
@ -351,17 +356,20 @@ export class SyncManager {
|
||||||
new Notice(`${message}: ${detail}`);
|
new Notice(`${message}: ${detail}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
async pushAllFiles(files: (TFile | string)[], onProgress?: (current: number, total: number, fileName: string) => void): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
async pushAllFiles(
|
||||||
return this.processBatch(files, 'push', onProgress);
|
files: (TFile | string)[],
|
||||||
|
onProgress?: (current: number, total: number, fileName: string) => void,
|
||||||
|
remoteTree?: GitTreeEntry[]
|
||||||
|
): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
||||||
|
return this.processPushBatch(files, onProgress, remoteTree);
|
||||||
}
|
}
|
||||||
|
|
||||||
async pullAllFiles(files: (TFile | string)[], onProgress?: (current: number, total: number, fileName: string) => void): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
async pullAllFiles(files: (TFile | string)[], onProgress?: (current: number, total: number, fileName: string) => void): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
||||||
return this.processBatch(files, 'pull', onProgress);
|
return this.processPullBatch(files, onProgress);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async processBatch(
|
private async processPullBatch(
|
||||||
files: (TFile | string)[],
|
files: (TFile | string)[],
|
||||||
op: 'push' | 'pull',
|
|
||||||
onProgress?: (current: number, total: number, fileName: string) => void
|
onProgress?: (current: number, total: number, fileName: string) => void
|
||||||
): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
||||||
const results = { success: 0, failed: 0, conflicts: 0, errors: [] as Array<{ file: string; error: string }> };
|
const results = { success: 0, failed: 0, conflicts: 0, errors: [] as Array<{ file: string; error: string }> };
|
||||||
|
|
@ -374,25 +382,218 @@ export class SyncManager {
|
||||||
onProgress?.(i + 1, files.length, name);
|
onProgress?.(i + 1, files.length, name);
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const outcome = op === 'push'
|
const outcome = await this.processSingleBatchPull(fileOrPath, path, name, isString);
|
||||||
? await this.processSingleBatchPush(fileOrPath, path, name, isString)
|
|
||||||
: await this.processSingleBatchPull(fileOrPath, path, name, isString);
|
|
||||||
|
|
||||||
if (outcome === 'done') results.success++;
|
if (outcome === 'done') results.success++;
|
||||||
else if (outcome === 'conflict') results.conflicts++;
|
else if (outcome === 'conflict') results.conflicts++;
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
logger.error(`Failed to ${op} ${path}:`, e);
|
logger.error(`Failed to pull ${path}:`, e);
|
||||||
results.failed++;
|
results.failed++;
|
||||||
results.errors.push({ file: path, error: e instanceof Error ? e.message : String(e) });
|
results.errors.push({ file: path, error: e instanceof Error ? e.message : String(e) });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
await this.saveSettings();
|
await this.saveSettings();
|
||||||
this.notifyBatchResult(op, results.success, results.failed, results.conflicts);
|
this.notifyBatchResult('pull', results.success, results.failed, results.conflicts);
|
||||||
|
|
||||||
return results;
|
return results;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async processPushBatch(
|
||||||
|
files: (TFile | string)[],
|
||||||
|
onProgress?: (current: number, total: number, fileName: string) => void,
|
||||||
|
remoteTree?: GitTreeEntry[]
|
||||||
|
): Promise<{ success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }> {
|
||||||
|
const results = { success: 0, failed: 0, conflicts: 0, errors: [] as Array<{ file: string; error: string }> };
|
||||||
|
|
||||||
|
const tree = remoteTree ?? await this.gitService.listFilesDetailed(this.settings.branch, false);
|
||||||
|
const treeByFullPath = new Map<string, GitTreeEntry>(tree.map(e => [e.path, e]));
|
||||||
|
const toPush: ToPushEntry[] = [];
|
||||||
|
|
||||||
|
for (let i = 0; i < files.length; i++) {
|
||||||
|
const fileOrPath = files[i];
|
||||||
|
if (!fileOrPath) continue;
|
||||||
|
|
||||||
|
const { path, name, isString } = this.getFileInfo(fileOrPath);
|
||||||
|
onProgress?.(i + 1, files.length, name);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const outcome = await this.classifyPushCandidate(fileOrPath, path, name, isString, treeByFullPath, toPush);
|
||||||
|
if (outcome === 'done') results.success++;
|
||||||
|
else if (outcome === 'conflict') results.conflicts++;
|
||||||
|
// 'unchanged' and 'queued' don't move any of the counters directly:
|
||||||
|
// 'unchanged' never did, and 'queued' is resolved by commitPushBatch below.
|
||||||
|
} catch (e) {
|
||||||
|
logger.error(`Failed to push ${path}:`, e);
|
||||||
|
results.failed++;
|
||||||
|
results.errors.push({ file: path, error: e instanceof Error ? e.message : String(e) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (toPush.length > 0) {
|
||||||
|
await this.commitPushBatch(toPush, results);
|
||||||
|
}
|
||||||
|
|
||||||
|
await this.saveSettings();
|
||||||
|
this.notifyBatchResult('push', results.success, results.failed, results.conflicts);
|
||||||
|
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Classifies one file for the batch-push flow using a purely local
|
||||||
|
* comparison (git blob sha vs. the pre-fetched remote tree's blob sha) —
|
||||||
|
* no getFile network call. Symlinks and confirmed renames are pushed
|
||||||
|
* immediately (as today) and never queued; everything else is either
|
||||||
|
* resolved immediately ('unchanged'/'conflict') or appended to `toPush`
|
||||||
|
* for the grouped commit.
|
||||||
|
*/
|
||||||
|
private async classifyPushCandidate(
|
||||||
|
fileOrPath: TFile | string,
|
||||||
|
path: string,
|
||||||
|
name: string,
|
||||||
|
isString: boolean,
|
||||||
|
treeByFullPath: Map<string, GitTreeEntry>,
|
||||||
|
toPush: ToPushEntry[]
|
||||||
|
): Promise<BatchOutcome | 'queued'> {
|
||||||
|
if (!await this.checkFileExists(path, isString)) throw new Error('File no longer exists');
|
||||||
|
|
||||||
|
// Symbolic link handling: real → push as a symlink (GitHub), skip → ignore.
|
||||||
|
const symlinkTarget = readLocalSymlinkTarget(this.app, path);
|
||||||
|
if (symlinkTarget !== null && await this.handleSymlinkPush({ path, name }, symlinkTarget, true)) {
|
||||||
|
return getEffectiveSymlinkHandling(this.settings) !== 'skip' ? 'done' : 'unchanged';
|
||||||
|
}
|
||||||
|
|
||||||
|
const content = await this.getFileContent(fileOrPath);
|
||||||
|
const repoPath = this.getNormalizedPath(path);
|
||||||
|
|
||||||
|
// Rename detection
|
||||||
|
if (!isString && fileOrPath instanceof TFile) {
|
||||||
|
const renamedFrom = await this.detectRename(fileOrPath, content);
|
||||||
|
if (renamedFrom) {
|
||||||
|
await this.handleRename(fileOrPath, renamedFrom, content);
|
||||||
|
return 'done';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const treeEntry = treeByFullPath.get(this.getFullPathForTree(repoPath));
|
||||||
|
const outcome = await this.classifyAgainstTreeEntry(path, content, treeEntry);
|
||||||
|
if (outcome !== 'queued') return outcome;
|
||||||
|
|
||||||
|
toPush.push({ path, name, repoPath, content, existingSha: treeEntry?.sha });
|
||||||
|
return 'queued';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decides a non-symlink, non-renamed file's outcome purely from a
|
||||||
|
* pre-fetched tree entry and a locally-computed git blob sha — no network
|
||||||
|
* call. Split out of classifyPushCandidate to keep both under the
|
||||||
|
* cognitive-complexity limit.
|
||||||
|
*/
|
||||||
|
private async classifyAgainstTreeEntry(
|
||||||
|
path: string,
|
||||||
|
content: string | ArrayBuffer,
|
||||||
|
treeEntry: GitTreeEntry | undefined
|
||||||
|
): Promise<BatchOutcome | 'queued'> {
|
||||||
|
// Don't convert a remote symlink into a regular file.
|
||||||
|
if (treeEntry?.symlink) return 'unchanged';
|
||||||
|
|
||||||
|
// Skip if already in sync — compared locally, no network round trip.
|
||||||
|
if (treeEntry?.sha) {
|
||||||
|
const localSha = await gitBlobSha(content);
|
||||||
|
if (localSha === treeEntry.sha) {
|
||||||
|
await this.updateMetadata(path, treeEntry.sha);
|
||||||
|
return 'unchanged';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Same conflict check as the single-file flow: if the remote has moved on
|
||||||
|
// from what we last synced, overwriting it here would silently discard
|
||||||
|
// whatever changed on the remote. Skip it instead of force-pushing so the
|
||||||
|
// batch action can't quietly clobber changes the way a single push would
|
||||||
|
// stop and ask about via SyncConflictModal.
|
||||||
|
const lastSynced = this.settings.syncMetadata[path];
|
||||||
|
if (treeEntry?.sha && lastSynced && treeEntry.sha !== lastSynced.lastSyncedSha) {
|
||||||
|
return 'conflict';
|
||||||
|
}
|
||||||
|
|
||||||
|
return 'queued';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Path relative to rootPath, matching how each git service's getFullPath
|
||||||
|
* would resolve `repoPath` — mirrors that logic locally so pre-fetched tree
|
||||||
|
* entries (always full repo paths) can be looked up without depending on
|
||||||
|
* each service's protected getFullPath.
|
||||||
|
*/
|
||||||
|
private getFullPathForTree(repoPath: string): string {
|
||||||
|
if (repoPath.startsWith('/')) return repoPath.slice(1);
|
||||||
|
const rootPath = this.settings.rootPath;
|
||||||
|
if (!rootPath) return repoPath;
|
||||||
|
const cleanRoot = rootPath.endsWith('/') ? rootPath : `${rootPath}/`;
|
||||||
|
if (repoPath.startsWith(cleanRoot)) return repoPath;
|
||||||
|
return cleanRoot + repoPath;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Commits every queued file in one or more grouped batch-commit calls. */
|
||||||
|
private async commitPushBatch(
|
||||||
|
toPush: ToPushEntry[],
|
||||||
|
results: { success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }
|
||||||
|
): Promise<void> {
|
||||||
|
if (!this.gitService.pushBatch) {
|
||||||
|
await this.pushSequentialFallback(toPush, results);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (let i = 0; i < toPush.length; i += MAX_BATCH_PUSH_SIZE) {
|
||||||
|
await this.commitOneChunk(toPush.slice(i, i + MAX_BATCH_PUSH_SIZE), results);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Provider doesn't support a batch/atomic multi-file commit — fall back to
|
||||||
|
* the same sequential per-file push used by the single-file flow. */
|
||||||
|
private async pushSequentialFallback(
|
||||||
|
toPush: ToPushEntry[],
|
||||||
|
results: { success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }
|
||||||
|
): Promise<void> {
|
||||||
|
for (const f of toPush) {
|
||||||
|
try {
|
||||||
|
await this.performPush({ path: f.path, name: f.name }, f.content, f.existingSha, true);
|
||||||
|
results.success++;
|
||||||
|
} catch (e) {
|
||||||
|
results.failed++;
|
||||||
|
results.errors.push({ file: f.path, error: e instanceof Error ? e.message : String(e) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async commitOneChunk(
|
||||||
|
chunk: ToPushEntry[],
|
||||||
|
results: { success: number; failed: number; conflicts: number; errors: Array<{ file: string; error: string }> }
|
||||||
|
): Promise<void> {
|
||||||
|
try {
|
||||||
|
const commitMessage = `Push ${chunk.length} file(s) from Obsidian`;
|
||||||
|
const batchResults = await this.gitService.pushBatch!(
|
||||||
|
chunk.map(f => ({ path: f.repoPath, content: f.content, existedRemotely: !!f.existingSha })),
|
||||||
|
this.settings.branch,
|
||||||
|
commitMessage
|
||||||
|
);
|
||||||
|
const shaByPath = new Map(batchResults.map(r => [r.path, r.sha]));
|
||||||
|
for (const f of chunk) {
|
||||||
|
const sha = shaByPath.get(f.repoPath);
|
||||||
|
if (sha) await this.updateMetadata(f.path, sha);
|
||||||
|
results.success++;
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Atomic per-provider failure: none of this chunk's files were
|
||||||
|
// actually written, so every file in it is failed, not dropped.
|
||||||
|
const message = e instanceof Error ? e.message : String(e);
|
||||||
|
for (const f of chunk) {
|
||||||
|
results.failed++;
|
||||||
|
results.errors.push({ file: f.path, error: message });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private notifyBatchResult(op: 'push' | 'pull', success: number, failed: number, conflicts: number): void {
|
private notifyBatchResult(op: 'push' | 'pull', success: number, failed: number, conflicts: number): void {
|
||||||
const opName = op === 'push' ? 'Pushed' : 'Pulled';
|
const opName = op === 'push' ? 'Pushed' : 'Pulled';
|
||||||
if (success > 0) {
|
if (success > 0) {
|
||||||
|
|
@ -443,52 +644,6 @@ export class SyncManager {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async processSingleBatchPush(fileOrPath: TFile | string, path: string, name: string, isString: boolean): Promise<BatchOutcome> {
|
|
||||||
if (!await this.checkFileExists(path, isString)) throw new Error('File no longer exists');
|
|
||||||
|
|
||||||
// Symbolic link handling: real → push as a symlink (GitHub), skip → ignore.
|
|
||||||
const symlinkTarget = readLocalSymlinkTarget(this.app, path);
|
|
||||||
if (symlinkTarget !== null && await this.handleSymlinkPush({ path, name }, symlinkTarget, true)) {
|
|
||||||
return getEffectiveSymlinkHandling(this.settings) !== 'skip' ? 'done' : 'unchanged';
|
|
||||||
}
|
|
||||||
|
|
||||||
const content = await this.getFileContent(fileOrPath);
|
|
||||||
const repoPath = this.getNormalizedPath(path);
|
|
||||||
|
|
||||||
// Rename detection
|
|
||||||
if (!isString && fileOrPath instanceof TFile) {
|
|
||||||
const renamedFrom = await this.detectRename(fileOrPath, content);
|
|
||||||
if (renamedFrom) {
|
|
||||||
await this.handleRename(fileOrPath, renamedFrom, content);
|
|
||||||
return 'done';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const remote = await this.gitService.getFile(repoPath, this.settings.branch);
|
|
||||||
|
|
||||||
// Don't convert a remote symlink into a regular file.
|
|
||||||
if (remote.isSymlink) return 'unchanged';
|
|
||||||
|
|
||||||
// Skip if already in sync
|
|
||||||
if (remote.sha && this.contentsEqual(content, remote.content)) {
|
|
||||||
await this.updateMetadata(path, remote.sha);
|
|
||||||
return 'unchanged';
|
|
||||||
}
|
|
||||||
|
|
||||||
// Same conflict check as the single-file flow: if the remote has moved on
|
|
||||||
// from what we last synced, overwriting it here would silently discard
|
|
||||||
// whatever changed on the remote. Skip it instead of force-pushing so the
|
|
||||||
// batch action can't quietly clobber changes the way a single push would
|
|
||||||
// stop and ask about via SyncConflictModal.
|
|
||||||
const lastSynced = this.settings.syncMetadata[path];
|
|
||||||
if (remote.sha && lastSynced && remote.sha !== lastSynced.lastSyncedSha) {
|
|
||||||
return 'conflict';
|
|
||||||
}
|
|
||||||
|
|
||||||
await this.performPush({ path, name }, content, remote.sha || undefined, true);
|
|
||||||
return 'done';
|
|
||||||
}
|
|
||||||
|
|
||||||
private async processSingleBatchPull(fileOrPath: TFile | string, path: string, name: string, isString: boolean): Promise<BatchOutcome> {
|
private async processSingleBatchPull(fileOrPath: TFile | string, path: string, name: string, isString: boolean): Promise<BatchOutcome> {
|
||||||
const repoPath = this.getNormalizedPath(path);
|
const repoPath = this.getNormalizedPath(path);
|
||||||
const remote = await this.gitService.getFile(repoPath, this.settings.branch);
|
const remote = await this.gitService.getFile(repoPath, this.settings.branch);
|
||||||
|
|
|
||||||
16
src/main.ts
16
src/main.ts
|
|
@ -3,7 +3,7 @@ import { DEFAULT_SETTINGS, GitLabFilesPushSettings, GitLabSyncSettingTab, getSer
|
||||||
import { GitLabService } from './services/gitlab-service';
|
import { GitLabService } from './services/gitlab-service';
|
||||||
import { GitHubService } from './services/github-service';
|
import { GitHubService } from './services/github-service';
|
||||||
import { GiteaService } from './services/gitea-service';
|
import { GiteaService } from './services/gitea-service';
|
||||||
import { GitServiceInterface } from './services/git-service-interface';
|
import { GitServiceInterface, GitTreeEntry } from './services/git-service-interface';
|
||||||
import { SyncManager } from './logic/sync-manager';
|
import { SyncManager } from './logic/sync-manager';
|
||||||
import { SyncStatusView, SYNC_STATUS_VIEW_TYPE } from './ui/SyncStatusView';
|
import { SyncStatusView, SYNC_STATUS_VIEW_TYPE } from './ui/SyncStatusView';
|
||||||
import { GitignoreManager } from './logic/gitignore-manager';
|
import { GitignoreManager } from './logic/gitignore-manager';
|
||||||
|
|
@ -201,8 +201,16 @@ export default class GitLabFilesPush extends Plugin {
|
||||||
const startPath = this.settings.vaultFolder || '';
|
const startPath = this.settings.vaultFolder || '';
|
||||||
const allPaths = await this.listAllFilesFromAdapter(startPath);
|
const allPaths = await this.listAllFilesFromAdapter(startPath);
|
||||||
|
|
||||||
await this.gitService.listFiles(this.settings.branch);
|
// Fetch the remote tree once and share it with both gitignore discovery
|
||||||
await this.gitignoreManager.loadGitignores();
|
// and (for push) the SHA-based diff, instead of each fetching it separately.
|
||||||
|
let tree: GitTreeEntry[] | undefined;
|
||||||
|
try {
|
||||||
|
tree = await this.gitService.listFilesDetailed(this.settings.branch, false);
|
||||||
|
} catch (e) {
|
||||||
|
logger.warn('Failed to fetch remote tree; falling back to per-call fetches', e);
|
||||||
|
}
|
||||||
|
|
||||||
|
await this.gitignoreManager.loadGitignores(tree);
|
||||||
const files = allPaths.filter(p => !this.gitignoreManager.isIgnored(this.getNormalizedPath(p)));
|
const files = allPaths.filter(p => !this.gitignoreManager.isIgnored(this.getNormalizedPath(p)));
|
||||||
|
|
||||||
if (files.length === 0) {
|
if (files.length === 0) {
|
||||||
|
|
@ -224,7 +232,7 @@ export default class GitLabFilesPush extends Plugin {
|
||||||
const results = op === 'push'
|
const results = op === 'push'
|
||||||
? await this.sync.pushAllFiles(files, (current, total, fileName) => {
|
? await this.sync.pushAllFiles(files, (current, total, fileName) => {
|
||||||
progressNotice.setMessage(t('main.progress.step', { verb: t('main.verb.pushing'), current, total, fileName }));
|
progressNotice.setMessage(t('main.progress.step', { verb: t('main.verb.pushing'), current, total, fileName }));
|
||||||
})
|
}, tree)
|
||||||
: await this.sync.pullAllFiles(files, (current, total, fileName) => {
|
: await this.sync.pullAllFiles(files, (current, total, fileName) => {
|
||||||
progressNotice.setMessage(t('main.progress.step', { verb: t('main.verb.pulling'), current, total, fileName }));
|
progressNotice.setMessage(t('main.progress.step', { verb: t('main.verb.pulling'), current, total, fileName }));
|
||||||
});
|
});
|
||||||
|
|
|
||||||
|
|
@ -55,6 +55,10 @@ export interface ConnectionTestResult {
|
||||||
error?: string;
|
error?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Max files per single batch-commit call. Guards against oversized request
|
||||||
|
* bodies / provider payload limits when a vault has thousands of files. */
|
||||||
|
export const MAX_BATCH_PUSH_SIZE = 200;
|
||||||
|
|
||||||
export abstract class BaseGitService {
|
export abstract class BaseGitService {
|
||||||
protected token: string = '';
|
protected token: string = '';
|
||||||
protected rootPath: string = '';
|
protected rootPath: string = '';
|
||||||
|
|
@ -258,6 +262,61 @@ export abstract class BaseGitService {
|
||||||
return e instanceof Error ? e : new Error(String(e));
|
return e instanceof Error ? e : new Error(String(e));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The base URL for a GitHub-shaped Git Data API (e.g.
|
||||||
|
* `https://api.github.com/repos/{owner}/{repo}` or
|
||||||
|
* `{baseUrl}/api/v1/repos/{owner}/{repo}` for Gitea). Only meaningful for
|
||||||
|
* providers that implement pushBatch/pushSymlink via this API shape.
|
||||||
|
*/
|
||||||
|
protected getGitDataApiBase(): string {
|
||||||
|
throw new Error('getGitDataApiBase is not implemented for this provider');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolves a branch to its latest commit sha and that commit's base tree
|
||||||
|
* sha, via GitHub's `git/ref/heads/{branch}` endpoint. Gitea's older
|
||||||
|
* versions require a different branch-resolution endpoint, so it provides
|
||||||
|
* its own override rather than using this helper.
|
||||||
|
*/
|
||||||
|
protected async resolveGitHubStyleBaseTree(branch: string): Promise<{ latestCommitSha: string; baseTreeSha: string }> {
|
||||||
|
const base = this.getGitDataApiBase();
|
||||||
|
const refResp = await this.safeRequest(`${base}/git/ref/heads/${branch}`, 'GET');
|
||||||
|
const latestCommitSha = this.parseJson<{ object: { sha: string } }>(refResp).object.sha;
|
||||||
|
|
||||||
|
const commitResp = await this.safeRequest(`${base}/git/commits/${latestCommitSha}`, 'GET');
|
||||||
|
const baseTreeSha = this.parseJson<{ tree: { sha: string } }>(commitResp).tree.sha;
|
||||||
|
|
||||||
|
return { latestCommitSha, baseTreeSha };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Commits N tree items (already-created blobs) in one shot: builds a new
|
||||||
|
* tree on top of baseTreeSha, commits it, and moves the branch ref to point
|
||||||
|
* at the new commit. Shared by GitHub/Gitea's pushSymlink and pushBatch.
|
||||||
|
*/
|
||||||
|
protected async commitGitHubStyleTree(
|
||||||
|
base: string,
|
||||||
|
branch: string,
|
||||||
|
baseTreeSha: string,
|
||||||
|
latestCommitSha: string,
|
||||||
|
treeItems: Array<{ path: string; mode: string; type: 'blob'; sha: string }>,
|
||||||
|
message: string
|
||||||
|
): Promise<string> {
|
||||||
|
const treeResp = await this.safeRequest(`${base}/git/trees`, 'POST', { base_tree: baseTreeSha, tree: treeItems });
|
||||||
|
const newTreeSha = this.parseJson<{ sha: string }>(treeResp).sha;
|
||||||
|
|
||||||
|
const newCommitResp = await this.safeRequest(`${base}/git/commits`, 'POST', {
|
||||||
|
message,
|
||||||
|
tree: newTreeSha,
|
||||||
|
parents: [latestCommitSha],
|
||||||
|
});
|
||||||
|
const newCommitSha = this.parseJson<{ sha: string }>(newCommitResp).sha;
|
||||||
|
|
||||||
|
await this.safeRequest(`${base}/git/refs/heads/${branch}`, 'PATCH', { sha: newCommitSha });
|
||||||
|
|
||||||
|
return newCommitSha;
|
||||||
|
}
|
||||||
|
|
||||||
async getRepoGitignores(branch: string): Promise<string[]> {
|
async getRepoGitignores(branch: string): Promise<string[]> {
|
||||||
try {
|
try {
|
||||||
const allFiles = await this.listFiles(branch, false); // Fetch ALL files to find gitignores
|
const allFiles = await this.listFiles(branch, false); // Fetch ALL files to find gitignores
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,26 @@ export interface GitTreeEntry {
|
||||||
sha?: string;
|
sha?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** One file's content to include in a batched multi-file commit. */
|
||||||
|
export interface BatchPushItem {
|
||||||
|
/** Path relative to rootPath, same shape pushFile's `path` param takes. */
|
||||||
|
path: string;
|
||||||
|
content: string | ArrayBuffer;
|
||||||
|
/**
|
||||||
|
* Whether this path already existed on the remote before this push, per the
|
||||||
|
* caller's pre-fetched tree. Only GitLab's Commits API needs this (to choose
|
||||||
|
* action 'create' vs 'update'); GitHub/Gitea's tree-based commit ignores it.
|
||||||
|
*/
|
||||||
|
existedRemotely?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Result for one file after a batch push completes. */
|
||||||
|
export interface BatchPushResult {
|
||||||
|
path: string;
|
||||||
|
/** New blob sha, when the provider can report it directly. */
|
||||||
|
sha?: string;
|
||||||
|
}
|
||||||
|
|
||||||
export interface GitServiceInterface {
|
export interface GitServiceInterface {
|
||||||
updateConfig(...args: unknown[]): void;
|
updateConfig(...args: unknown[]): void;
|
||||||
getFile(path: string, branch: string): Promise<GitFile>;
|
getFile(path: string, branch: string): Promise<GitFile>;
|
||||||
|
|
@ -37,6 +57,14 @@ export interface GitServiceInterface {
|
||||||
* implement it; callers must fall back to pushFile when it's absent.
|
* implement it; callers must fall back to pushFile when it's absent.
|
||||||
*/
|
*/
|
||||||
pushSymlink?(path: string, target: string, branch: string, commitMessage: string): Promise<{ path: string, sha?: string }>;
|
pushSymlink?(path: string, target: string, branch: string, commitMessage: string): Promise<{ path: string, sha?: string }>;
|
||||||
|
/**
|
||||||
|
* Push many files in a single commit. Optional: only providers with a way to
|
||||||
|
* write multiple files atomically implement it; callers must fall back to
|
||||||
|
* sequential pushFile calls when it's absent (mirrors pushSymlink?). Must be
|
||||||
|
* atomic: on failure it throws rather than returning partial results, so the
|
||||||
|
* caller can mark every item in the attempted batch as failed.
|
||||||
|
*/
|
||||||
|
pushBatch?(items: BatchPushItem[], branch: string, commitMessage: string): Promise<BatchPushResult[]>;
|
||||||
deleteFile(path: string, branch: string, commitMessage: string): Promise<void>;
|
deleteFile(path: string, branch: string, commitMessage: string): Promise<void>;
|
||||||
getRepoGitignores(branch: string): Promise<string[]>;
|
getRepoGitignores(branch: string): Promise<string[]>;
|
||||||
/**
|
/**
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
import { GitServiceInterface, GitTreeEntry } from './git-service-interface';
|
import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult } from './git-service-interface';
|
||||||
import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE } from './git-service-base';
|
import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE } from './git-service-base';
|
||||||
import { logger } from '../utils/logger';
|
import { logger } from '../utils/logger';
|
||||||
|
|
||||||
|
|
@ -25,6 +25,25 @@ export class GiteaService extends BaseGitService implements GitServiceInterface
|
||||||
return `${this.baseUrl}/api/v1/repos/${this.owner}/${this.repo}/contents/${encodedPath}`;
|
return `${this.baseUrl}/api/v1/repos/${this.owner}/${this.repo}/contents/${encodedPath}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
protected getGitDataApiBase(): string {
|
||||||
|
return `${this.baseUrl}/api/v1/repos/${this.owner}/${this.repo}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Gitea's git/commits/{sha} endpoint needs a resolved commit sha, not a
|
||||||
|
// branch ref name, and older Gitea versions don't expose GitHub's
|
||||||
|
// git/ref/heads/{branch} endpoint at all — resolve via /branches/{branch}
|
||||||
|
// instead, same as listFilesDetailed already does.
|
||||||
|
private async resolveBaseTree(branch: string): Promise<{ latestCommitSha: string; baseTreeSha: string }> {
|
||||||
|
const branchUrl = `${this.baseUrl}/api/v1/repos/${this.owner}/${this.repo}/branches/${branch}`;
|
||||||
|
const branchResp = await this.safeRequest(branchUrl, 'GET');
|
||||||
|
const latestCommitSha = this.parseJson<{ commit: { id: string } }>(branchResp).commit.id;
|
||||||
|
|
||||||
|
const commitResp = await this.safeRequest(`${this.getGitDataApiBase()}/git/commits/${latestCommitSha}`, 'GET');
|
||||||
|
const baseTreeSha = this.parseJson<{ tree: { sha: string } }>(commitResp).tree.sha;
|
||||||
|
|
||||||
|
return { latestCommitSha, baseTreeSha };
|
||||||
|
}
|
||||||
|
|
||||||
async getFile(path: string, branch: string): Promise<GitFile> {
|
async getFile(path: string, branch: string): Promise<GitFile> {
|
||||||
try {
|
try {
|
||||||
const url = `${this.getApiUrl(path)}?ref=${branch}`;
|
const url = `${this.getApiUrl(path)}?ref=${branch}`;
|
||||||
|
|
@ -56,6 +75,32 @@ export class GiteaService extends BaseGitService implements GitServiceInterface
|
||||||
return { path: data.content.path, sha: data.content.sha };
|
return { path: data.content.path, sha: data.content.sha };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async pushBatch(items: BatchPushItem[], branch: string, message: string): Promise<BatchPushResult[]> {
|
||||||
|
if (items.length === 0) return [];
|
||||||
|
const base = this.getGitDataApiBase();
|
||||||
|
const { latestCommitSha, baseTreeSha } = await this.resolveBaseTree(branch);
|
||||||
|
|
||||||
|
const blobShas: string[] = [];
|
||||||
|
for (const item of items) {
|
||||||
|
const blobResp = await this.safeRequest(`${base}/git/blobs`, 'POST', {
|
||||||
|
content: this.encodeContent(item.content),
|
||||||
|
encoding: 'base64',
|
||||||
|
});
|
||||||
|
blobShas.push(this.parseJson<{ sha: string }>(blobResp).sha);
|
||||||
|
}
|
||||||
|
|
||||||
|
const treeItems = items.map((item, i) => ({
|
||||||
|
path: this.getFullPath(item.path),
|
||||||
|
mode: '100644',
|
||||||
|
type: 'blob' as const,
|
||||||
|
sha: blobShas[i] as string,
|
||||||
|
}));
|
||||||
|
|
||||||
|
await this.commitGitHubStyleTree(base, branch, baseTreeSha, latestCommitSha, treeItems, message);
|
||||||
|
|
||||||
|
return items.map((item, i) => ({ path: item.path, sha: blobShas[i] }));
|
||||||
|
}
|
||||||
|
|
||||||
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
||||||
// Resolve branch name to commit SHA first for compatibility with all Gitea versions,
|
// Resolve branch name to commit SHA first for compatibility with all Gitea versions,
|
||||||
// since the git/trees endpoint requires a SHA (not a ref name) on older instances.
|
// since the git/trees endpoint requires a SHA (not a ref name) on older instances.
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
import { GitServiceInterface, GitTreeEntry } from './git-service-interface';
|
import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult } from './git-service-interface';
|
||||||
import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE } from './git-service-base';
|
import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE } from './git-service-base';
|
||||||
import { logger } from '../utils/logger';
|
import { logger } from '../utils/logger';
|
||||||
|
|
||||||
|
|
@ -23,6 +23,10 @@ export class GitHubService extends BaseGitService implements GitServiceInterface
|
||||||
return `https://api.github.com/repos/${this.owner}/${this.repo}/contents/${encodedPath}`;
|
return `https://api.github.com/repos/${this.owner}/${this.repo}/contents/${encodedPath}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
protected getGitDataApiBase(): string {
|
||||||
|
return `https://api.github.com/repos/${this.owner}/${this.repo}`;
|
||||||
|
}
|
||||||
|
|
||||||
async getFile(path: string, branch: string): Promise<GitFile> {
|
async getFile(path: string, branch: string): Promise<GitFile> {
|
||||||
try {
|
try {
|
||||||
const url = `${this.getApiUrl(path)}?ref=${branch}`;
|
const url = `${this.getApiUrl(path)}?ref=${branch}`;
|
||||||
|
|
@ -67,35 +71,48 @@ export class GitHubService extends BaseGitService implements GitServiceInterface
|
||||||
// (mode 120000) must be committed through the lower-level Git Data API:
|
// (mode 120000) must be committed through the lower-level Git Data API:
|
||||||
// blob -> tree (with the symlink mode) -> commit -> move the branch ref.
|
// blob -> tree (with the symlink mode) -> commit -> move the branch ref.
|
||||||
const fullPath = this.getFullPath(path);
|
const fullPath = this.getFullPath(path);
|
||||||
const base = `https://api.github.com/repos/${this.owner}/${this.repo}`;
|
const base = this.getGitDataApiBase();
|
||||||
|
|
||||||
const refResp = await this.safeRequest(`${base}/git/ref/heads/${branch}`, 'GET');
|
const { latestCommitSha, baseTreeSha } = await this.resolveGitHubStyleBaseTree(branch);
|
||||||
const latestCommitSha = this.parseJson<{ object: { sha: string } }>(refResp).object.sha;
|
|
||||||
|
|
||||||
const commitResp = await this.safeRequest(`${base}/git/commits/${latestCommitSha}`, 'GET');
|
|
||||||
const baseTreeSha = this.parseJson<{ tree: { sha: string } }>(commitResp).tree.sha;
|
|
||||||
|
|
||||||
const blobResp = await this.safeRequest(`${base}/git/blobs`, 'POST', { content: target, encoding: 'utf-8' });
|
const blobResp = await this.safeRequest(`${base}/git/blobs`, 'POST', { content: target, encoding: 'utf-8' });
|
||||||
const blobSha = this.parseJson<{ sha: string }>(blobResp).sha;
|
const blobSha = this.parseJson<{ sha: string }>(blobResp).sha;
|
||||||
|
|
||||||
const treeResp = await this.safeRequest(`${base}/git/trees`, 'POST', {
|
await this.commitGitHubStyleTree(
|
||||||
base_tree: baseTreeSha,
|
base, branch, baseTreeSha, latestCommitSha,
|
||||||
tree: [{ path: fullPath, mode: GIT_SYMLINK_MODE, type: 'blob', sha: blobSha }],
|
[{ path: fullPath, mode: GIT_SYMLINK_MODE, type: 'blob', sha: blobSha }],
|
||||||
});
|
message
|
||||||
const newTreeSha = this.parseJson<{ sha: string }>(treeResp).sha;
|
);
|
||||||
|
|
||||||
const newCommitResp = await this.safeRequest(`${base}/git/commits`, 'POST', {
|
|
||||||
message,
|
|
||||||
tree: newTreeSha,
|
|
||||||
parents: [latestCommitSha],
|
|
||||||
});
|
|
||||||
const newCommitSha = this.parseJson<{ sha: string }>(newCommitResp).sha;
|
|
||||||
|
|
||||||
await this.safeRequest(`${base}/git/refs/heads/${branch}`, 'PATCH', { sha: newCommitSha });
|
|
||||||
|
|
||||||
return { path: fullPath, sha: blobSha };
|
return { path: fullPath, sha: blobSha };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async pushBatch(items: BatchPushItem[], branch: string, message: string): Promise<BatchPushResult[]> {
|
||||||
|
if (items.length === 0) return [];
|
||||||
|
const base = this.getGitDataApiBase();
|
||||||
|
const { latestCommitSha, baseTreeSha } = await this.resolveGitHubStyleBaseTree(branch);
|
||||||
|
|
||||||
|
const blobShas: string[] = [];
|
||||||
|
for (const item of items) {
|
||||||
|
const blobResp = await this.safeRequest(`${base}/git/blobs`, 'POST', {
|
||||||
|
content: this.encodeContent(item.content),
|
||||||
|
encoding: 'base64',
|
||||||
|
});
|
||||||
|
blobShas.push(this.parseJson<{ sha: string }>(blobResp).sha);
|
||||||
|
}
|
||||||
|
|
||||||
|
const treeItems = items.map((item, i) => ({
|
||||||
|
path: this.getFullPath(item.path),
|
||||||
|
mode: '100644',
|
||||||
|
type: 'blob' as const,
|
||||||
|
sha: blobShas[i] as string,
|
||||||
|
}));
|
||||||
|
|
||||||
|
await this.commitGitHubStyleTree(base, branch, baseTreeSha, latestCommitSha, treeItems, message);
|
||||||
|
|
||||||
|
return items.map((item, i) => ({ path: item.path, sha: blobShas[i] }));
|
||||||
|
}
|
||||||
|
|
||||||
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
||||||
const url = `https://api.github.com/repos/${this.owner}/${this.repo}/git/trees/${branch}?recursive=1`;
|
const url = `https://api.github.com/repos/${this.owner}/${this.repo}/git/trees/${branch}?recursive=1`;
|
||||||
let data: GitHubTreeResponse;
|
let data: GitHubTreeResponse;
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
import { GitServiceInterface, GitTreeEntry } from './git-service-interface';
|
import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult } from './git-service-interface';
|
||||||
import { BaseGitService, ConnectionTestResult, GitFile, GitLabFileResponse, GitLabTreeItem, GIT_SYMLINK_MODE } from './git-service-base';
|
import { BaseGitService, ConnectionTestResult, GitFile, GitLabFileResponse, GitLabTreeItem, GIT_SYMLINK_MODE } from './git-service-base';
|
||||||
|
|
||||||
export class GitLabService extends BaseGitService implements GitServiceInterface {
|
export class GitLabService extends BaseGitService implements GitServiceInterface {
|
||||||
|
|
@ -55,6 +55,28 @@ export class GitLabService extends BaseGitService implements GitServiceInterface
|
||||||
return { path: data.file_path };
|
return { path: data.file_path };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async pushBatch(items: BatchPushItem[], branch: string, message: string): Promise<BatchPushResult[]> {
|
||||||
|
if (items.length === 0) return [];
|
||||||
|
const encodedProjectId = encodeURIComponent(this.projectId);
|
||||||
|
const url = `${this.baseUrl}/api/v4/projects/${encodedProjectId}/repository/commits`;
|
||||||
|
|
||||||
|
const actions = items.map(item => ({
|
||||||
|
action: item.existedRemotely ? 'update' : 'create',
|
||||||
|
file_path: this.getFullPath(item.path),
|
||||||
|
content: this.encodeContent(item.content),
|
||||||
|
encoding: 'base64',
|
||||||
|
}));
|
||||||
|
|
||||||
|
await this.safeRequest(url, 'POST', { branch, commit_message: message, actions });
|
||||||
|
|
||||||
|
// The Commits API response doesn't include each file's new blob sha, so
|
||||||
|
// read it back via a single follow-up tree fetch (one extra call for the
|
||||||
|
// whole batch, not per file) rather than per-file getFile calls.
|
||||||
|
const freshTree = await this.listFilesDetailed(branch, false);
|
||||||
|
const shaByPath = new Map(freshTree.map(e => [e.path, e.sha]));
|
||||||
|
return items.map(item => ({ path: item.path, sha: shaByPath.get(this.getFullPath(item.path)) }));
|
||||||
|
}
|
||||||
|
|
||||||
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
async listFilesDetailed(branch: string, useFilter = true): Promise<GitTreeEntry[]> {
|
||||||
const encodedProjectId = encodeURIComponent(this.projectId);
|
const encodedProjectId = encodeURIComponent(this.projectId);
|
||||||
let allEntries: GitTreeEntry[] = [];
|
let allEntries: GitTreeEntry[] = [];
|
||||||
|
|
|
||||||
|
|
@ -120,6 +120,26 @@ describe('GitignoreManager', () => {
|
||||||
expect(manager.isIgnored('sub/root-ignored.txt')).toBe(true);
|
expect(manager.isIgnored('sub/root-ignored.txt')).toBe(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('scans a pre-fetched remote tree for .gitignore paths instead of calling getRepoGitignores', async () => {
|
||||||
|
const adapter = mockApp.vault.adapter as Mocked<DataAdapter>;
|
||||||
|
vi.mocked(adapter.exists).mockResolvedValue(false);
|
||||||
|
vi.mocked(mockGitService.getFile).mockImplementation((path) => {
|
||||||
|
if (path === '/.gitignore') return Promise.resolve({ content: 'node_modules/', sha: '1' });
|
||||||
|
if (path === '/sub/.gitignore') return Promise.resolve({ content: '*.tmp', sha: '2' });
|
||||||
|
return Promise.resolve({ content: '', sha: '' });
|
||||||
|
});
|
||||||
|
|
||||||
|
await manager.loadGitignores([
|
||||||
|
{ path: '.gitignore', symlink: false, sha: 'a' },
|
||||||
|
{ path: 'sub/.gitignore', symlink: false, sha: 'b' },
|
||||||
|
{ path: 'src/main.ts', symlink: false, sha: 'c' },
|
||||||
|
]);
|
||||||
|
|
||||||
|
expect(mockGitService.getRepoGitignores).not.toHaveBeenCalled();
|
||||||
|
expect(manager.isIgnored('node_modules/test.js')).toBe(true);
|
||||||
|
expect(manager.isIgnored('sub/test.tmp')).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
it('should pick up local-only subdirectory .gitignore not yet on remote', async () => {
|
it('should pick up local-only subdirectory .gitignore not yet on remote', async () => {
|
||||||
// Remote only knows about root .gitignore; sub/.gitignore exists locally but not pushed yet
|
// Remote only knows about root .gitignore; sub/.gitignore exists locally but not pushed yet
|
||||||
vi.mocked(mockGitService.getRepoGitignores).mockResolvedValue(['.gitignore']);
|
vi.mocked(mockGitService.getRepoGitignores).mockResolvedValue(['.gitignore']);
|
||||||
|
|
|
||||||
|
|
@ -20,6 +20,8 @@ describe('SyncManager Batch Operations', () => {
|
||||||
exists: vi.fn(),
|
exists: vi.fn(),
|
||||||
read: vi.fn(),
|
read: vi.fn(),
|
||||||
write: vi.fn(),
|
write: vi.fn(),
|
||||||
|
readBinary: vi.fn(),
|
||||||
|
writeBinary: vi.fn(),
|
||||||
} as unknown as Mocked<DataAdapter>;
|
} as unknown as Mocked<DataAdapter>;
|
||||||
|
|
||||||
mockApp = {
|
mockApp = {
|
||||||
|
|
@ -40,6 +42,7 @@ describe('SyncManager Batch Operations', () => {
|
||||||
getFile: vi.fn(),
|
getFile: vi.fn(),
|
||||||
testConnection: vi.fn(),
|
testConnection: vi.fn(),
|
||||||
listFiles: vi.fn(),
|
listFiles: vi.fn(),
|
||||||
|
listFilesDetailed: vi.fn().mockResolvedValue([]),
|
||||||
deleteFile: vi.fn(),
|
deleteFile: vi.fn(),
|
||||||
getRepoGitignores: vi.fn(),
|
getRepoGitignores: vi.fn(),
|
||||||
updateConfig: vi.fn(),
|
updateConfig: vi.fn(),
|
||||||
|
|
@ -100,6 +103,95 @@ describe('SyncManager Batch Operations', () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('batch commit via gitService.pushBatch', () => {
|
||||||
|
it('groups all queued files into one pushBatch call when the provider supports it', async () => {
|
||||||
|
const files = ['a.md', 'b.md'];
|
||||||
|
const adapter = mockApp.vault.adapter as Mocked<DataAdapter>;
|
||||||
|
|
||||||
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
|
vi.mocked(adapter.read).mockImplementation(async (p) => (p === 'a.md' ? 'content-a' : 'content-b'));
|
||||||
|
// No tree entries: both files are new, so both are queued.
|
||||||
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([]);
|
||||||
|
mockGitService.pushBatch = vi.fn().mockResolvedValue([
|
||||||
|
{ path: 'a.md', sha: 'sha-a' },
|
||||||
|
{ path: 'b.md', sha: 'sha-b' },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const results = await manager.pushAllFiles(files);
|
||||||
|
|
||||||
|
expect(results.success).toBe(2);
|
||||||
|
expect(results.failed).toBe(0);
|
||||||
|
expect(mockGitService.pushBatch).toHaveBeenCalledTimes(1);
|
||||||
|
expect(mockGitService.pushFile).not.toHaveBeenCalled();
|
||||||
|
const [items] = vi.mocked(mockGitService.pushBatch).mock.calls[0]!;
|
||||||
|
expect(items).toEqual([
|
||||||
|
{ path: 'a.md', content: 'content-a', existedRemotely: false },
|
||||||
|
{ path: 'b.md', content: 'content-b', existedRemotely: false },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('handles a mixed binary + text batch, computing blob shas for both', async () => {
|
||||||
|
const files = ['note.md', 'image.png'];
|
||||||
|
const adapter = mockApp.vault.adapter as Mocked<DataAdapter>;
|
||||||
|
|
||||||
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
|
vi.mocked(adapter.read).mockResolvedValue('text content');
|
||||||
|
vi.mocked(adapter.readBinary).mockResolvedValue(new Uint8Array([1, 2, 3]).buffer);
|
||||||
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([]);
|
||||||
|
mockGitService.pushBatch = vi.fn().mockResolvedValue([
|
||||||
|
{ path: 'note.md', sha: 'sha-note' },
|
||||||
|
{ path: 'image.png', sha: 'sha-image' },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const results = await manager.pushAllFiles(files);
|
||||||
|
|
||||||
|
expect(results.success).toBe(2);
|
||||||
|
expect(mockGitService.pushBatch).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('marks every file in a failed chunk as failed, not dropped', async () => {
|
||||||
|
const files = ['a.md', 'b.md'];
|
||||||
|
const adapter = mockApp.vault.adapter as Mocked<DataAdapter>;
|
||||||
|
|
||||||
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
|
vi.mocked(adapter.read).mockResolvedValue('content');
|
||||||
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([]);
|
||||||
|
mockGitService.pushBatch = vi.fn().mockRejectedValue(new Error('commit failed'));
|
||||||
|
|
||||||
|
const results = await manager.pushAllFiles(files);
|
||||||
|
|
||||||
|
expect(results.success).toBe(0);
|
||||||
|
expect(results.failed).toBe(2);
|
||||||
|
expect(results.errors).toEqual([
|
||||||
|
{ file: 'a.md', error: 'commit failed' },
|
||||||
|
{ file: 'b.md', error: 'commit failed' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('skips both getFile and pushBatch when the local blob sha already matches the tree', async () => {
|
||||||
|
const path = 'unchanged.md';
|
||||||
|
const adapter = mockApp.vault.adapter as Mocked<DataAdapter>;
|
||||||
|
|
||||||
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
|
vi.mocked(adapter.read).mockResolvedValue('same content');
|
||||||
|
|
||||||
|
// Compute the real git blob sha for the content so it matches the tree entry.
|
||||||
|
const { gitBlobSha } = await import('../../src/utils/git-blob-sha');
|
||||||
|
const sha = await gitBlobSha('same content');
|
||||||
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([{ path, symlink: false, sha }]);
|
||||||
|
mockGitService.pushBatch = vi.fn();
|
||||||
|
|
||||||
|
const results = await manager.pushAllFiles([path]);
|
||||||
|
|
||||||
|
expect(results.success).toBe(0);
|
||||||
|
expect(results.conflicts).toBe(0);
|
||||||
|
expect(results.failed).toBe(0);
|
||||||
|
expect(mockGitService.getFile).not.toHaveBeenCalled();
|
||||||
|
expect(mockGitService.pushBatch).not.toHaveBeenCalled();
|
||||||
|
expect(mockSettings.syncMetadata[path]?.lastSyncedSha).toBe(sha);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('pullAllAllFiles', () => {
|
describe('pullAllAllFiles', () => {
|
||||||
it('should pull multiple files correctly (strings and TFiles)', async () => {
|
it('should pull multiple files correctly (strings and TFiles)', async () => {
|
||||||
const mockFile = Object.assign(new TFile(), { path: 'file2.md', name: 'file2.md' });
|
const mockFile = Object.assign(new TFile(), { path: 'file2.md', name: 'file2.md' });
|
||||||
|
|
@ -161,8 +253,10 @@ describe('SyncManager Batch Operations', () => {
|
||||||
|
|
||||||
vi.mocked(adapter.exists).mockResolvedValue(true);
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
vi.mocked(adapter.read).mockResolvedValue('local edit');
|
vi.mocked(adapter.read).mockResolvedValue('local edit');
|
||||||
// Remote sha differs from what we last synced, and content differs too.
|
// Remote tree entry's sha differs from what we last synced.
|
||||||
vi.mocked(mockGitService.getFile).mockResolvedValue({ content: 'remote edit', sha: 'sha-changed-on-remote' });
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([
|
||||||
|
{ path, symlink: false, sha: 'sha-changed-on-remote' }
|
||||||
|
]);
|
||||||
|
|
||||||
const results = await manager.pushAllFiles([path]);
|
const results = await manager.pushAllFiles([path]);
|
||||||
|
|
||||||
|
|
@ -170,6 +264,7 @@ describe('SyncManager Batch Operations', () => {
|
||||||
expect(results.conflicts).toBe(1);
|
expect(results.conflicts).toBe(1);
|
||||||
expect(results.failed).toBe(0);
|
expect(results.failed).toBe(0);
|
||||||
expect(mockGitService.pushFile).not.toHaveBeenCalled();
|
expect(mockGitService.pushFile).not.toHaveBeenCalled();
|
||||||
|
expect(mockGitService.getFile).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should skip (not overwrite) a pull when the local file has diverged since last sync', async () => {
|
it('should skip (not overwrite) a pull when the local file has diverged since last sync', async () => {
|
||||||
|
|
@ -197,13 +292,16 @@ describe('SyncManager Batch Operations', () => {
|
||||||
|
|
||||||
vi.mocked(adapter.exists).mockResolvedValue(true);
|
vi.mocked(adapter.exists).mockResolvedValue(true);
|
||||||
vi.mocked(adapter.read).mockResolvedValue('local content');
|
vi.mocked(adapter.read).mockResolvedValue('local content');
|
||||||
vi.mocked(mockGitService.getFile).mockResolvedValue({ content: 'remote content', sha: 'some-sha' });
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([
|
||||||
|
{ path, symlink: false, sha: 'some-sha' }
|
||||||
|
]);
|
||||||
vi.mocked(mockGitService.pushFile).mockResolvedValue({ path, sha: 'new-sha' });
|
vi.mocked(mockGitService.pushFile).mockResolvedValue({ path, sha: 'new-sha' });
|
||||||
|
|
||||||
const results = await manager.pushAllFiles([path]);
|
const results = await manager.pushAllFiles([path]);
|
||||||
|
|
||||||
expect(results.success).toBe(1);
|
expect(results.success).toBe(1);
|
||||||
expect(results.conflicts).toBe(0);
|
expect(results.conflicts).toBe(0);
|
||||||
|
expect(mockGitService.pushFile).toHaveBeenCalledWith(path, 'local content', 'main', expect.any(String), 'some-sha');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
@ -274,10 +372,15 @@ describe('SyncManager Batch Operations', () => {
|
||||||
vi.mocked(mockApp.vault.read).mockResolvedValue('unrelated content');
|
vi.mocked(mockApp.vault.read).mockResolvedValue('unrelated content');
|
||||||
vi.mocked(mockApp.vault.adapter.exists as ReturnType<typeof vi.fn>).mockResolvedValue(true);
|
vi.mocked(mockApp.vault.adapter.exists as ReturnType<typeof vi.fn>).mockResolvedValue(true);
|
||||||
vi.mocked(mockGitService.pushFile).mockResolvedValue({ path: pushedPath, sha: 'new-sha' });
|
vi.mocked(mockGitService.pushFile).mockResolvedValue({ path: pushedPath, sha: 'new-sha' });
|
||||||
|
// detectRename still checks the orphaned metadata entry's remote content directly.
|
||||||
vi.mocked(mockGitService.getFile).mockImplementation(async (path) => {
|
vi.mocked(mockGitService.getFile).mockImplementation(async (path) => {
|
||||||
if (path === orphanedPath) return { content: 'totally different content', sha: 'orphaned-sha' };
|
if (path === orphanedPath) return { content: 'totally different content', sha: 'orphaned-sha' };
|
||||||
return { content: 'old content', sha: 'remote-sha' };
|
return { content: '', sha: '' };
|
||||||
});
|
});
|
||||||
|
// The pushed path's own remote state comes from the pre-fetched tree, not getFile.
|
||||||
|
vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([
|
||||||
|
{ path: pushedPath, symlink: false, sha: 'remote-sha' }
|
||||||
|
]);
|
||||||
|
|
||||||
const results = await manager.pushAllFiles([mockFile]);
|
const results = await manager.pushAllFiles([mockFile]);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -38,6 +38,7 @@ export function createSyncManagerMocks(): SyncManagerMocks {
|
||||||
getFile: vi.fn(),
|
getFile: vi.fn(),
|
||||||
testConnection: vi.fn(),
|
testConnection: vi.fn(),
|
||||||
listFiles: vi.fn(),
|
listFiles: vi.fn(),
|
||||||
|
listFilesDetailed: vi.fn().mockResolvedValue([]),
|
||||||
deleteFile: vi.fn(),
|
deleteFile: vi.fn(),
|
||||||
getRepoGitignores: vi.fn(),
|
getRepoGitignores: vi.fn(),
|
||||||
updateConfig: vi.fn(),
|
updateConfig: vi.fn(),
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||||
import { GitHubService } from '../../src/services/github-service';
|
import { GitHubService } from '../../src/services/github-service';
|
||||||
import { requestUrl, RequestUrlResponse } from 'obsidian';
|
import { requestUrl, RequestUrlResponse } from 'obsidian';
|
||||||
|
import { MAX_BATCH_PUSH_SIZE } from '../../src/services/git-service-base';
|
||||||
|
|
||||||
// Tests for BaseGitService protected methods exercised via GitHubService
|
// Tests for BaseGitService protected methods exercised via GitHubService
|
||||||
describe('BaseGitService', () => {
|
describe('BaseGitService', () => {
|
||||||
|
|
@ -166,6 +167,12 @@ describe('BaseGitService', () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('MAX_BATCH_PUSH_SIZE', () => {
|
||||||
|
it('is a sane positive chunk size', () => {
|
||||||
|
expect(MAX_BATCH_PUSH_SIZE).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('encodeContent / decodeContent round-trip', () => {
|
describe('encodeContent / decodeContent round-trip', () => {
|
||||||
it('should correctly encode and decode UTF-8 content', async () => {
|
it('should correctly encode and decode UTF-8 content', async () => {
|
||||||
const original = 'Hello, 世界! 🌍';
|
const original = 'Hello, 世界! 🌍';
|
||||||
|
|
|
||||||
|
|
@ -117,6 +117,40 @@ describe('GiteaService', () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('pushBatch', () => {
|
||||||
|
it('returns [] and makes no requests for an empty item list', async () => {
|
||||||
|
const result = await service.pushBatch([], 'main', 'push nothing');
|
||||||
|
expect(result).toEqual([]);
|
||||||
|
expect(requestUrl).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('resolves branch via /branches/{branch}, then commits N files in one commit', async () => {
|
||||||
|
vi.mocked(requestUrl)
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: { commit: { id: 'commit1' } } } as unknown as RequestUrlResponse) // resolve branch
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: { tree: { sha: 'tree1' } } } as unknown as RequestUrlResponse) // get commit
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'blob-a' } } as unknown as RequestUrlResponse) // blob a
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'tree2' } } as unknown as RequestUrlResponse) // create tree
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'commit2' } } as unknown as RequestUrlResponse) // create commit
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: {} } as unknown as RequestUrlResponse); // update ref
|
||||||
|
|
||||||
|
const result = await service.pushBatch([{ path: 'a.md', content: 'hello' }], 'main', 'Push 1 file(s) from Obsidian');
|
||||||
|
|
||||||
|
expect(result).toEqual([{ path: 'a.md', sha: 'blob-a' }]);
|
||||||
|
|
||||||
|
const calls = vi.mocked(requestUrl).mock.calls.map(c => c[0] as RequestUrlParam);
|
||||||
|
expect(calls).toHaveLength(6);
|
||||||
|
expect(calls[0]?.url).toBe(`${baseUrl}/api/v1/repos/${owner}/${repo}/branches/main`);
|
||||||
|
expect(calls[1]?.url).toBe(`${baseUrl}/api/v1/repos/${owner}/${repo}/git/commits/commit1`);
|
||||||
|
|
||||||
|
const blobBody = JSON.parse(calls[2]?.body as string) as { content: string; encoding: string };
|
||||||
|
expect(blobBody.encoding).toBe('base64');
|
||||||
|
expect(atob(blobBody.content)).toBe('hello');
|
||||||
|
|
||||||
|
expect(calls[5]?.method).toBe('PATCH');
|
||||||
|
expect(calls[5]?.url).toBe(`${baseUrl}/api/v1/repos/${owner}/${repo}/git/refs/heads/main`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('listFiles', () => {
|
describe('listFiles', () => {
|
||||||
const commitSha = 'abc123commit';
|
const commitSha = 'abc123commit';
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -98,6 +98,54 @@ describe('GitHubService', () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('pushBatch', () => {
|
||||||
|
it('returns [] and makes no requests for an empty item list', async () => {
|
||||||
|
const result = await service.pushBatch([], 'main', 'push nothing');
|
||||||
|
expect(result).toEqual([]);
|
||||||
|
expect(requestUrl).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('commits N files in one commit via ref -> commit -> blobs -> tree -> commit -> ref', async () => {
|
||||||
|
vi.mocked(requestUrl)
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: { object: { sha: 'commit1' } } } as unknown as RequestUrlResponse) // get ref
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: { tree: { sha: 'tree1' } } } as unknown as RequestUrlResponse) // get commit
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'blob-a' } } as unknown as RequestUrlResponse) // blob a
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'blob-b' } } as unknown as RequestUrlResponse) // blob b
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'tree2' } } as unknown as RequestUrlResponse) // create tree
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { sha: 'commit2' } } as unknown as RequestUrlResponse) // create commit
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: {} } as unknown as RequestUrlResponse); // update ref
|
||||||
|
|
||||||
|
const result = await service.pushBatch(
|
||||||
|
[{ path: 'a.md', content: 'hello' }, { path: 'b.md', content: 'world' }],
|
||||||
|
'main',
|
||||||
|
'Push 2 file(s) from Obsidian'
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(result).toEqual([{ path: 'a.md', sha: 'blob-a' }, { path: 'b.md', sha: 'blob-b' }]);
|
||||||
|
|
||||||
|
const calls = vi.mocked(requestUrl).mock.calls.map(c => c[0] as RequestUrlParam);
|
||||||
|
expect(calls).toHaveLength(7);
|
||||||
|
|
||||||
|
// blobs are base64-encoded (not pushSymlink's raw utf-8 path), so binary content works too.
|
||||||
|
const blobABody = JSON.parse(calls[2]?.body as string) as { content: string; encoding: string };
|
||||||
|
expect(blobABody.encoding).toBe('base64');
|
||||||
|
expect(atob(blobABody.content)).toBe('hello');
|
||||||
|
|
||||||
|
const treeBody = JSON.parse(calls[4]?.body as string) as { base_tree: string; tree: Array<{ path: string; mode: string; type: string; sha: string }> };
|
||||||
|
expect(treeBody.base_tree).toBe('tree1');
|
||||||
|
expect(treeBody.tree).toEqual([
|
||||||
|
{ path: 'a.md', mode: '100644', type: 'blob', sha: 'blob-a' },
|
||||||
|
{ path: 'b.md', mode: '100644', type: 'blob', sha: 'blob-b' },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const commitBody = JSON.parse(calls[5]?.body as string) as { message: string; tree: string; parents: string[] };
|
||||||
|
expect(commitBody).toEqual({ message: 'Push 2 file(s) from Obsidian', tree: 'tree2', parents: ['commit1'] });
|
||||||
|
|
||||||
|
expect(calls[6]?.method).toBe('PATCH');
|
||||||
|
expect(JSON.parse(calls[6]?.body as string)).toEqual({ sha: 'commit2' });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('pushFile', () => {
|
describe('pushFile', () => {
|
||||||
it('should push new file correctly (no sha provided)', async () => {
|
it('should push new file correctly (no sha provided)', async () => {
|
||||||
vi.mocked(requestUrl).mockResolvedValueOnce({
|
vi.mocked(requestUrl).mockResolvedValueOnce({
|
||||||
|
|
|
||||||
|
|
@ -68,6 +68,56 @@ describe('GitLabService', () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('pushBatch', () => {
|
||||||
|
it('returns [] and makes no requests for an empty item list', async () => {
|
||||||
|
const result = await service.pushBatch([], 'main', 'push nothing');
|
||||||
|
expect(result).toEqual([]);
|
||||||
|
expect(requestUrl).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('commits via the Commits API actions array, then reads back shas from a follow-up tree fetch', async () => {
|
||||||
|
vi.mocked(requestUrl)
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { id: 'commit-sha' } } as unknown as RequestUrlResponse) // POST commits
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: [
|
||||||
|
{ path: 'a.md', type: 'blob', id: 'new-sha-a' },
|
||||||
|
{ path: 'b.md', type: 'blob', id: 'new-sha-b' },
|
||||||
|
] } as unknown as RequestUrlResponse); // follow-up listFilesDetailed
|
||||||
|
|
||||||
|
const result = await service.pushBatch(
|
||||||
|
[
|
||||||
|
{ path: 'a.md', content: 'hello', existedRemotely: true },
|
||||||
|
{ path: 'b.md', content: 'world', existedRemotely: false },
|
||||||
|
],
|
||||||
|
'main',
|
||||||
|
'Push 2 file(s) from Obsidian'
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(result).toEqual([{ path: 'a.md', sha: 'new-sha-a' }, { path: 'b.md', sha: 'new-sha-b' }]);
|
||||||
|
|
||||||
|
const calls = vi.mocked(requestUrl).mock.calls;
|
||||||
|
expect(calls).toHaveLength(2);
|
||||||
|
const commitCall = calls[0]?.[0] as { url: string; method: string; body: string };
|
||||||
|
expect(commitCall.url).toBe(`${baseUrl}/api/v4/projects/${projectId}/repository/commits`);
|
||||||
|
expect(commitCall.method).toBe('POST');
|
||||||
|
const body = JSON.parse(commitCall.body) as { branch: string; commit_message: string; actions: Array<{ action: string; file_path: string; content: string; encoding: string }> };
|
||||||
|
expect(body.branch).toBe('main');
|
||||||
|
expect(body.commit_message).toBe('Push 2 file(s) from Obsidian');
|
||||||
|
expect(body.actions).toEqual([
|
||||||
|
{ action: 'update', file_path: 'a.md', content: btoa('hello'), encoding: 'base64' },
|
||||||
|
{ action: 'create', file_path: 'b.md', content: btoa('world'), encoding: 'base64' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns sha: undefined for a pushed path missing from the follow-up tree', async () => {
|
||||||
|
vi.mocked(requestUrl)
|
||||||
|
.mockResolvedValueOnce({ status: 201, json: { id: 'commit-sha' } } as unknown as RequestUrlResponse)
|
||||||
|
.mockResolvedValueOnce({ status: 200, json: [] } as unknown as RequestUrlResponse);
|
||||||
|
|
||||||
|
const result = await service.pushBatch([{ path: 'a.md', content: 'hello' }], 'main', 'push');
|
||||||
|
expect(result).toEqual([{ path: 'a.md', sha: undefined }]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('listFiles', () => {
|
describe('listFiles', () => {
|
||||||
it('should list blob files from tree API', async () => {
|
it('should list blob files from tree API', async () => {
|
||||||
mockRequest({ status: 200, json: [
|
mockRequest({ status: 200, json: [
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue