support for backlink context collection

This commit is contained in:
Chris Lettieri 2026-01-29 08:41:25 -05:00
parent 513b007a6a
commit 250d6bb207
10 changed files with 415 additions and 44 deletions

View file

@ -144,6 +144,7 @@ Gather all project documentation:
### Key Features
**Link depth traversal** - Go up to 3 levels deep (breadth-first search)
**Backlink context** - Also gather context from notes that link TO your notes
**Checkbox review** - Toggle individual notes before processing
**Character limits** - Prevents token overflow (default: 100k)
**Dual output modes** - Distill (atomic notes) OR Publish (blog post)
@ -151,6 +152,36 @@ Gather all project documentation:
**Raw context saving** - Skip AI, just aggregate content
**Custom prompts** - Use lenses for focused processing
### Backlink Context Extraction
**NEW**: Gather richer context by including notes that reference your discovered notes.
When you discuss an idea in multiple places and link back to a central note, OpenAugi can now extract those references automatically. Instead of just following forward links, it also finds notes that link TO each discovered note and extracts the header section containing the reference.
**How to use:**
1. Run "Process notes" on any note
2. Backlinks are included by default (toggle off with **"Include backlinks"** if needed)
3. Optionally adjust **"Backlink context"** (0 = header section, 1-5 = lines before/after)
4. Review discovered notes - backlinks show with a "← backlink" badge
5. Backlink snippets appear in a separate "# Backlinks" section in the output
**Example:**
```
You have:
- [[Project Alpha]] - your main project note
- [[Meeting Notes 2025-01-15]] - contains "discussed [[Project Alpha]] timeline"
- [[Ideas]] - contains "this reminds me of [[Project Alpha]]"
With backlinks enabled:
- Project Alpha's forward links are included (full content)
- Meeting Notes snippet: "discussed [[Project Alpha]] timeline"
- Ideas snippet: "this reminds me of [[Project Alpha]]"
```
**Settings:**
- **Include backlinks by default** - Enabled by default; toggle in Settings → Context Gathering
- **Backlink context lines** - Default lines to extract (0 = header section, 1-5 = fixed lines)
---
## Legacy Commands
@ -343,6 +374,8 @@ Configure defaults for the unified context gathering system:
- **Default Link Depth**: Initial depth for link traversal (1-3, default: 1)
- **Default Max Characters**: Character limit before stopping discovery (default: 100,000)
- **Filter Recent Sections by Default**: Automatically enable journal section filtering (default: On)
- **Include Backlinks by Default**: Also discover notes that link to discovered notes (default: On)
- **Backlink Context Lines**: Lines to extract around each backlink reference (0 = header section, default: 0)
### Recent Activity Settings
Configure defaults for recent activity processing:

View file

@ -104,7 +104,9 @@ OpenAugi/
Content discovery and aggregation.
**Discovery Methods:**
- `getLinkedNotes(file)` - Get all linked notes from a file
- `getLinkedNotes(file)` - Get all forward-linked notes from a file
- `getBacklinksForFile(targetFile)` - Get all notes that link TO the target file
- `getBacklinkSnippets(targetFile, sourceFile, contextLines)` - Extract context around backlinks
- `getRecentlyModifiedNotes(daysBack, excludeFolders, fromDate?, toDate?)` - Time-based discovery
**Aggregation:**
@ -114,6 +116,7 @@ Content discovery and aggregation.
- `[[wikilinks]]` and embeds
- Dataview queries (if plugin available)
- Checkbox collections: only `[x]` checked items
- **Backlinks** - Notes that link TO discovered notes (header section/line extraction)
**Journal Filtering:**
- Detects date headers (e.g., `### YYYY-MM-DD`)
@ -136,9 +139,12 @@ gatherContext(config: ContextGatheringConfig): Promise<GatheredContext>
2. **Recent Activity** - Time-based discovery
**Features:**
- Bidirectional link traversal (forward links + backlinks at each depth level)
- Forward links: extract full note content
- Backlinks: extract header section/lines around the link reference
- Character limit enforcement
- Folder exclusion filtering
- Returns discovered notes with metadata (depth, source, size)
- Returns discovered notes with metadata (depth, source, size, isBacklink)
---
@ -167,6 +173,8 @@ interface OpenAugiSettings {
linkDepth: 1 | 2 | 3 // Default: 1
maxCharacters: number // Default: 100000
filterRecentSectionsOnly: boolean
includeBacklinks: boolean // Default: true
backlinkContextLines: number // Default: 0 (0 = header section)
}
}
```
@ -188,14 +196,19 @@ interface ContextGatheringConfig {
excludeFolders: string[]
filterRecentSectionsOnly: boolean
journalSectionDays?: number
includeBacklinks?: boolean // Enable backlink discovery
backlinkContextLines?: number // 0 = header section, N = lines before/after
}
interface DiscoveredNote {
file: TFile
depth: number // 0 = root, 1-3 = linked depth
discoveredVia: string // "root" | "linked from [[X]]" | "recent activity"
discoveredVia: string // "root" | "linked from [[X]]" | "backlink from [[X]]" | "recent activity"
estimatedChars: number
included: boolean // false if exceeded character limit
isBacklink: boolean // true if discovered via backlink
backlinkSnippet?: string // Extracted header section/lines (backlinks only)
backlinkLine?: number // Line number of link (backlinks only)
}
interface GatheredContext {

View file

@ -316,7 +316,24 @@ export default class OpenAugiPlugin extends Plugin {
}
async loadSettings() {
this.settings = Object.assign({}, DEFAULT_SETTINGS, await this.loadData());
const savedData = await this.loadData();
this.settings = Object.assign({}, DEFAULT_SETTINGS, savedData);
// Deep merge nested settings to pick up new defaults
if (savedData?.contextGatheringDefaults) {
this.settings.contextGatheringDefaults = Object.assign(
{},
DEFAULT_SETTINGS.contextGatheringDefaults,
savedData.contextGatheringDefaults
);
}
if (savedData?.recentActivityDefaults) {
this.settings.recentActivityDefaults = Object.assign(
{},
DEFAULT_SETTINGS.recentActivityDefaults,
savedData.recentActivityDefaults
);
}
}
async saveSettings() {
@ -404,10 +421,9 @@ export default class OpenAugiPlugin extends Plugin {
this.loadingIndicator?.show('Aggregating content...');
// Re-aggregate with only selected notes
const files = selectedNotes.map(n => n.file);
const aggregated = await this.distillService.aggregateContent(
files,
// Re-aggregate with only selected notes (preserving backlink info)
const aggregated = await this.contextGatheringService.aggregateContent(
selectedNotes,
context.config.filterRecentSectionsOnly ? context.config.journalSectionDays : undefined
);

View file

@ -34,11 +34,7 @@ export class ContextGatheringService {
if (!config.rootNote) {
throw new Error('Root note required for linked-notes mode');
}
discoveredNotes = await this.discoverLinkedNotes(
config.rootNote,
config.linkDepth,
config.maxCharacters
);
discoveredNotes = await this.discoverLinkedNotes(config);
} else {
if (!config.timeWindow) {
throw new Error('Time window required for recent-activity mode');
@ -72,18 +68,22 @@ export class ContextGatheringService {
/**
* Discover notes by traversing links up to specified depth using BFS
* @param rootNote The starting note
* @param maxDepth Maximum depth to traverse (1-3)
* @param maxCharacters Stop when this many characters have been gathered
* Traverses both forward links and backlinks at each level
* @param config The context gathering configuration
* @returns Array of discovered notes with metadata
*/
private async discoverLinkedNotes(
rootNote: TFile,
maxDepth: number,
maxCharacters: number
config: ContextGatheringConfig
): Promise<DiscoveredNote[]> {
const rootNote = config.rootNote!;
const maxDepth = config.linkDepth;
const maxCharacters = config.maxCharacters;
const includeBacklinks = config.includeBacklinks ?? false;
const backlinkContextLines = config.backlinkContextLines ?? 2;
const discovered = new Map<string, DiscoveredNote>();
const queue: Array<{ file: TFile; depth: number; via: string }> = [];
const queued = new Set<string>(); // Track queued files to prevent duplicates
const queue: Array<{ file: TFile; depth: number; via: string; isBacklink: boolean }> = [];
// Start with root note
const rootContent = await this.app.vault.read(rootNote);
@ -92,29 +92,66 @@ export class ContextGatheringService {
depth: 0,
discoveredVia: 'root',
estimatedChars: rootContent.length,
included: true
included: true,
isBacklink: false
});
// Get direct links from root
// Get direct forward links from root
const rootLinks = await this.distillService.getLinkedNotes(rootNote);
for (const link of rootLinks) {
queue.push({ file: link, depth: 1, via: rootNote.basename });
queue.push({ file: link, depth: 1, via: rootNote.basename, isBacklink: false });
queued.add(link.path);
}
// Get backlinks to root if enabled
if (includeBacklinks) {
const rootBacklinks = this.distillService.getBacklinksForFile(rootNote);
for (const backlink of rootBacklinks) {
// Forward links take priority - only add if not already queued
if (!queued.has(backlink.path)) {
queue.push({ file: backlink, depth: 1, via: rootNote.basename, isBacklink: true });
queued.add(backlink.path);
}
}
}
// BFS traversal
let totalChars = rootContent.length;
while (queue.length > 0) {
const { file, depth, via } = queue.shift()!;
const { file, depth, via, isBacklink } = queue.shift()!;
// Skip if already discovered
if (discovered.has(file.path)) {
continue;
}
// Read content to check size
const content = await this.app.vault.read(file);
const chars = content.length;
let chars: number;
let snippetContent: string | undefined;
if (isBacklink) {
// For backlinks, get snippets instead of full content
// Find the target file (the note this backlink points TO) from discovered notes
const targetEntry = Array.from(discovered.entries()).find(
([_, note]) => note.file.basename === via
);
const targetFile = targetEntry ? targetEntry[1].file : rootNote;
const snippets = await this.distillService.getBacklinkSnippets(
targetFile,
file,
backlinkContextLines
);
// Deduplicate snippets (multiple links in same section would produce identical snippets)
const uniqueSnippets = [...new Set(snippets.map(s => s.snippet))];
snippetContent = uniqueSnippets.join('\n\n---\n\n');
chars = snippetContent.length;
} else {
// For forward links, read full content
const content = await this.app.vault.read(file);
chars = content.length;
}
// Check if adding this note would exceed character limit
if (totalChars + chars > maxCharacters) {
@ -122,9 +159,11 @@ export class ContextGatheringService {
discovered.set(file.path, {
file,
depth,
discoveredVia: `linked from [[${via}]]`,
discoveredVia: isBacklink ? `backlink from [[${via}]]` : `linked from [[${via}]]`,
estimatedChars: chars,
included: false // Excluded due to size limit
included: false, // Excluded due to size limit
isBacklink,
backlinkSnippet: isBacklink ? snippetContent : undefined
});
continue;
}
@ -133,21 +172,39 @@ export class ContextGatheringService {
discovered.set(file.path, {
file,
depth,
discoveredVia: `linked from [[${via}]]`,
discoveredVia: isBacklink ? `backlink from [[${via}]]` : `linked from [[${via}]]`,
estimatedChars: chars,
included: true
included: true,
isBacklink,
backlinkSnippet: isBacklink ? snippetContent : undefined
});
console.log(`[OpenAugi] Discovered: ${file.basename} | isBacklink: ${isBacklink} | via: ${via} | depth: ${depth}`);
totalChars += chars;
// If we haven't reached max depth, get links from this note
if (depth < maxDepth) {
try {
// Get forward links
const linkedNotes = await this.distillService.getLinkedNotes(file);
for (const linkedNote of linkedNotes) {
// Don't queue if already discovered
if (!discovered.has(linkedNote.path)) {
queue.push({ file: linkedNote, depth: depth + 1, via: file.basename });
// Don't queue if already queued or discovered
if (!queued.has(linkedNote.path) && !discovered.has(linkedNote.path)) {
queue.push({ file: linkedNote, depth: depth + 1, via: file.basename, isBacklink: false });
queued.add(linkedNote.path);
}
}
// Get backlinks if enabled
if (includeBacklinks) {
const backlinks = this.distillService.getBacklinksForFile(file);
for (const backlink of backlinks) {
// Don't queue if already queued or discovered
if (!queued.has(backlink.path) && !discovered.has(backlink.path)) {
queue.push({ file: backlink, depth: depth + 1, via: file.basename, isBacklink: true });
queued.add(backlink.path);
}
}
}
} catch (error) {
@ -190,7 +247,8 @@ export class ContextGatheringService {
depth: 0, // No depth concept for recent activity
discoveredVia: 'recent activity',
estimatedChars: content.length,
included: true
included: true,
isBacklink: false
});
}
@ -223,17 +281,45 @@ export class ContextGatheringService {
/**
* Aggregate content from notes
* For forward links: uses full content (with optional journal filtering)
* For backlinks: uses only the snippet
* @param notes Notes to aggregate
* @param filterJournalDays Optional days back for journal section filtering
* @returns Aggregated content and source note names
*/
private async aggregateContent(
async aggregateContent(
notes: DiscoveredNote[],
filterJournalDays?: number
): Promise<{ content: string; sourceNotes: string[] }> {
// Reuse existing aggregateContent from DistillService
const files = notes.map(n => n.file);
return await this.distillService.aggregateContent(files, filterJournalDays);
// Separate forward links and backlinks
const forwardLinks = notes.filter(n => !n.isBacklink);
const backlinks = notes.filter(n => n.isBacklink);
// Aggregate forward links using existing method
const forwardFiles = forwardLinks.map(n => n.file);
const forwardResult = await this.distillService.aggregateContent(forwardFiles, filterJournalDays);
// Build backlinks section
let backlinkContent = '';
const backlinkSourceNotes: string[] = [];
for (const note of backlinks) {
backlinkContent += `\n\n# Backlink: ${note.file.basename}\n`;
backlinkContent += `*${note.discoveredVia}*\n\n`;
if (note.backlinkSnippet) {
backlinkContent += note.backlinkSnippet;
} else {
// Fallback: read full content if snippet is missing
const fullContent = await this.app.vault.read(note.file);
backlinkContent += fullContent;
}
backlinkSourceNotes.push(note.file.basename);
}
return {
content: forwardResult.content + backlinkContent,
sourceNotes: [...forwardResult.sourceNotes, ...backlinkSourceNotes]
};
}
/**

View file

@ -477,6 +477,109 @@ ${content}
return uniqueFiles;
}
/**
* Get all files that link TO the given target file (backlinks)
* Uses the metadata cache's resolvedLinks for efficient lookup
* @param targetFile The file to find backlinks for
* @returns Array of files that link to the target
*/
getBacklinksForFile(targetFile: TFile): TFile[] {
const backlinks: TFile[] = [];
const resolvedLinks = this.app.metadataCache.resolvedLinks;
// resolvedLinks is Record<sourcePath, Record<targetPath, linkCount>>
for (const sourcePath in resolvedLinks) {
const targetLinks = resolvedLinks[sourcePath];
if (targetLinks && targetFile.path in targetLinks) {
// This source file links to our target
const sourceFile = this.app.vault.getAbstractFileByPath(sourcePath);
if (sourceFile instanceof TFile) {
backlinks.push(sourceFile);
}
}
}
return backlinks;
}
/**
* Extract context snippets from a source file for all links pointing to target
* @param targetFile The file being linked TO
* @param sourceFile The file containing the backlinks
* @param contextLines Number of lines before/after (0 = full paragraph)
* @returns Array of snippets with line numbers
*/
async getBacklinkSnippets(
targetFile: TFile,
sourceFile: TFile,
contextLines: number = 2
): Promise<Array<{ snippet: string; line: number }>> {
const snippets: Array<{ snippet: string; line: number }> = [];
const cache = this.app.metadataCache.getFileCache(sourceFile);
if (!cache?.links) {
return snippets;
}
const content = await this.app.vault.read(sourceFile);
for (const link of cache.links) {
// Check if this link points to our target file
const resolvedFile = this.app.metadataCache.getFirstLinkpathDest(
link.link,
sourceFile.path
);
if (resolvedFile?.path === targetFile.path) {
const lineNumber = link.position.start.line;
const snippet = this.extractContextAroundLine(content, lineNumber, contextLines);
snippets.push({ snippet, line: lineNumber });
}
}
return snippets;
}
/**
* Extract context around a specific line
* @param content The full file content
* @param targetLine The line number (0-indexed) to extract context around
* @param contextLines Number of lines before/after (0 = header section)
* @returns The extracted context string
*/
private extractContextAroundLine(
content: string,
targetLine: number,
contextLines: number
): string {
const lines = content.split('\n');
if (contextLines === 0) {
// Header section mode: find the containing section
let start = targetLine;
let end = targetLine;
// Walk backward to find the nearest header (or start of file)
while (start > 0 && !lines[start].match(/^#+\s/)) {
start--;
}
// Walk forward to find the next header (or end of file)
end = targetLine + 1;
while (end < lines.length && !lines[end].match(/^#+\s/)) {
end++;
}
end--; // Don't include the next header
return lines.slice(start, end + 1).join('\n');
} else {
// Fixed lines mode: N lines before and after
const start = Math.max(0, targetLine - contextLines);
const end = Math.min(lines.length - 1, targetLine + contextLines);
return lines.slice(start, end + 1).join('\n');
}
}
/**
* Convert date format string to regex pattern
* @param format The date format string (e.g., "### YYYY-MM-DD")

View file

@ -29,6 +29,10 @@ export interface ContextGatheringConfig {
// For recent activity: how many days back to filter journal sections
journalSectionDays?: number;
// Backlink discovery settings
includeBacklinks?: boolean; // Whether to also discover backlinks at each level
backlinkContextLines?: number; // 0 = header section, N = lines before/after link
}
/**
@ -37,9 +41,14 @@ export interface ContextGatheringConfig {
export interface DiscoveredNote {
file: TFile;
depth: number; // 0 = root, 1 = direct link, 2 = second level, etc.
discoveredVia: string; // "root" | "linked from [[Note]]" | "recent activity"
discoveredVia: string; // "root" | "linked from [[Note]]" | "backlink from [[Note]]" | "recent activity"
estimatedChars: number;
included: boolean; // User can toggle in checkbox modal
// Backlink-specific fields
isBacklink: boolean; // true if discovered via backlink
backlinkSnippet?: string; // The extracted header section/block (only for backlinks)
backlinkLine?: number; // Line number where link appears
}
/**

View file

@ -11,6 +11,8 @@ export interface ContextGatheringDefaults {
linkDepth: number;
maxCharacters: number;
filterRecentSectionsOnly: boolean;
includeBacklinks: boolean;
backlinkContextLines: number; // 0 = header section, N = lines before/after
}
export interface OpenAugiSettings {
@ -48,6 +50,8 @@ export const DEFAULT_SETTINGS: OpenAugiSettings = {
contextGatheringDefaults: {
linkDepth: 1,
maxCharacters: 100000,
filterRecentSectionsOnly: true
filterRecentSectionsOnly: true,
includeBacklinks: true,
backlinkContextLines: 0
}
};

View file

@ -33,7 +33,9 @@ export class ContextGatheringModal extends Modal {
excludeFolders: settings.recentActivityDefaults.excludeFolders,
filterRecentSectionsOnly: settings.contextGatheringDefaults.filterRecentSectionsOnly,
dateHeaderFormat: settings.recentActivityDefaults.dateHeaderFormat,
journalSectionDays: settings.recentActivityDefaults.daysBack
journalSectionDays: settings.recentActivityDefaults.daysBack,
includeBacklinks: settings.contextGatheringDefaults.includeBacklinks,
backlinkContextLines: settings.contextGatheringDefaults.backlinkContextLines
};
}
@ -193,6 +195,69 @@ export class ContextGatheringModal extends Modal {
}
});
});
// Backlink settings container
const backlinkContainer = this.modeSpecificContainer.createDiv({ cls: 'backlink-settings' });
// Include backlinks toggle
new Setting(backlinkContainer)
.setName('Include backlinks')
.setDesc('Also discover notes that link to discovered notes')
.addToggle(toggle => {
toggle
.setValue(this.config.includeBacklinks ?? false)
.onChange(async value => {
this.config.includeBacklinks = value;
this.renderBacklinkContextSetting(backlinkContainer);
await this.updateEstimate();
});
});
// Render context lines setting if backlinks enabled
this.renderBacklinkContextSetting(backlinkContainer);
}
private renderBacklinkContextSetting(container: HTMLElement) {
// Remove existing setting if present
const existing = container.querySelector('.backlink-context-setting');
if (existing) {
existing.remove();
}
if (!this.config.includeBacklinks) {
return;
}
const contextLines = this.config.backlinkContextLines ?? 2;
const contextDesc = contextLines === 0
? 'Header section'
: `${contextLines} line${contextLines === 1 ? '' : 's'} before/after`;
new Setting(container)
.setName('Backlink context')
.setDesc(`Extract: ${contextDesc}`)
.setClass('backlink-context-setting')
.addSlider(slider => {
slider
.setLimits(0, 5, 1)
.setValue(contextLines)
.setDynamicTooltip()
.onChange(value => {
this.config.backlinkContextLines = value;
// Update description
const desc = value === 0
? 'Header section'
: `${value} line${value === 1 ? '' : 's'} before/after`;
const settingEl = container.querySelector('.backlink-context-setting .setting-item-description');
if (settingEl) {
settingEl.textContent = `Extract: ${desc}`;
}
});
})
.addExtraButton(button => {
button.setIcon('info');
button.setTooltip('0 = header section (text under current markdown header)\n1-5 = fixed number of lines before and after the link');
});
}
private renderRecentActivitySettings() {

View file

@ -187,15 +187,33 @@ export class ContextSelectionModal extends Modal {
contentDiv.style.flexDirection = 'column';
contentDiv.style.gap = '2px';
const titleEl = contentDiv.createEl('span');
const titleRow = contentDiv.createDiv();
titleRow.style.display = 'flex';
titleRow.style.alignItems = 'center';
titleRow.style.gap = '8px';
const titleEl = titleRow.createEl('span');
titleEl.setText(note.file.basename);
titleEl.style.fontWeight = '500';
// Backlink indicator badge
if (note.isBacklink) {
const backlinkBadge = titleRow.createEl('span');
backlinkBadge.setText('← backlink');
backlinkBadge.style.fontSize = '0.75em';
backlinkBadge.style.padding = '2px 6px';
backlinkBadge.style.borderRadius = '10px';
backlinkBadge.style.backgroundColor = 'var(--interactive-accent)';
backlinkBadge.style.color = 'var(--text-on-accent)';
backlinkBadge.style.fontWeight = 'normal';
}
const metaEl = contentDiv.createEl('span');
metaEl.style.fontSize = '0.85em';
metaEl.style.color = 'var(--text-muted)';
const sizeKb = (note.estimatedChars / 1000).toFixed(1);
metaEl.setText(`${sizeKb}k chars · ${note.discoveredVia}`);
const contentType = note.isBacklink ? 'snippet' : 'full note';
metaEl.setText(`${sizeKb}k chars (${contentType}) · ${note.discoveredVia}`);
});
});

View file

@ -326,6 +326,30 @@ export class OpenAugiSettingTab extends PluginSettingTab {
})
);
new Setting(containerEl)
.setName('Include backlinks by default')
.setDesc('Also discover notes that link to discovered notes')
.addToggle(toggle => toggle
.setValue(this.plugin.settings.contextGatheringDefaults.includeBacklinks)
.onChange(async (value) => {
this.plugin.settings.contextGatheringDefaults.includeBacklinks = value;
await this.plugin.saveSettings();
})
);
new Setting(containerEl)
.setName('Backlink context lines')
.setDesc('Lines to extract around each backlink (0 = header section)')
.addSlider(slider => slider
.setLimits(0, 5, 1)
.setValue(this.plugin.settings.contextGatheringDefaults.backlinkContextLines)
.setDynamicTooltip()
.onChange(async (value) => {
this.plugin.settings.contextGatheringDefaults.backlinkContextLines = value;
await this.plugin.saveSettings();
})
);
// Advanced Settings Header
containerEl.createEl('h3', { text: 'Advanced Settings' });