When defaultSaveFolder is a hidden directory (e.g. .copilot/conversations),
Obsidian's metadata cache doesn't index it, causing saves to fail and
chat history to show empty.
Add adapter-level fallback utilities in src/utils/vaultAdapterUtils.ts:
- resolveFileByPath: vault lookup with synthetic TFile fallback
- listMarkdownFiles: folder listing with adapter fallback
- patchFrontmatter: processFrontMatter with adapter YAML patching fallback
- readFrontmatterViaAdapter: parse frontmatter from raw file content
Apply fallbacks across ChatPersistenceManager (save, load, list, find)
and main.ts (history picker, load, delete, title update, lastAccessedAt).
Also removes obsolete design docs from docs/.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Miyo PR review findings (privacy, dedup, architecture)
- Gate request body logging behind debug flag to prevent note content
leaking to vault log files on every upsert (P1 privacy fix)
- Replace duplicated grace period logic in findRelevantNotes with
isSelfHostAccessValid() from plusUtils (P1 dedup fix)
- Spread-copy metadata in fromMiyoDocument to avoid mutating API response
- Remove dead if(enabled) branch in CopilotPlusSettings toggle handler
- Gate per-document upsert log lines behind debug to prevent log churn
- Rename misleading test; add no-embeddings fallback path test coverage
- Add TODO for cross-platform service discovery paths (Windows/Linux)
- Use this.client directly in MiyoSemanticRetriever.getExplicitChunks
instead of VectorStoreManager singleton to avoid backend mismatch
- Remove unused apiKeyOverride param from buildHeaders; document dual auth
- Read pagination total only from first response in getIndexedFiles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address PR review comments on Miyo retriever and index backend
- Guard response.documents with ?? [] in getExplicitChunks to prevent
crash when backend omits the field or parseResponseJson returns {}
- Remove redundant getSettings().debug guard around logInfo calls since
the logging utilities already respect the debug flag internally
- Document the debug-flag behavior in AGENTS.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor: remove explicit path reads from Miyo semantic retriever
* fix: rename remaining enableMiyoSearch references
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename `enableMiyoSearch` → `enableMiyo` across settings, UI, and all consumers
- Skip fulltext search in performLexicalSearch (Plus/agent path) when Miyo is active;
filterRetriever still runs for tag/title matching
- VaultQA chain now passes `{ enableMiyo: false }` to RetrieverFactory so it never
routes through Miyo
- Expose `RetrieverFactory.isMiyoActive()` for clean Miyo-state checks
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: upgrade @langchain/google-genai to fix Gemini 3 preview image support (#2178)
The @langchain/google-genai v1.0.0 had a hardcoded _isMultimodalModel check
that only recognized gemini-1.5, gemini-2, and "vision" model names. Gemini 3
preview models (gemini-3-flash-preview, gemini-3-pro-preview) were rejected
with "This model does not support images" when sending image content.
Upgrade to v1.0.2 which adds gemini-3 to the multimodal model pattern.
Also bump @google/generative-ai from ^0.21.0 to ^0.24.0 (required dep).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update built-in Gemini Pro to 3.1 preview model IDs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Firecrawl and Supadata self-host support for web search and YouTube
Self-host mode users can now use their own API keys for web search (Firecrawl)
and YouTube transcripts (Supadata) instead of routing through the Brevilabs
backend. Tools gracefully fall back to Brevilabs when no self-host key is set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Perplexity Sonar as alternative self-host web search provider
Add provider dispatch pattern for self-host web search, supporting both
Firecrawl and Perplexity Sonar via a dropdown selector in settings UI.
Changes:
- Refactor selfHostWebSearch into provider dispatch (firecrawl/perplexity)
- Add hasSelfHostSearchKey() helper for provider-agnostic key checks
- Add settings UI dropdown for web search provider selection
- Fix Mention.ts YouTube transcription to route via Supadata in self-host mode
- Fix cached failed URL results never being retried (retry-on-error)
- Clear Miyo Search state when self-host mode is disabled
- Add 19 unit tests for selfHostServices
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Firecrawl and Supadata self-host support for web search and YouTube
Self-host mode users can now use their own API keys for web search (Firecrawl)
and YouTube transcripts (Supadata) instead of routing through the Brevilabs
backend. Tools gracefully fall back to Brevilabs when no self-host key is set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: drop legacy Perplexity wrapper from self-host web search
selfHostWebSearch now returns { content, citations } directly instead of
wrapping results in the OpenAI-chat-completion shaped WebSearchResponse
that Brevilabs inherited from Perplexity. SearchTools unwraps each path
independently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move Miyo Search under self-host conditional and improve description
Miyo Search toggle is now only visible when self-host mode is enabled,
grouped alongside Firecrawl and Supadata settings. Updated description
to be less technical and highlight vault size advantage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add token budget enforcement analysis and fix plan
Document root cause of context window overflow (2.7M tokens sent to 1M
model): L4 chat history bypasses all compaction systems. Add fix plan
with phased approach to enforce token budget at message assembly point.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: correct root cause analysis — L1 unbudgeted, not L4
Previous analysis incorrectly blamed L4 chat history as the primary
culprit. Investigation shows L4 stores only bare L5 text + compacted
responses. The real issue is systemic: no total payload enforcement,
with L1 (project context) being the largest unbudgeted layer and
PROJECT_COMPACT_THRESHOLD being blind to L1 size.
Updated fix plan to be model-agnostic (use autoCompactThreshold as
single budget, no model-specific lookup tables), added contextTurns
deprecation, and history guarantee principle.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: hard cap L1 project context at 600k tokens + payload diagnostics
- Truncate project context at 2.4M chars (~600k tokens) to prevent
total payload from exceeding model context windows (temporary fix
until full token budget enforcement is implemented)
- Add per-layer token estimate logging when payload exceeds 2M chars
to help diagnose context window overflow reports
- Document Obsidian CLI dev tools in CLAUDE.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix "Cannot set properties of undefined (setting 'embedding')" during indexing
Rebuild internalDocumentIDStore after loading partitioned chunks to eliminate
ghost IDs from removed documents that cause position mismatches. Also add
missing nchars field to Orama schema to match the OramaDocument interface.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove Notice popup spam for per-file indexing errors
Errors during indexing (e.g. "Semantic index database not found" when DB
isn't loaded yet) were triggering a Notice popup for every file event,
flooding the user with popups. Errors are already logged to console and
surfaced in the in-chat IndexingProgressCard.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix unwanted indexing when semantic search disabled + editor lag during indexing
Add defense-in-depth enableSemanticSearchV3 guard to both VectorStoreManager
and IndexOperations to prevent indexing when semantic search is off (fixes
plusUtils.applyPlusSettings bypass). Throttle Jotai atom updates to 500ms
intervals during indexing to reduce React re-renders. Yield to main thread
between batches to keep the editor responsive.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove yieldToMainThread — existing awaits already yield to the event loop
The batch loop already has multiple await points (rateLimiter.wait,
embedDocuments, upsert, save) that yield to the main thread. The explicit
setTimeout(0) was redundant and added no user-perceived benefit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard throttled callback against stale indexing sessions
Cancel pending throttle timer in resetIndexingProgressState() so a
trailing write from a previous (failed/aborted) run cannot overwrite
the freshly-reset atom state of a new run. Also reorder declarations
so throttle variables are defined before the function that references them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move badge close button to left icon overlay on hover
Centralize the icon + X overlay pattern in ContextBadgeWrapper instead of
copy-pasting the X button across all 8 badge types. The X now appears on
hover over the left icon position using invisible/visible toggling for
zero layout shift.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* support mobile
* fix keyboard control
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add optional streamUsage setting to CustomModel interface that allows users
to control whether stream_options is sent to the API. This fixes compatibility
with 3rd party providers (e.g., Databricks, MLFlow) that do not support the
stream_options parameter.
- Add streamUsage?: boolean to CustomModel interface in aiParams.ts
- Pass streamUsage to ChatOpenAI config for OPENAI_FORMAT and LM_STUDIO providers
- Add UI toggle in ModelAddDialog and ModelEditDialog with helpful tooltip
- Default to false to maintain compatibility with providers that error on stream_options
Closes#2046
Sanitize LLM-generated file paths before creating files to prevent
ENAMETOOLONG crashes on Linux (ext4 255-byte filename limit). Truncates
overlong basenames while preserving file extensions and multi-byte
character boundaries. Sanitization happens at the tool entry point so
the preview flow (ApplyView) also receives the safe path.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* Replace indexing Notice with in-chat progress card & fix phantom re-indexing
Replace the Obsidian Notice popup for indexing progress with a persistent
in-chat progress card using the same Jotai atom pattern as project mode.
Key changes:
- New IndexingProgressCard component with progress bar, pause/resume/stop,
error display, and auto-close on completion
- Jotai atom (indexingProgressAtom) driven by IndexOperations, read by React
- Card only appears when files actually need indexing (no flash on mode switch)
- User-initiated actions (refresh/reindex buttons) show "Index Up to Date"
feedback with green check icon
- Fix phantom re-indexing: checkIndexIntegrity() is now diagnostic-only
- Fix progress bar accuracy: totalFiles counted after chunk preparation
- Remove duplicate completion Notices from ChatControls and RelevantNotes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix insert/replace at cursor including agent reasoning blocks (#2174)
Strip <!--AGENT_REASONING:...--> markers in cleanMessageForCopy() so
that "Insert / Replace at cursor" only inserts the actual AI response,
not the internal agent reasoning metadata.
Uses greedy .* (single-line) so the regex matches to the real closing
--> even if the JSON payload contains that sequence.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Restore integrity check marking files for re-indexing
checkIndexIntegrity() was made diagnostic-only to fix phantom
re-indexing, but this left files with missing embeddings permanently
skipped. The real safeguard is clearFilesMissingEmbeddings() at the
start of each indexing run, which prevents the repeat cycle. Restore
the mark so the next indexing trigger picks up integrity failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Strip <!--AGENT_REASONING:...--> markers in cleanMessageForCopy() so
that "Insert / Replace at cursor" only inserts the actual AI response,
not the internal agent reasoning metadata.
Uses greedy .* (single-line) so the regex matches to the real closing
--> even if the JSON payload contains that sequence.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Simplify search pipeline: remove expandedTerms, use metadata-cache tag matching
- Remove `expandedTerms` concept from QueryExpander, FullTextEngine, SearchCore,
and all interfaces — LLM expansion now only produces `salientTerms` and
`expandedQueries`, no separate related-term list
- Remove weighted 90/10 two-phase scoring from FullTextEngine; use single BM25+
search with salient terms for ranking
- Replace tag-based `returnAll` mode with Obsidian metadata-cache tag matching:
`getTagMatches()` uses `getAllTags()` with hierarchical prefix matching and
injects tag-matched notes as guaranteed full-note documents (score 1.0)
- Remove `buildTagRecallQueries()` tag-hierarchy splitting from SearchCore
- Add `isGrepWorthy()` filter in GrepScanner to skip short ASCII terms (< 3 chars)
- Reduce default candidateLimit and grepLimit from 500 to 200
- Rewrite search v3 README to reflect current implementation
- Update all tests for removed fields and new tag-match injection behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix isGrepWorthy filtering out valid 2-char terms like "AI", "ML", "UI"
SearchCore lowercases all recall terms before passing to GrepScanner, so
acronym detection by casing is impossible. Lower the threshold to filter
only single-char terms — grep is capped at 200 results anyway, and BM25+
handles precision downstream.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Extract FilterRetriever from TieredLexicalRetriever for guaranteed filter matching
Move deterministic title/tag/time-range matching into standalone FilterRetriever
class that runs at the orchestration layer (SearchTools, VaultQAChainRunner,
customSearchDB). This ensures filter results bypass all downstream retriever
processing (top-K slicing, score normalization) and reach the LLM context via
separate <filterResults>/<searchResults> XML sections with continuous ID numbering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Cap FilterRetriever tag and time-range results to maxK/RETURN_ALL_LIMIT
Prevents unbounded filter result sets (e.g. common tags like #daily) from
blowing up prompt/context size and read latency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Clean up legacy parser, dead code, and DRY violations
- Remove parseLegacyFormat from QueryExpander (fall back to fallbackExpansion for non-XML LLM responses)
- Remove dead fuzzy variant generation in fallbackExpansion
- Remove deprecated expandQueries backward-compat method
- Extract shared mapDoc helper in SearchTools to eliminate duplicated document mapping
- Use existing toIsoString helper in formatSplitSearchResultsForLLM
- Add FilterRetriever cap regression tests for maxK and RETURN_ALL_LIMIT
- Convert legacy-format tests to XML format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Restore expanded limits for tag queries in VaultQAChainRunner
Pass returnAll: true when tagTerms are present so FilterRetriever
and the main retriever use RETURN_ALL_LIMIT instead of maxK cap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Cap merged filter+search results to prevent oversized prompts
Filter and search results are now capped post-merge (filter results
prioritized) to prevent 100-200 doc payloads on tag-heavy queries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add retrieval cap notice in VaultQA to prompt Copilot Plus upgrade
When merged filter+search results exceed the retrieval cap, the AI
is instructed to inform the user that some documents were omitted
and suggest upgrading to Copilot Plus for more complete answers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add RELEASES.md with full release history and v3.2.0 draft
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* 3.2.0
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Show Self-Host Mode section to all users with disabled toggle for non-lifetime
The Self-Host Mode settings section is now visible to everyone instead of
being hidden behind an eligibility check. The toggle is disabled (greyed out)
for users who are not on a lifetime plan (believer/supporter), making the
feature discoverable while preserving the access restriction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix self-host toggle not disabled after removing license key
Three issues fixed:
1. useIsSelfHostEligible() checked grace period before license key,
so removing the key didn't revoke eligibility while in grace period.
Moved license key check to the top and added auto-revocation of
enableSelfHostMode when key is absent.
2. isSelfHostModeValid() didn't check license key at all, so backend
code (RetrieverFactory, isPlusEnabled) continued to grant self-host
access after key removal. Added license key guard.
3. useIsPlusUser() self-host bypass didn't check license key. Added
the same guard for consistency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix self-host toggle not disabled when switching to non-lifetime key
useIsSelfHostEligible() trusted cached grace period / permanent
validation before consulting the API, so swapping from a believer
key to a plus key kept the toggle enabled.
Now always verifies via API when a license key is present. Cached
validation (grace period, permanent count) is only used as an
offline fallback when the API call fails. Also auto-revokes
enableSelfHostMode when the API confirms the key is not eligible.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Replace warning triangle with Build Index button in Relevant Notes
Improves UX when semantic search is off or index is unavailable by replacing
the yellow warning triangle and text with a clear "Build Index" button that
enables semantic search (with confirmation modal) and triggers vault indexing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add reactive index signal and move Build Index button to header
- Create indexSignal pub-sub so UI reacts to external index changes
(clear, rebuild, reindex) without manual refresh
- Fire notifyIndexChanged() from VectorStoreManager (single source)
- Move Build Index button to header area so it's always visible when
!hasIndex, even when link-based relevant notes are showing
- Update help tooltip to mention semantic search requirement
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
HyDE (Hypothetical Document Embeddings) rewrites user queries into
hypothetical answer passages before embedding them for vector search.
This produces garbage for non-question queries like "find all my ai
digests" — the LLM generates an instructional essay instead of useful
search terms, polluting semantic results and causing Plus mode to miss
relevant notes that Agent mode finds via multiple search iterations.
Removing HyDE:
- Eliminates an extra LLM call per search (faster responses)
- Uses the raw query for vector embedding search (better matching)
- QueryExpander already handles LLM-based query expansion correctly
- hybridRetriever.ts is already marked DEPRECATED
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Add Claude Opus 4.6, Sonnet 4.5 (replacing Sonnet 4)
- Add Gemini 3 Flash/Pro preview (Google direct + OpenRouter)
- Add GPT-5.3 Codex, upgrade OpenRouter GPT to 5.2/5-mini
- Upgrade Grok 4-fast to 4.1-fast (xAI + OpenRouter)
- Remove Gemini 2.5 Flash Lite, demote GPT-4.1 to non-core
- Preview models are not core or project-enabled
- Extend thinking model detection for Claude Opus 4.x
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve search recall with note-diverse top-K, chunk-aware scoring, and adaptive cutoff module
- Add note-diverse top-K selection so every unique note gets representation
before any note gets a second chunk slot (fixes "find my ai digests" missing notes)
- Fix GraphBoostCalculator: resolve chunk IDs to note paths for metadata lookups,
deduplicate to note level in filterCandidates
- Fix FolderBoostCalculator: count unique notes per folder instead of raw chunks
- Bump title field weight 3→5 for stronger filename match priority
- Add AdaptiveCutoff module (standalone, not yet plugged in) for score-based
adaptive result limiting — designed for future Jina Reranker v3 integration
- Extract shared chunkIdUtils for consistent chunk ID → note path conversion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Hardcode maxSourceChunks to 30 and remove user setting
Replace settings.maxSourceChunks with DEFAULT_MAX_SOURCE_CHUNKS (30)
across all retriever call sites. Remove the "Max Sources" slider from
QA settings — recall and precision are now handled automatically by
diverse top-K selection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: expose returnAll to LLM for exhaustive "find all" vault queries
Queries like "find all my AI digests" previously capped at 30 results.
Time-range and tag queries already bypassed this via shouldReturnAll,
but the trigger was mechanical. This exposes the returnAll concept to
both Agent chain (via tool schema + instructions) and CopilotPlus chain
(via planning prompt extraction) so the LLM can set it based on user
intent, returning up to 100 results when appropriate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace raw tool names with human-friendly summaries in reasoning block
Add explicit cases for all tools (getFileTree, getTagList, time tools,
updateMemory, indexVault, youtubeTranscription) so the agent reasoning
block never exposes internal tool names. Default fallbacks now show
"Processing" / "Done" instead of "Calling toolName" / "Completed toolName".
Failure messages also use the friendly summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: align context block tags and make parseContextIntoSegments registry-driven
The context envelope L2 layer was silently dropping URL/media context across
turns due to three compounding issues:
1. Tag mismatch: Mention.ts created <youtube_transcript> blocks but the
contextBlockRegistry expected <youtube_video_context>. Fixed the tag and
inner element (<transcript> → <content>) to match the registry.
2. Missing registration: twitter_content blocks had no registry entry, so
they were invisible to compaction and segment parsing. Added the entry.
3. Hardcoded parser: parseContextIntoSegments only matched <note_context>,
<active_note>, and <prior_context> via hardcoded regexes, silently
ignoring all URL/media block types. Rewrote it to dynamically build its
regex from CONTEXT_BLOCK_TYPES, ensuring any future block type added to
the registry is automatically handled.
Extracted parseContextIntoSegments into src/context/parseContextSegments.ts
as a standalone pure function for direct testability.
Added 19 new tests covering all registered block types, mixed blocks,
stable/unstable flags, and a registry completeness guard test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: context envelope improvements - regeneration, dedup, per-artifact IDs, XML escaping
Phase A: Fix regeneration from loaded chats by lazily reprocessing context
when contextEnvelope is missing (e.g., messages loaded from disk).
Phase B: Replace static segment IDs ("urls", "selected_text", "web_tabs")
with per-artifact IDs from parseContextIntoSegments, enabling accurate
smart referencing in LayerToMessagesConverter.
Phase C: Deduplicate L2 content by segment ID (last-write-wins) to prevent
linear growth when the same artifact appears across multiple turns.
Phase D: Escape note title/basename in XML context blocks for consistency
with URL/YouTube content. Keep path unescaped as it's used as identifier.
Additional fixes from Codex review:
- Unique IDs for blocks without source extractors (selected_text counter)
- Keep note.path raw to avoid dedup mismatch with TFile.path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent context artifact duplication in LLM payload and L4 memory
CopilotPlusChainRunner was re-injecting processedText (which includes
context XML artifacts) into the user message via ensureUserQueryLabel,
bypassing the envelope's clean L2/L3/L5 separation. This caused context
like YouTube transcripts to appear 3x in the LLM payload.
Fix: use L5_USER envelope text (expanded user query without context XML)
instead of processedText for cleanedUserMessage, trimmedQuestion fallback,
and messageForAnalysis. Also fix BaseChainRunner.handleResponse to save
L5 text to L4 memory instead of processedText.
Other changes:
- Remove dead updateMemoryWithLoadedMessages from chainManager
- Update CONTEXT_ENGINEERING.md with example walkthrough, L4 behavior
docs, and Phase 6 integration test suite roadmap
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add chain runner envelope usage section and fix Step 0 log
Document per-runner envelope behavior, CopilotPlus/Agent tool flows,
and token efficiency audit in CONTEXT_ENGINEERING.md. Fix Step 0 log
to show L5 user query instead of processedText with context XML.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: trim trailing whitespace from chat input display text
Trailing newlines from Lexical editor were preserved in displayText
and rendered as empty lines in chat bubbles due to whitespace-pre-wrap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Route Twitter/X URLs (x.com, twitter.com with /status/) to the new
brevilabs twitter4llm endpoint instead of the generic url4llm endpoint.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
LangChain's default getNumTokens() fetches a ~3MB gpt2.json BPE
vocabulary from tiktoken.pages.dev at runtime. When this CDN is
unreachable, all LLM calls (stream, invoke) hang until timeout,
completely blocking the plugin. Override getNumTokens on every model
instance with a simple char-based estimation — modern LLM APIs return
accurate token usage in response metadata anyway.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: modular context compaction architecture
Split context compaction into focused modules:
- contextBlockRegistry: single source of truth for block types
- compactionUtils: shared pure functions for truncation/extraction
- ChatHistoryCompactor: tool result compaction at memory save time
- L2ContextCompactor: L3→L2 context compaction with previews
Key changes:
- Never compact selected_text (non-recoverable content)
- Single consolidated re-fetch instruction at end of L2 context
- Write-time compaction via memoryManager.saveContext()
- Parse prior_context blocks in segment extraction
- Backwards compatibility via re-exports
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: use brace-balanced extraction for readNote JSON parsing
The previous regex-based approach stopped at the first closing brace,
truncating JSON payloads containing nested braces (e.g., code snippets).
This caused JSON.parse to fail and skip compaction entirely.
Now uses a proper brace-balanced parser that correctly handles:
- Nested objects and arrays
- Braces inside string literals
- Escaped characters in strings
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve all documents in localSearch compaction
Previously, compactToolResultBlock used extractContentFromBlock which
only extracts the first <content> match. For localSearch results with
multiple <document> blocks, this caused all documents after the first
to be lost during compaction.
Now localSearch blocks are handled specially:
- All documents are extracted and preserved
- Each document keeps its title and path
- Content is truncated to preview length
- Output lists all documents with [[wikilink]] format
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: skip flaky composer integration tests
Skip composer integration tests due to intermittent 503 errors from
Gemini API causing CI failures. The tests depend on external API
availability which is unreliable.
TODO(@logancyang, @wenzhengjiang): Re-enable once composer is revamped.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve all blocks when compacting multi-block L2 segments
When URL context segments contain multiple concatenated <url_content>
blocks (from processUrlList), the previous compaction only processed
the first block. Now detects multiple blocks of the same type and
compacts each one independently, preserving all URL context.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve mixed XML block types during L2 compaction
The previous fix only handled multiple blocks of the same type. When
segments contain mixed block types (e.g., web_tab_context mixed with
youtube_video_context from processContextWebTabs), blocks of other
types were dropped.
Now uses CONTEXT_BLOCK_TYPES registry to match all known block types
and compacts each one independently, preserving all context regardless
of block type mix.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: improve mobile keyboard/navbar CSS scoping and platform detection
* fix: clear drawer keyboard class when view moves out of drawer
Track the last drawer element so that when the Copilot view is moved
out of a drawer while the soft keyboard is open, the copilot-keyboard-open
class is properly removed from the old drawer, restoring its header/tab UI.
* Fix missing opening <think> tag for models like nvidia/nemotron
Some models output thinking content with only a closing </think> tag
due to misconfigured chat templates in LM Studio. This adds post-processing
to detect and fix this by prepending the opening <think> tag, with a
warning logged to help users identify the template issue.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix agent mode to properly strip thinking tokens from response
Agent mode was showing thinking content in final responses because
streamModelResponse() bypassed ThinkBlockStreamer entirely. Now
integrates ThinkBlockStreamer with excludeThinking=true to properly
filter thinking tokens from all sources (Claude, Deepseek, OpenRouter,
LM Studio text-level <think> tags).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add reasoning/thinking token support for LM Studio
- Use ChatOpenRouter for LM Studio to capture delta.reasoning from streaming responses
- Add enableReasoning and reasoningEffort config for LM Studio models
- Show Reasoning Effort UI for any model with REASONING capability enabled
- Consolidate duplicate OpenRouter reasoning effort UI into unified component
- Translate Chinese comments to English
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Update GPT-5 to GPT-5.2 and add xhigh reasoning effort
- Rename GPT_5 to GPT_5_2 (model name "gpt-5.2")
- Add XHIGH reasoning effort level for GPT-5.2 models
- Update ChatOpenRouter type to support "xhigh" effort
- Show Extra High option only for GPT-5.2 OpenAI models
- Translate Chinese comments to English
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix tool_calls not forwarded in ChatOpenRouter message conversion
The toOpenRouterMessages() method was only handling legacy function_call
but not modern tool_calls format, causing autonomous agent tool flows
to break for LM Studio and OpenRouter models.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add PatternListEditor component for include/exclude settings
Add a new PatternListEditor component to manage include/exclude patterns
in QA settings with support for folders, tags, notes, and file extensions.
- Collapsible list UI with smooth expand/collapse animation
- Dropdown menu for adding different pattern types via modals
- Robust URI decoding with try/catch to prevent settings page crash
- Pattern deduplication via getUniquePatterns helper
- Test coverage for malformed URI sequences
# Conflicts:
# src/styles/tailwind.css
# Conflicts:
# src/styles/tailwind.css
* chore: translate Chinese comments to English in PatternListEditor
* Use openrouter openai embedding small as default embedding model
* Increase agent max iterations to 16 and add 5-minute loop timeout
- Raise max tool call iterations limit from 8 to 16
- Add AGENT_LOOP_TIMEOUT_MS constant (5 minutes) to prevent runaway loops
- Update slider max in settings UI to match new limit
- Show appropriate message when loop exits due to timeout vs max iterations
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Update agent reasoning spinner to sigma shape with smooth animation
- Change from 9-dot diamond to 7-dot sigma (Σ) pattern
- Animation traces sigma shape: top-right → top-left → center → bottom
- Use ease-in-out timing and gradual opacity transitions for smooth motion
- Adjust size to 13.5px for better alignment with text
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Improve agent reasoning step summaries and fix composer tool UX
- Add meaningful summaries for writeToFile/replaceInFile tools
(e.g., "Writing to filename" instead of "Calling writeToFile")
- Show "Edit rejected" when user rejects changes in diff view
- Fix "Changes applied successfully" notice showing on reject
- Extract model's intermediate reasoning as finding summaries
- Add TODO doc for composer tool redesign to address 30s feedback gap
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add self-host mode eligibility gating and permanent validation
- Increase tool timeout from 60s to 120s
- Show self-host mode section only to Believer/Supporter plan users
- Add permanent validation after 3 consecutive successful checks
- Add useIsSelfHostEligible hook for offline-friendly eligibility check
- Preserve permanent validation status when re-enabling self-host mode
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix self-host validation to require 14-day intervals between count increments
- Only increment validation count when 14+ days have passed since last check
- Document need for structured ComposerTools results in TODO doc
- Add TODO for no-op case handling in AgentReasoningState
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix self-host validation timer and offline re-enable issues
- Don't reset timestamp when < 14 days (preserves interval countdown)
- Allow permanently validated users to see section even if toggle is off
- Supports offline re-enable for users with 3+ successful validations
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Wire refreshSelfHostModeValidation into plugin startup
Enables the 14-day periodic validation to actually increment the count
toward permanent self-host mode (3 successful checks).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Improve self-host mode validation with offline support and documentation
- Add grace period check in validateSelfHostMode for offline re-enable
- Change grace period from 14 to 15 days
- Add comprehensive documentation explaining validation flow
- Fix offline re-enable for users within grace period
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Update ollama chat model to fix thinking
* fix: consolidate tool call parsing and add actionable error logging
- Remove duplicate tool call parsing in AutonomousAgentChainRunner
by using shared buildToolCallsFromChunks() function
- Add sanitizeToolArgs() to remove empty objects from tool args
(fixes Ollama returning timeRange: {} which fails Zod validation)
- Add actionable logging for tool call failures with full args context
- Clean up unused parameters in AutonomousAgentChainRunner
- Simplify ToolManager.callTool to throw instead of swallowing errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: add repeatPenalty for Ollama to reduce repetitive outputs
Local models via Ollama are prone to repetitive hallucination loops.
Adding repeatPenalty=1.1 applies a slight penalty to repeated tokens,
which helps reduce this pattern.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: increase Ollama context window to 128k
Default numCtx of 8192 causes prompt truncation with larger contexts.
Setting numCtx=131072 (128k) to support models with large context.
Ollama automatically caps at model's actual maximum if exceeded.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: handle Ollama empty object tool args at schema level
- Make timeRange inner fields (startTime, endTime) optional in schema
- Update validateTimeRange to handle empty or partial objects from LLMs
- Apply validation in all search tools (local, lexical, semantic)
- Remove sanitizeToolArgs function that was too aggressive
- Fixes tool calls that legitimately accept {} as valid input
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore: translate Chinese comments to English and apply formatting
- Translate all Chinese comments in replaceGuard.ts to English for consistency
- Apply Prettier formatting to 33 files
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: add unit tests for Quick Ask core modules
Add comprehensive tests for:
- replaceGuard.ts: validation logic, error messages, MapPos and Highlight strategies
- persistentHighlight.ts: CM6 state field, range mapping, show/hide behavior
- quickCommandPrompts.ts: placeholder appending logic
64 new test cases covering edge cases like:
- Leaf/editor/file change detection
- Document bounds validation
- Content change detection
- Range mapping through document changes
- Case-insensitive placeholder detection
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: implement floating quick command with AI-powered inline editing
Add a new floating quick command feature that allows users to interact
with AI directly from the editor. Key features include:
- Quick Ask panel with draggable modal interface
- Multiple interaction modes: ask, edit, and edit-direct
- Selection highlight and replace guard for safe text operations
- Streaming chat session support for real-time AI responses
- Context menu integration for quick access
- Resizable and draggable UI components
# Conflicts:
# src/commands/CustomCommandChatModal.tsx
# src/styles/tailwind.css
* feat: add persistent selection highlight for Chat panel
* refactor: extract persistent highlight factory and fix state consistency
* fix: improve popup positioning with flip logic and line-start trap fix
- Use selection.head instead of selection.to for anchor point
- Fix line-start trap: only apply when head is selection end
- Add flip logic: show above when not enough space below
- Add horizontal visibility check
- Single-line selection: center popup horizontally
- Multi-line selection: follow head position
- Cursor mode: align near cursor instead of centering
When the typeahead menu is triggered but the query matches nothing,
the menu visually disappears but state.isOpen remains true. Arrow key
handlers were checking only isOpen, intercepting events even with no
options to navigate. This trapped the cursor.
Add early return (return false) in ArrowDown/ArrowUp handlers when
options.length === 0, consistent with how Enter/Tab already handle
this case. Arrow keys now propagate normally for cursor movement
when there are no typeahead matches.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Implement self-host mode
* Migrate from XML tool calls to native tool calls and reimplement agent mode
* Use ChatOpenRouter for copilot-plus-flash for native tool call sse support
* Fix QA exclusion for search v3
* Update plan and remove debug messages in agent
* Migrate chains off of xml and clean up related logic
* Implement agent reasoning block
* Update css for responsiveness
* Implement agent query pre-expansion and proper reasoning block display for search
* Update agent docs
* Refine agent reason block
* Refine agent reasoning UX
* Fallback to plus non-agent if native tool call is not supported by the model
* Fix time filter query
* Update self-host description
* Skip agent reasoning block and old tool call banners during chat save and load
* Fix projects mode switch to plus and chat auto saves as non-project bug
* fix: increase grep limit for larger vaults and enhance search prioritization
* fix: unify chunk management over index-free and semantic search
* fix: batch process files in prepareAllChunks to bypass ChunkManager's 1000-file limit
* fix: update regex to correctly find closing frontmatter delimiter at EOF
* fix: enhance security checks to prevent path traversal vulnerabilities
* fix: refactor to use shared ChunkManager instance across components for improved cache management
* fix: enhance cache retrieval in ChunkManager to support non-default chunk options
* fix: record usage when selecting slash command from input
Slash commands selected via "/" in the chat input were not updating
lastUsedMs, causing recency sorting to not work properly. Other entry
points (command palette, context menu) already called recordUsage.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add default indicator for system prompts
- Rename "(Global)" to "(Default)" in system prompt dropdown labels
- Add "(Default)" marker in AdvancedSettings for selected default prompt
- Sync default flag to frontmatter when default system prompt changes
- Handle edge case when folder and default title change simultaneously
- Add logWarn when prompt file not found during default flag update
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
