The Miyo toggle is only visible when Self-Host Mode is already enabled,
which requires passing validateSelfHostMode() first. Calling it again
when enabling Miyo was redundant and could incorrectly block users who
are offline after their grace period expires mid-session.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The keyboard shortcuts for Replace (Ctrl/Cmd+Enter) and Insert
(Ctrl/Cmd+Shift+Enter) were lost during the Quick Ask refactor. This
restores them with proper popout-window and multi-modal scoping.
Fixes#2219
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When "Auto-Add Selection to Context" was used, changing the selection
and sending a new message would still show the old selection to the LLM.
The root cause was that selected_text / web_selected_text segments from
previous turns were promoted into L2 context verbatim, shadowing the
current turn's fresh context.
Fix: in compactSegmentForL2(), strip non-recoverable blocks (derived
from contextBlockRegistry) and prior_context_note before compacting.
Uses a "remove blacklist" approach that preserves unknown tags and
inter-block text. Also tighten hasCompactedContent detection to require
the source= attribute, avoiding false-positives from literal XML in
user note content.
* fix: close Radix portaled layers when mobile drawer hides
Radix popover/dropdown portals to document.body, causing them to jump
to (0,0) when the mobile drawer hides and the anchor element disappears.
- Add hideWhenDetached on PopoverContent and DropdownMenuContent for mobile
- Observe drawer is-hidden class change and dispatch Escape to close state
- Add cancelable: true to synthetic Escape for correct preventDefault semantics
* fix: reattach drawer hide observer when view moves between containers
The drawer hide observer was bound once during onOpen, but the view can
move between containers without onOpen firing again. Listen for
layout-change events to re-bind the observer to the current drawer.
Buffer.byteLength is unavailable on Obsidian mobile (Android/iOS WebView),
causing indexing to fail with "Buffer is not defined". Use the existing
MemoryManager.getByteSize() helper which uses TextEncoder instead.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* 3.2.3
* fix: rename "Copy command" to "Duplicate command" with copy-plus icon (#2150)
Improves UX clarity by distinguishing the duplicate action from clipboard copy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace native fetch() with safeFetchNoThrow() in makeRequest() so API
calls route through Obsidian's requestUrl (Electron/Node layer) instead
of Chromium's network stack, eliminating intermittent CORS failures.
makeFormDataRequest retains native fetch() because safeFetch hardcodes
Content-Type: application/json and calls body.toString(), which breaks
FormData payloads.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add a user-configurable setting to save converted document markdown
(from pdf4llm/docs4llm) to a specified vault folder. Previously,
converted content was only stored in internal cache with opaque names.
Also routes PDFs through Miyo in project mode when self-host is enabled,
preserving privacy by never falling back to cloud APIs in self-host mode.
Changes:
- New "Store converted markdown at" setting under Document Processor
(empty = disabled, suggested: copilot/converteddocs)
- Flat file naming ({basename}.md) with double-underscore disambiguation
for collisions, source-header traceability, and content-dedup to avoid
mtime churn
- Export on cache hits so enabling the setting retroactively works
- SelfHostPdfParser in Docs4LLMParser for Miyo-first PDF parsing in
project mode
- MiyoParseResult type (null | {content} | {error}) propagates detailed
Miyo errors to the user instead of generic failure messages
- Self-host privacy: PDF parse failures return an error with details
rather than silently uploading to cloud API
- TODO comments for request timeout, batch progress feedback
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When Miyo returns a non-"ok" status (e.g. "degraded"), isBackendAvailable
silently returned false with no log, making users think the health check
passed while still seeing the "Miyo app is not available" notification.
Now logs the actual status unconditionally so users can diagnose why
availability check failed. Also removes the debug-only guard on the
catch block so network errors are always surfaced.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When defaultSaveFolder is a hidden directory (e.g. .copilot/conversations),
Obsidian's metadata cache doesn't index it, causing saves to fail and
chat history to show empty.
Add adapter-level fallback utilities in src/utils/vaultAdapterUtils.ts:
- resolveFileByPath: vault lookup with synthetic TFile fallback
- listMarkdownFiles: folder listing with adapter fallback
- patchFrontmatter: processFrontMatter with adapter YAML patching fallback
- readFrontmatterViaAdapter: parse frontmatter from raw file content
Apply fallbacks across ChatPersistenceManager (save, load, list, find)
and main.ts (history picker, load, delete, title update, lastAccessedAt).
Also removes obsolete design docs from docs/.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Miyo PR review findings (privacy, dedup, architecture)
- Gate request body logging behind debug flag to prevent note content
leaking to vault log files on every upsert (P1 privacy fix)
- Replace duplicated grace period logic in findRelevantNotes with
isSelfHostAccessValid() from plusUtils (P1 dedup fix)
- Spread-copy metadata in fromMiyoDocument to avoid mutating API response
- Remove dead if(enabled) branch in CopilotPlusSettings toggle handler
- Gate per-document upsert log lines behind debug to prevent log churn
- Rename misleading test; add no-embeddings fallback path test coverage
- Add TODO for cross-platform service discovery paths (Windows/Linux)
- Use this.client directly in MiyoSemanticRetriever.getExplicitChunks
instead of VectorStoreManager singleton to avoid backend mismatch
- Remove unused apiKeyOverride param from buildHeaders; document dual auth
- Read pagination total only from first response in getIndexedFiles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address PR review comments on Miyo retriever and index backend
- Guard response.documents with ?? [] in getExplicitChunks to prevent
crash when backend omits the field or parseResponseJson returns {}
- Remove redundant getSettings().debug guard around logInfo calls since
the logging utilities already respect the debug flag internally
- Document the debug-flag behavior in AGENTS.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor: remove explicit path reads from Miyo semantic retriever
* fix: rename remaining enableMiyoSearch references
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename `enableMiyoSearch` → `enableMiyo` across settings, UI, and all consumers
- Skip fulltext search in performLexicalSearch (Plus/agent path) when Miyo is active;
filterRetriever still runs for tag/title matching
- VaultQA chain now passes `{ enableMiyo: false }` to RetrieverFactory so it never
routes through Miyo
- Expose `RetrieverFactory.isMiyoActive()` for clean Miyo-state checks
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: upgrade @langchain/google-genai to fix Gemini 3 preview image support (#2178)
The @langchain/google-genai v1.0.0 had a hardcoded _isMultimodalModel check
that only recognized gemini-1.5, gemini-2, and "vision" model names. Gemini 3
preview models (gemini-3-flash-preview, gemini-3-pro-preview) were rejected
with "This model does not support images" when sending image content.
Upgrade to v1.0.2 which adds gemini-3 to the multimodal model pattern.
Also bump @google/generative-ai from ^0.21.0 to ^0.24.0 (required dep).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update built-in Gemini Pro to 3.1 preview model IDs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Firecrawl and Supadata self-host support for web search and YouTube
Self-host mode users can now use their own API keys for web search (Firecrawl)
and YouTube transcripts (Supadata) instead of routing through the Brevilabs
backend. Tools gracefully fall back to Brevilabs when no self-host key is set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Perplexity Sonar as alternative self-host web search provider
Add provider dispatch pattern for self-host web search, supporting both
Firecrawl and Perplexity Sonar via a dropdown selector in settings UI.
Changes:
- Refactor selfHostWebSearch into provider dispatch (firecrawl/perplexity)
- Add hasSelfHostSearchKey() helper for provider-agnostic key checks
- Add settings UI dropdown for web search provider selection
- Fix Mention.ts YouTube transcription to route via Supadata in self-host mode
- Fix cached failed URL results never being retried (retry-on-error)
- Clear Miyo Search state when self-host mode is disabled
- Add 19 unit tests for selfHostServices
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Firecrawl and Supadata self-host support for web search and YouTube
Self-host mode users can now use their own API keys for web search (Firecrawl)
and YouTube transcripts (Supadata) instead of routing through the Brevilabs
backend. Tools gracefully fall back to Brevilabs when no self-host key is set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: drop legacy Perplexity wrapper from self-host web search
selfHostWebSearch now returns { content, citations } directly instead of
wrapping results in the OpenAI-chat-completion shaped WebSearchResponse
that Brevilabs inherited from Perplexity. SearchTools unwraps each path
independently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move Miyo Search under self-host conditional and improve description
Miyo Search toggle is now only visible when self-host mode is enabled,
grouped alongside Firecrawl and Supadata settings. Updated description
to be less technical and highlight vault size advantage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add token budget enforcement analysis and fix plan
Document root cause of context window overflow (2.7M tokens sent to 1M
model): L4 chat history bypasses all compaction systems. Add fix plan
with phased approach to enforce token budget at message assembly point.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: correct root cause analysis — L1 unbudgeted, not L4
Previous analysis incorrectly blamed L4 chat history as the primary
culprit. Investigation shows L4 stores only bare L5 text + compacted
responses. The real issue is systemic: no total payload enforcement,
with L1 (project context) being the largest unbudgeted layer and
PROJECT_COMPACT_THRESHOLD being blind to L1 size.
Updated fix plan to be model-agnostic (use autoCompactThreshold as
single budget, no model-specific lookup tables), added contextTurns
deprecation, and history guarantee principle.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: hard cap L1 project context at 600k tokens + payload diagnostics
- Truncate project context at 2.4M chars (~600k tokens) to prevent
total payload from exceeding model context windows (temporary fix
until full token budget enforcement is implemented)
- Add per-layer token estimate logging when payload exceeds 2M chars
to help diagnose context window overflow reports
- Document Obsidian CLI dev tools in CLAUDE.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix "Cannot set properties of undefined (setting 'embedding')" during indexing
Rebuild internalDocumentIDStore after loading partitioned chunks to eliminate
ghost IDs from removed documents that cause position mismatches. Also add
missing nchars field to Orama schema to match the OramaDocument interface.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove Notice popup spam for per-file indexing errors
Errors during indexing (e.g. "Semantic index database not found" when DB
isn't loaded yet) were triggering a Notice popup for every file event,
flooding the user with popups. Errors are already logged to console and
surfaced in the in-chat IndexingProgressCard.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix unwanted indexing when semantic search disabled + editor lag during indexing
Add defense-in-depth enableSemanticSearchV3 guard to both VectorStoreManager
and IndexOperations to prevent indexing when semantic search is off (fixes
plusUtils.applyPlusSettings bypass). Throttle Jotai atom updates to 500ms
intervals during indexing to reduce React re-renders. Yield to main thread
between batches to keep the editor responsive.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove yieldToMainThread — existing awaits already yield to the event loop
The batch loop already has multiple await points (rateLimiter.wait,
embedDocuments, upsert, save) that yield to the main thread. The explicit
setTimeout(0) was redundant and added no user-perceived benefit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard throttled callback against stale indexing sessions
Cancel pending throttle timer in resetIndexingProgressState() so a
trailing write from a previous (failed/aborted) run cannot overwrite
the freshly-reset atom state of a new run. Also reorder declarations
so throttle variables are defined before the function that references them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move badge close button to left icon overlay on hover
Centralize the icon + X overlay pattern in ContextBadgeWrapper instead of
copy-pasting the X button across all 8 badge types. The X now appears on
hover over the left icon position using invisible/visible toggling for
zero layout shift.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* support mobile
* fix keyboard control
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add optional streamUsage setting to CustomModel interface that allows users
to control whether stream_options is sent to the API. This fixes compatibility
with 3rd party providers (e.g., Databricks, MLFlow) that do not support the
stream_options parameter.
- Add streamUsage?: boolean to CustomModel interface in aiParams.ts
- Pass streamUsage to ChatOpenAI config for OPENAI_FORMAT and LM_STUDIO providers
- Add UI toggle in ModelAddDialog and ModelEditDialog with helpful tooltip
- Default to false to maintain compatibility with providers that error on stream_options
Closes#2046
Sanitize LLM-generated file paths before creating files to prevent
ENAMETOOLONG crashes on Linux (ext4 255-byte filename limit). Truncates
overlong basenames while preserving file extensions and multi-byte
character boundaries. Sanitization happens at the tool entry point so
the preview flow (ApplyView) also receives the safe path.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* Replace indexing Notice with in-chat progress card & fix phantom re-indexing
Replace the Obsidian Notice popup for indexing progress with a persistent
in-chat progress card using the same Jotai atom pattern as project mode.
Key changes:
- New IndexingProgressCard component with progress bar, pause/resume/stop,
error display, and auto-close on completion
- Jotai atom (indexingProgressAtom) driven by IndexOperations, read by React
- Card only appears when files actually need indexing (no flash on mode switch)
- User-initiated actions (refresh/reindex buttons) show "Index Up to Date"
feedback with green check icon
- Fix phantom re-indexing: checkIndexIntegrity() is now diagnostic-only
- Fix progress bar accuracy: totalFiles counted after chunk preparation
- Remove duplicate completion Notices from ChatControls and RelevantNotes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix insert/replace at cursor including agent reasoning blocks (#2174)
Strip <!--AGENT_REASONING:...--> markers in cleanMessageForCopy() so
that "Insert / Replace at cursor" only inserts the actual AI response,
not the internal agent reasoning metadata.
Uses greedy .* (single-line) so the regex matches to the real closing
--> even if the JSON payload contains that sequence.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Restore integrity check marking files for re-indexing
checkIndexIntegrity() was made diagnostic-only to fix phantom
re-indexing, but this left files with missing embeddings permanently
skipped. The real safeguard is clearFilesMissingEmbeddings() at the
start of each indexing run, which prevents the repeat cycle. Restore
the mark so the next indexing trigger picks up integrity failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Strip <!--AGENT_REASONING:...--> markers in cleanMessageForCopy() so
that "Insert / Replace at cursor" only inserts the actual AI response,
not the internal agent reasoning metadata.
Uses greedy .* (single-line) so the regex matches to the real closing
--> even if the JSON payload contains that sequence.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Simplify search pipeline: remove expandedTerms, use metadata-cache tag matching
- Remove `expandedTerms` concept from QueryExpander, FullTextEngine, SearchCore,
and all interfaces — LLM expansion now only produces `salientTerms` and
`expandedQueries`, no separate related-term list
- Remove weighted 90/10 two-phase scoring from FullTextEngine; use single BM25+
search with salient terms for ranking
- Replace tag-based `returnAll` mode with Obsidian metadata-cache tag matching:
`getTagMatches()` uses `getAllTags()` with hierarchical prefix matching and
injects tag-matched notes as guaranteed full-note documents (score 1.0)
- Remove `buildTagRecallQueries()` tag-hierarchy splitting from SearchCore
- Add `isGrepWorthy()` filter in GrepScanner to skip short ASCII terms (< 3 chars)
- Reduce default candidateLimit and grepLimit from 500 to 200
- Rewrite search v3 README to reflect current implementation
- Update all tests for removed fields and new tag-match injection behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix isGrepWorthy filtering out valid 2-char terms like "AI", "ML", "UI"
SearchCore lowercases all recall terms before passing to GrepScanner, so
acronym detection by casing is impossible. Lower the threshold to filter
only single-char terms — grep is capped at 200 results anyway, and BM25+
handles precision downstream.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Extract FilterRetriever from TieredLexicalRetriever for guaranteed filter matching
Move deterministic title/tag/time-range matching into standalone FilterRetriever
class that runs at the orchestration layer (SearchTools, VaultQAChainRunner,
customSearchDB). This ensures filter results bypass all downstream retriever
processing (top-K slicing, score normalization) and reach the LLM context via
separate <filterResults>/<searchResults> XML sections with continuous ID numbering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Cap FilterRetriever tag and time-range results to maxK/RETURN_ALL_LIMIT
Prevents unbounded filter result sets (e.g. common tags like #daily) from
blowing up prompt/context size and read latency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Clean up legacy parser, dead code, and DRY violations
- Remove parseLegacyFormat from QueryExpander (fall back to fallbackExpansion for non-XML LLM responses)
- Remove dead fuzzy variant generation in fallbackExpansion
- Remove deprecated expandQueries backward-compat method
- Extract shared mapDoc helper in SearchTools to eliminate duplicated document mapping
- Use existing toIsoString helper in formatSplitSearchResultsForLLM
- Add FilterRetriever cap regression tests for maxK and RETURN_ALL_LIMIT
- Convert legacy-format tests to XML format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Restore expanded limits for tag queries in VaultQAChainRunner
Pass returnAll: true when tagTerms are present so FilterRetriever
and the main retriever use RETURN_ALL_LIMIT instead of maxK cap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Cap merged filter+search results to prevent oversized prompts
Filter and search results are now capped post-merge (filter results
prioritized) to prevent 100-200 doc payloads on tag-heavy queries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add retrieval cap notice in VaultQA to prompt Copilot Plus upgrade
When merged filter+search results exceed the retrieval cap, the AI
is instructed to inform the user that some documents were omitted
and suggest upgrading to Copilot Plus for more complete answers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add RELEASES.md with full release history and v3.2.0 draft
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* 3.2.0
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Show Self-Host Mode section to all users with disabled toggle for non-lifetime
The Self-Host Mode settings section is now visible to everyone instead of
being hidden behind an eligibility check. The toggle is disabled (greyed out)
for users who are not on a lifetime plan (believer/supporter), making the
feature discoverable while preserving the access restriction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix self-host toggle not disabled after removing license key
Three issues fixed:
1. useIsSelfHostEligible() checked grace period before license key,
so removing the key didn't revoke eligibility while in grace period.
Moved license key check to the top and added auto-revocation of
enableSelfHostMode when key is absent.
2. isSelfHostModeValid() didn't check license key at all, so backend
code (RetrieverFactory, isPlusEnabled) continued to grant self-host
access after key removal. Added license key guard.
3. useIsPlusUser() self-host bypass didn't check license key. Added
the same guard for consistency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix self-host toggle not disabled when switching to non-lifetime key
useIsSelfHostEligible() trusted cached grace period / permanent
validation before consulting the API, so swapping from a believer
key to a plus key kept the toggle enabled.
Now always verifies via API when a license key is present. Cached
validation (grace period, permanent count) is only used as an
offline fallback when the API call fails. Also auto-revokes
enableSelfHostMode when the API confirms the key is not eligible.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Replace warning triangle with Build Index button in Relevant Notes
Improves UX when semantic search is off or index is unavailable by replacing
the yellow warning triangle and text with a clear "Build Index" button that
enables semantic search (with confirmation modal) and triggers vault indexing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add reactive index signal and move Build Index button to header
- Create indexSignal pub-sub so UI reacts to external index changes
(clear, rebuild, reindex) without manual refresh
- Fire notifyIndexChanged() from VectorStoreManager (single source)
- Move Build Index button to header area so it's always visible when
!hasIndex, even when link-based relevant notes are showing
- Update help tooltip to mention semantic search requirement
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
HyDE (Hypothetical Document Embeddings) rewrites user queries into
hypothetical answer passages before embedding them for vector search.
This produces garbage for non-question queries like "find all my ai
digests" — the LLM generates an instructional essay instead of useful
search terms, polluting semantic results and causing Plus mode to miss
relevant notes that Agent mode finds via multiple search iterations.
Removing HyDE:
- Eliminates an extra LLM call per search (faster responses)
- Uses the raw query for vector embedding search (better matching)
- QueryExpander already handles LLM-based query expansion correctly
- hybridRetriever.ts is already marked DEPRECATED
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Add Claude Opus 4.6, Sonnet 4.5 (replacing Sonnet 4)
- Add Gemini 3 Flash/Pro preview (Google direct + OpenRouter)
- Add GPT-5.3 Codex, upgrade OpenRouter GPT to 5.2/5-mini
- Upgrade Grok 4-fast to 4.1-fast (xAI + OpenRouter)
- Remove Gemini 2.5 Flash Lite, demote GPT-4.1 to non-core
- Preview models are not core or project-enabled
- Extend thinking model detection for Claude Opus 4.x
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve search recall with note-diverse top-K, chunk-aware scoring, and adaptive cutoff module
- Add note-diverse top-K selection so every unique note gets representation
before any note gets a second chunk slot (fixes "find my ai digests" missing notes)
- Fix GraphBoostCalculator: resolve chunk IDs to note paths for metadata lookups,
deduplicate to note level in filterCandidates
- Fix FolderBoostCalculator: count unique notes per folder instead of raw chunks
- Bump title field weight 3→5 for stronger filename match priority
- Add AdaptiveCutoff module (standalone, not yet plugged in) for score-based
adaptive result limiting — designed for future Jina Reranker v3 integration
- Extract shared chunkIdUtils for consistent chunk ID → note path conversion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Hardcode maxSourceChunks to 30 and remove user setting
Replace settings.maxSourceChunks with DEFAULT_MAX_SOURCE_CHUNKS (30)
across all retriever call sites. Remove the "Max Sources" slider from
QA settings — recall and precision are now handled automatically by
diverse top-K selection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: expose returnAll to LLM for exhaustive "find all" vault queries
Queries like "find all my AI digests" previously capped at 30 results.
Time-range and tag queries already bypassed this via shouldReturnAll,
but the trigger was mechanical. This exposes the returnAll concept to
both Agent chain (via tool schema + instructions) and CopilotPlus chain
(via planning prompt extraction) so the LLM can set it based on user
intent, returning up to 100 results when appropriate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace raw tool names with human-friendly summaries in reasoning block
Add explicit cases for all tools (getFileTree, getTagList, time tools,
updateMemory, indexVault, youtubeTranscription) so the agent reasoning
block never exposes internal tool names. Default fallbacks now show
"Processing" / "Done" instead of "Calling toolName" / "Completed toolName".
Failure messages also use the friendly summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: align context block tags and make parseContextIntoSegments registry-driven
The context envelope L2 layer was silently dropping URL/media context across
turns due to three compounding issues:
1. Tag mismatch: Mention.ts created <youtube_transcript> blocks but the
contextBlockRegistry expected <youtube_video_context>. Fixed the tag and
inner element (<transcript> → <content>) to match the registry.
2. Missing registration: twitter_content blocks had no registry entry, so
they were invisible to compaction and segment parsing. Added the entry.
3. Hardcoded parser: parseContextIntoSegments only matched <note_context>,
<active_note>, and <prior_context> via hardcoded regexes, silently
ignoring all URL/media block types. Rewrote it to dynamically build its
regex from CONTEXT_BLOCK_TYPES, ensuring any future block type added to
the registry is automatically handled.
Extracted parseContextIntoSegments into src/context/parseContextSegments.ts
as a standalone pure function for direct testability.
Added 19 new tests covering all registered block types, mixed blocks,
stable/unstable flags, and a registry completeness guard test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: context envelope improvements - regeneration, dedup, per-artifact IDs, XML escaping
Phase A: Fix regeneration from loaded chats by lazily reprocessing context
when contextEnvelope is missing (e.g., messages loaded from disk).
Phase B: Replace static segment IDs ("urls", "selected_text", "web_tabs")
with per-artifact IDs from parseContextIntoSegments, enabling accurate
smart referencing in LayerToMessagesConverter.
Phase C: Deduplicate L2 content by segment ID (last-write-wins) to prevent
linear growth when the same artifact appears across multiple turns.
Phase D: Escape note title/basename in XML context blocks for consistency
with URL/YouTube content. Keep path unescaped as it's used as identifier.
Additional fixes from Codex review:
- Unique IDs for blocks without source extractors (selected_text counter)
- Keep note.path raw to avoid dedup mismatch with TFile.path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent context artifact duplication in LLM payload and L4 memory
CopilotPlusChainRunner was re-injecting processedText (which includes
context XML artifacts) into the user message via ensureUserQueryLabel,
bypassing the envelope's clean L2/L3/L5 separation. This caused context
like YouTube transcripts to appear 3x in the LLM payload.
Fix: use L5_USER envelope text (expanded user query without context XML)
instead of processedText for cleanedUserMessage, trimmedQuestion fallback,
and messageForAnalysis. Also fix BaseChainRunner.handleResponse to save
L5 text to L4 memory instead of processedText.
Other changes:
- Remove dead updateMemoryWithLoadedMessages from chainManager
- Update CONTEXT_ENGINEERING.md with example walkthrough, L4 behavior
docs, and Phase 6 integration test suite roadmap
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add chain runner envelope usage section and fix Step 0 log
Document per-runner envelope behavior, CopilotPlus/Agent tool flows,
and token efficiency audit in CONTEXT_ENGINEERING.md. Fix Step 0 log
to show L5 user query instead of processedText with context XML.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: trim trailing whitespace from chat input display text
Trailing newlines from Lexical editor were preserved in displayText
and rendered as empty lines in chat bubbles due to whitespace-pre-wrap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Route Twitter/X URLs (x.com, twitter.com with /status/) to the new
brevilabs twitter4llm endpoint instead of the generic url4llm endpoint.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
LangChain's default getNumTokens() fetches a ~3MB gpt2.json BPE
vocabulary from tiktoken.pages.dev at runtime. When this CDN is
unreachable, all LLM calls (stream, invoke) hang until timeout,
completely blocking the plugin. Override getNumTokens on every model
instance with a simple char-based estimation — modern LLM APIs return
accurate token usage in response metadata anyway.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: modular context compaction architecture
Split context compaction into focused modules:
- contextBlockRegistry: single source of truth for block types
- compactionUtils: shared pure functions for truncation/extraction
- ChatHistoryCompactor: tool result compaction at memory save time
- L2ContextCompactor: L3→L2 context compaction with previews
Key changes:
- Never compact selected_text (non-recoverable content)
- Single consolidated re-fetch instruction at end of L2 context
- Write-time compaction via memoryManager.saveContext()
- Parse prior_context blocks in segment extraction
- Backwards compatibility via re-exports
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: use brace-balanced extraction for readNote JSON parsing
The previous regex-based approach stopped at the first closing brace,
truncating JSON payloads containing nested braces (e.g., code snippets).
This caused JSON.parse to fail and skip compaction entirely.
Now uses a proper brace-balanced parser that correctly handles:
- Nested objects and arrays
- Braces inside string literals
- Escaped characters in strings
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve all documents in localSearch compaction
Previously, compactToolResultBlock used extractContentFromBlock which
only extracts the first <content> match. For localSearch results with
multiple <document> blocks, this caused all documents after the first
to be lost during compaction.
Now localSearch blocks are handled specially:
- All documents are extracted and preserved
- Each document keeps its title and path
- Content is truncated to preview length
- Output lists all documents with [[wikilink]] format
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: skip flaky composer integration tests
Skip composer integration tests due to intermittent 503 errors from
Gemini API causing CI failures. The tests depend on external API
availability which is unreliable.
TODO(@logancyang, @wenzhengjiang): Re-enable once composer is revamped.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve all blocks when compacting multi-block L2 segments
When URL context segments contain multiple concatenated <url_content>
blocks (from processUrlList), the previous compaction only processed
the first block. Now detects multiple blocks of the same type and
compacts each one independently, preserving all URL context.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve mixed XML block types during L2 compaction
The previous fix only handled multiple blocks of the same type. When
segments contain mixed block types (e.g., web_tab_context mixed with
youtube_video_context from processContextWebTabs), blocks of other
types were dropped.
Now uses CONTEXT_BLOCK_TYPES registry to match all known block types
and compacts each one independently, preserving all context regardless
of block type mix.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: improve mobile keyboard/navbar CSS scoping and platform detection
* fix: clear drawer keyboard class when view moves out of drawer
Track the last drawer element so that when the Copilot view is moved
out of a drawer while the soft keyboard is open, the copilot-keyboard-open
class is properly removed from the old drawer, restoring its header/tab UI.
