* fix(anthropic): use thinking.type=adaptive for claude-opus-4-7+
claude-opus-4-7 rejects the legacy thinking shape with a 400:
`thinking.type.enabled` is not supported for this model.
Use `thinking.type.adaptive` and `output_config.effort`.
Detect opus-4 minor version in getModelInfo and emit
{ type: "adaptive" } for >=7, keeping { type: "enabled", budget_tokens }
for opus-4-6 and earlier, sonnet-4, and 3-7-sonnet.
Bumps @langchain/anthropic ^1.0.0 -> ^1.3.29 to pick up the adaptive
ThinkingConfigParam variant and an updated tool-schema typing that lets
BedrockChatModel.convertTools drop its now-redundant Record cast.
Fixes#2361. Replaces stale PR #2362.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(bedrock): use thinking.type=adaptive for claude-opus-4-7+
Same 400 from claude-opus-4-7+ as the direct Anthropic path (fixed in
the previous commit), but the Bedrock provider has its own payload
builder in BedrockChatModel.buildRequestBody that was unaffected.
Add the same minor-version detection inside that payload builder.
Uses an unanchored regex /claude-opus-4-(\d+)/ because Bedrock model
IDs carry a provider/profile prefix (e.g.
"global.anthropic.claude-opus-4-7-20260115-v1:0").
Fixes#2384.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: use real sonnet model IDs
claude-sonnet-4-7 doesn't exist; tests were asserting against a
fabricated ID. Switch the non-opus negative cases to a real
claude-sonnet-4-5 (and keep the existing claude-3-7-sonnet test).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(thinking): set output_config.display=summarized for claude-opus-4-7+
Adaptive thinking on Opus 4.7+ alone restores the request shape but
leaves the UI empty — Anthropic flipped the default for
output_config.display from "summarized" to "omitted" on this model
line, so the API stops emitting visible thinking content blocks.
Pre-4.7 models still default to "summarized" server-side.
Set display: "summarized" alongside thinking: { type: "adaptive" } on
both the direct Anthropic and Bedrock paths so opus-4-7 renders
thinking summaries the same way sonnet 4.6 does today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(thinking): nest display=summarized inside thinking config
The Anthropic API places `display: 'summarized' | 'omitted'` on the
thinking config itself (ThinkingConfigAdaptive / ThinkingConfigEnabled),
not on a top-level `output_config`. The previous commit set
`outputConfig: { display: "summarized" }` on the ChatAnthropic config and
`output_config: { display: "summarized" }` on the Bedrock request body,
which broke the TS build (the SDK's OutputConfig only exposes
effort/format) and would not have produced the intended server behavior
for the adaptive thinking summarization toggle.
- chatModelManager.ts: emit `thinking: { type: "adaptive", display: "summarized" }`
- BedrockChatModel.ts: same, on the request payload
- BedrockChatModel.test.ts: update assertions to the new shape
* fix(thinking): constrain Opus minor regex so dated snapshots don't match
The previous `^claude-opus-4-(\d+)` (Anthropic) and unanchored
`claude-opus-4-(\d+)` (Bedrock) regex captured the date portion of
snapshot IDs like `claude-opus-4-20250514` as if it were the minor
version, so `usesAdaptiveThinking` evaluated as `true` (20250514 >= 7)
for pre-4.7 Opus models. That would send `thinking: { type: "adaptive" }`
for the legacy 4.0 snapshot and could trigger 400s on REASONING-enabled
runs.
Constrain the minor capture to 1-2 digits followed by `-`/`.` or end of
string. Real Anthropic minor versions are single/double digit; dated
snapshots have an 8-digit date that this regex now rejects.
- src/utils.ts: anchored fix for the Anthropic path
- src/LLMProviders/BedrockChatModel.ts: unanchored fix for Bedrock IDs
- regression tests for dated 4.0/4.1 snapshots in both src/utils.test.ts
and src/LLMProviders/BedrockChatModel.test.ts
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(deps): drop @langchain/community, hand-roll Jina embeddings
Replaces the single use of @langchain/community (JinaEmbeddings) with a small
hand-rolled implementation that extends @langchain/core/embeddings directly and
calls the Jina /embeddings endpoint via Obsidian's requestUrl (CORS-safe). The
request shape and response handling mirror the upstream Python reference.
* chore(jina): adapt upstream community implementation with attribution
Restores batching, dimensions/normalization defaults, and the multi-modal
input type from the original @langchain/community JinaEmbeddings, with
upstream copyright notice and MIT attribution preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(deps): bump openai to ^6.10.0 to dedupe with @langchain/openai
@langchain/openai@1 pins openai@^6.10.0 while we pinned ^4.95.1, so
esbuild was shipping both copies. Aligning to v6 dedupes the bundle.
Stacks on #2461, which drops @langchain/community (and its stagehand
peer that pinned openai@^4.62.1), so npm install no longer needs
--legacy-peer-deps to resolve.
ChatOpenRouter.ts is the only direct consumer; its imports
(OpenAI default, ChatCompletionChunk/MessageParam/Role types,
chat.completions.create streaming) are all unchanged in v6.
main.js: 3,447,540 -> 3,371,972 bytes (-75 KB).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the "off" overrides for depend/ban-dependencies, configures the
package.json check with an allowed-list for deps we deliberately keep
(crypto-js, lodash.debounce, eslint-plugin-react, lint-staged, npm-run-all),
and drops three unused direct deps: axios, builtin-modules, and node-fetch
(plus @types/node-fetch). The integration-test fetch polyfills are no longer
needed in Node 18+ jsdom.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the deferred "off" overrides from eslint.config.mjs so the
obsidianmd recommended config's "error" severity applies. Scopes
import/no-nodejs-modules off for test setup, mocks, and Node-context
build scripts (e.g. jest.setup.js polyfills TextEncoder/TextDecoder
from node:util). import/no-extraneous-dependencies is now enabled
everywhere; declared js-yaml, dotenv, and @jest/globals as explicit
devDependencies so transitive uses pass.
Fixes the one real violation in src/utils.ts by migrating two moment
call sites (formatDateTime, stringToFormattedDateTime) to Luxon, which
is already in dependencies. Adds 10 unit tests covering UTC/local
formatting, zero-padding, round-tripping, and invalid-input fallback.
Replace ChatCohere / ChatMistralAI / CohereEmbeddings with ChatOpenAI /
OpenAIEmbeddings pointed at each provider's OpenAI-compatible endpoint:
- Cohere: https://api.cohere.ai/compatibility/v1
- Mistral: https://api.mistral.ai/v1 (already OpenAI-shaped)
Dropping @langchain/cohere, @langchain/mistralai, cohere-ai, and
@mistralai/mistralai also eliminates the @aws-sdk/* and @smithy/* chain
that cohere-ai dragged in for Bedrock auth (a path this plugin doesn't use).
Production bundle: 5.21MB -> 3.39MB.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
First of nine workspaces splitting #2397 into focused PRs (see
designdocs/todo/SCORECARD_WARNINGS.md, landing in a later workspace).
This PR is deps-only so the source-touching workspaces (types,
promise safety, popout compat, styles, trash semantics, brevilabs
rewrite, eslint rules) can each land independently.
Dependencies:
- @codemirror/state, @codemirror/view: peer deps for newer @lexical/* usage
- @langchain/classic: required by langchain ^1 split layout
- openai: direct consumer in upcoming provider work
- uuid + @types/uuid: replaces ad-hoc id generation
- zod: schema validation used in upcoming type tightening
DevDependencies:
- eslint-plugin-obsidianmd: installed now but not wired into .eslintrc
until the final workspace (W9) once all flagged sites are cleaned up
No source changes; no behavior changes. npm run lint / build / test
all pass on this branch.
The modal and its REMOVE_FILES_FROM_COPILOT_INDEX command were orphaned in
the vault search v3 migration (#1703): the addCommand registration was
removed but the modal file and its constant entries were left behind.
Nothing in the codebase has instantiated it since. Deleting it also clears
the risk-report finding about contentEl.createEl("style", ...), which
Obsidian disallows at runtime.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* release: v3.2.4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* release: add detailed changelog to v3.2.4 notes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Upgrade @langchain/google-genai from 1.0.2 to 2.1.23 to fix a crash
where `candidateContent.parts.reduce()` throws when `parts` is undefined
during Gemini streaming responses. The new version uses optional chaining
(`candidateContent.parts?.reduce()`).
Also upgrades @langchain/core (1.1.13 → 1.1.29) and langchain (1.2.8 →
1.2.28) to align peer dependencies.
Fixes#2248
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* 3.2.3
* fix: rename "Copy command" to "Duplicate command" with copy-plus icon (#2150)
Improves UX clarity by distinguishing the duplicate action from clipboard copy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: upgrade @langchain/google-genai to fix Gemini 3 preview image support (#2178)
The @langchain/google-genai v1.0.0 had a hardcoded _isMultimodalModel check
that only recognized gemini-1.5, gemini-2, and "vision" model names. Gemini 3
preview models (gemini-3-flash-preview, gemini-3-pro-preview) were rejected
with "This model does not support images" when sending image content.
Upgrade to v1.0.2 which adds gemini-3 to the multimodal model pattern.
Also bump @google/generative-ai from ^0.21.0 to ^0.24.0 (required dep).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update built-in Gemini Pro to 3.1 preview model IDs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add RELEASES.md with full release history and v3.2.0 draft
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* 3.2.0
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Update ollama chat model to fix thinking
* fix: consolidate tool call parsing and add actionable error logging
- Remove duplicate tool call parsing in AutonomousAgentChainRunner
by using shared buildToolCallsFromChunks() function
- Add sanitizeToolArgs() to remove empty objects from tool args
(fixes Ollama returning timeRange: {} which fails Zod validation)
- Add actionable logging for tool call failures with full args context
- Clean up unused parameters in AutonomousAgentChainRunner
- Simplify ToolManager.callTool to throw instead of swallowing errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: add repeatPenalty for Ollama to reduce repetitive outputs
Local models via Ollama are prone to repetitive hallucination loops.
Adding repeatPenalty=1.1 applies a slight penalty to repeated tokens,
which helps reduce this pattern.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: increase Ollama context window to 128k
Default numCtx of 8192 causes prompt truncation with larger contexts.
Setting numCtx=131072 (128k) to support models with large context.
Ollama automatically caps at model's actual maximum if exceeded.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: handle Ollama empty object tool args at schema level
- Make timeRange inner fields (startTime, endTime) optional in schema
- Update validateTimeRange to handle empty or partial objects from LLMs
- Apply validation in all search tools (local, lexical, semantic)
- Remove sanitizeToolArgs function that was too aggressive
- Fixes tool calls that legitimately accept {} as valid input
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: implement floating quick command with AI-powered inline editing
Add a new floating quick command feature that allows users to interact
with AI directly from the editor. Key features include:
- Quick Ask panel with draggable modal interface
- Multiple interaction modes: ask, edit, and edit-direct
- Selection highlight and replace guard for safe text operations
- Streaming chat session support for real-time AI responses
- Context menu integration for quick access
- Resizable and draggable UI components
# Conflicts:
# src/commands/CustomCommandChatModal.tsx
# src/styles/tailwind.css
* feat: add persistent selection highlight for Chat panel
* refactor: extract persistent highlight factory and fix state consistency
* fix: improve popup positioning with flip logic and line-start trap fix
- Use selection.head instead of selection.to for anchor point
- Fix line-start trap: only apply when head is selection end
- Add flip logic: show above when not enough space below
- Add horizontal visibility check
- Single-line selection: center popup horizontally
- Multi-line selection: follow head position
- Cursor mode: align near cursor instead of centering
* feat: Add comprehensive system prompt management system.
* fix：fixed some bugs
* fix: prevent false negatives in migration content verification
Strip leading newlines before comparing saved vs original content.
Obsidian may insert extra blank line after frontmatter block,
causing stripFrontmatter to leave a leading newline and fail verification.
* feat: Add Web Viewer bridge for referencing open web tabs in chat.
* refactor: Simplify markdown image extraction parser。
* fix: Prevent web selection from auto-reappearing after removal or new chat.
* fix: Suppress active web tab when web selection exists.
* feat: Add YouTube transcript extraction for Web Viewer tabs.
* feat: Built-in slash prompts for web clippers.
* feat: Improve web selection tracking and simplify context settings
- Merge context settings: combine note/web active content and selection settings
- Add auto-clear for web selection badge when deselected on page
- Make note and web selections mutually exclusive
- Show website favicon in web selection badge
- Add migration logic and tests for settings
* Replace FlexSearch with MiniSearch for improved search functionality and scoring
* Enhance QueryExpander and README for improved recall and ranking clarity
- Update QueryExpander to clearly separate salient and expanded terms for recall and ranking.
- Modify README to reflect changes in the search system's architecture and clarify the distinction between recall and ranking processes.
- Adjust FullTextEngine comments to emphasize BM25's native multi-term scoring capabilities.
* Implement weighted query expansion in FullTextEngine with 90/10 scoring for salient and expanded terms
* Building persistent memory for copilot (#1848)
* Implement chat input v3 (#1794)
* Fix chat crash React 409 (#1849)
* Fix mobile typeahead menu size (#1853)
* Enhance progress bar and bug fix (#1814)
* feat: Add edit context for progress card.
* fix: When adding a model, do not set the key in the single model setting.
* fix: fix the width of the popover on mobile.
* Refactor Brevilabs API integration to use models base URL (#1855)
- Updated constants to switch from BREVILABS_API_BASE_URL to BREVILABS_MODELS_BASE_URL for model-related API calls.
- Adjusted ChatModelManager and EmbeddingManager to utilize the new models base URL for configuration settings.
* feat: Add new chat history popover. (#1850)
* feat: Add new chat history popover.
* feat: Add open source file button to chat history popover
- Add ArrowUpRight button to each chat history item
- Implement openChatSourceFile method in main.ts
- Add onOpenSourceFile callback prop chain through components
- Open chat files in new Obsidian tabs when clicked
- Optimize error handling to prevent duplicate notices
* feat: Implement localSearch CiC prompting flow in CopilotPlusChainRunner (#1856)
* Properly extract response text in UserMemoryManager (#1857)
* feat: Temporarily disable autocomplete features (#1858)
* chore: Update version to 3.1.0-preview-250927 in manifest.json
* Support space in typeahead trigger and improve search (#1859)
* More chat input enhancement (#1864)
* Fix dropdown color
* Fix badge border in light theme
* Increase search result number
* support paste image
* fix folder context
* Enhance ChatPersistenceManager with filename sanitization tests (#1865)
- Added tests to ensure proper sanitization of wiki link brackets and illegal characters in filenames when saving chat messages.
- Updated filename generation logic to handle empty sanitized topics by defaulting to 'Untitled Chat'.
- Refactored imports in ChatPersistenceManager for better organization.
* Update ApplyView accept button styles for improved visibility and interaction (#1866)
- Adjusted the positioning of the action button to be further from the bottom of the viewport for better accessibility.
- Increased the z-index to ensure the button is always on top of other elements.
- Added a shadow effect to enhance the button's visibility against the background.
* Expose add selection to chat context to free users (#1867)
* New chat input improvements (#1868)
* Improve search logic in at mention search
* Subscribe to file changes
* Detect changes for folders and tags
* Make context badge tooltip always show
* Add full note path in preview
* Fix enter being blocked when no search result
* Rebuild "Active Note" and more chat input improvement (#1873)
* Performance improvement and extend "active note" to note typeahead menu (#1875)
* Add better agent prompt logging (#1882)
* Add read tool (#1883)
* Add readNote tool for reading notes in chunks
* Enhance readNote tool to support dynamic note path display and linked note extraction
* Refine readNote tool instructions for improved clarity and efficiency in note content retrieval
* Add support for readNote tool: integrate emoji, format results, and refine instructions
* Add inline citation reminder functionality to user questions
* Add an alert when user hits the maxToken limit (#1884)
* Refactor AutonomousAgentChainRunner (#1887)
* Refactor AutonomousAgentChainRunner to enhance agent workflow, streamline context preparation, and improve response handling
* Enhance type safety in addChatHistoryToMessages by specifying message structure
* Improve max iteration limit message for clarity and formatting
* Refactor tool display name handling for improved clarity and maintainability
* Refactor tool call ID generation and visibility handling for improved clarity and uniqueness
* Fix agent tool call ID (#1890)
* Enhance tool call ID generation and improve readNote handling in prompts
Update readNote description for clarity and improve prompt instructions
* Update instructions for registerFileTreeTool to clarify usage guidelines
* Clarify instructions for readNote tool to improve context inference and handling of partial note titles
* Add token counter (#1889)
* Fix index rebuild on semantic search toggle (#1891)
* Update token counter label (#1892)
* Implement tag search v3 (#1893)
* feat: Enhance TieredLexicalRetriever to support tag-based retrieval and improve search scoring
- Added support for returning all matching tags in TieredLexicalRetriever.
- Introduced new options for tag terms and returnAllTags in the retriever.
- Updated FullTextEngine to prioritize tag matches and improve scoring for documents with tags.
- Enhanced tokenization to handle hierarchical tags and prevent splitting hyphenated tags.
- Improved handling of frontmatter tags in documents.
- Updated search tools to accommodate new tag handling features.
- Added tests to verify the functionality of tag-based retrieval and scoring improvements.
* fix: Improve logging for query expansion in SearchCore
* feat: Add explanation for non-tag matches in FullTextEngine search results
* feat: Enhance QueryExpander to preserve standalone terms in tag handling and improve term validation
* feat: Enhance QueryExpander and TieredLexicalRetriever to improve tag handling and standalone term extraction
* Update version to 3.1.0-preview-251006 in manifest.json
* Normalize tag queries for case-insensitive matching and improve search functionality (#1894)
* Implement merge retriever (#1896)
* Fix note read (#1897)
* Enhance note resolution logic and add tests for wiki-linked notes and basename matching
* Implement note resolution outcome types and enhance readNoteTool tests for ambiguous matches
* Add deriveReadNoteDisplayName function and enhance readNoteTool tests for edge cases
* Fix note tool UI freeze (#1898)
* Update how tags work in context (#1895)
* Enhance file creation instructions in modelAdapter and update tool usage guidelines in builtinTools (#1901)
- Added instructions for confirming folder existence before creating new files in modelAdapter.
- Updated custom prompt instructions in builtinTools to clarify the use of getFileTree for folder lookups when creating new notes.
* Do not add url context for youtube url (#1899)
* Update YouTube Script command and modal title to indicate Plus feature (#1903)
* Enhance ChatPersistenceManager to handle file save conflicts (#1904)
* Enhance ChatPersistenceManager to handle file save conflicts and improve epoch handling
* Fix type checking for existing files in ChatPersistenceManager to prevent errors
* Fix verify add (#1905)
- Rename "Verify" button to "Test" in Add Model dialog
- Make verification not required for adding model in Set Keys
- Upgrade chatAnthropic client to fix the Top P -1 error for Claude Opus models
* Enhance XML parsing to handle tool calls missing closing tags
---------
Co-authored-by: Wenzheng Jiang <jwzh.hi@gmail.com>
Co-authored-by: Zero Liu <zero@lumos.com>
Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com>
* feat: add mobile-responsive components for settings.
* fix: Restrict HelpTooltip components to only affect mobile devices.
* fix: fix the args.diff case test.
* feat: enhance optional parameter controls with toggle functionality.
* fix: fix the issue of the ParameterControl Fails with Undefined and Zero.
* feat: Change Reasoning Effort/Verbosity parameters to optional parameters.
* Add LexicalEngine and related interfaces for enhanced search functionality
- Implement LexicalEngine class utilizing FlexSearch for efficient document indexing and searching.
- Create Hit, SearchResult, SearchOptions, and RetrieverEngine interfaces to standardize search operations.
- Update package.json and package-lock.json to include new dependencies: flexsearch and p-queue.
* Add QueryExpander class and tests for enhanced search query expansion
* Add README for Tiered Note-Level Lexical Retrieval with multilingual support and enhanced search pipeline
* Implement v3 tiered search with GrepScanner, FullTextEngine, and GraphExpander
- Add GrepScanner for fast substring search (L0)
- Implement FullTextEngine with ephemeral FlexSearch index (L1)
- Add GraphExpander for link-based candidate expansion
- Create MemoryManager with platform-aware limits
- Implement weighted RRF fusion for result combination
- Add SemanticReranker placeholder for future integration
- Include comprehensive tests for core components
- Simplify code based on review: use TextEncoder, extract methods, streamline RRF
- Add clear logging for debugging search pipeline
The architecture follows a tiered approach:
1. Grep scan for initial candidates
2. Graph expansion to increase recall
3. Full-text search on expanded set
4. Optional semantic reranking
5. RRF fusion to combine signals
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Enhance tiered retrieval recall with both rewritten queries and the expanded salient terms
- Added detailed example of end-to-end query processing in README.md
- Updated TieredRetriever to log salient terms during query expansion
- Modified FullTextEngine to index link basenames for improved searchability
- Adjusted NoteDoc interface to clarify link handling and indexing
* Refactor QueryExpander configuration and caching logic for improved clarity and performance; enhance GraphExpander documentation and testing; remove unused fields from NoteDoc interface.
* Refactor search retrieval system to wire in TieredLexicalRetriever
- Replaced HybridRetriever with TieredLexicalRetriever in ChainManager, VaultQAChainRunner, and main.ts for improved multi-stage retrieval.
- Updated README.md to reflect new integration tasks and performance benchmarks.
- Introduced TieredLexicalRetriever class to handle on-demand indexing and retrieval.
- Enhanced GrepScanner to support inclusion/exclusion patterns for file indexing.
- Modified TieredRetriever to combine expanded salient terms and improved logging for final results.
- Removed legacy index management in favor of ephemeral indexing with TieredLexicalRetriever.
- Updated interfaces and utility functions to support new retrieval logic.
* feat: Implement first working TieredLexicalRetriever and refactor search scoring
- Added comprehensive tests for TieredLexicalRetriever, covering folder boosting and result combination.
- Refactored TieredLexicalRetriever to utilize SearchCore instead of TieredRetriever.
- Enhanced FullTextEngine with improved scoring mechanisms, including field weighting and multi-field match bonuses.
- Introduced FuzzyMatcher utility for fuzzy matching capabilities, including Levenshtein distance and variant generation.
- Updated GrepScanner to prioritize path matching for faster search results.
- Implemented normalized scoring in RRF for better ranking consistency.
* feat: Update vault search result display to show snippet of content instead of full text
* feat: Enhance FullTextEngine to index frontmatter property values and improve search scoring
* feat: Improve scoring and logging
* feat: Refactor TieredLexicalRetriever to support time-based queries and improve document retrieval logic
* Remove TODO.md from tracking and add to .gitignore
- TODO.md is now a local-only development session tracker
- Prevents accidental commits of work-in-progress task lists
- Each developer can maintain their own TODO.md without conflicts
* feat: Update dependencies and enhance search functionality
- Upgraded axios to version 1.11.0 and electron to version 27.3.11 for improved performance and security.
- Refactored QueryExpander to limit queries to the original for strict fallback tests.
- Enhanced SearchCore to rank grep hits by evidence quality before fusion, improving retrieval accuracy.
- Implemented a new method in SearchCore to rank grep hits based on evidence strength.
- Updated TieredLexicalRetriever tests to ensure proper integration with mocked SearchCore.
- Improved FuzzyMatcher to include plural/singular normalization for better fuzzy matching.
* feat: Add tool call marker encoding/decoding
- Introduced new utility functions for encoding and decoding tool call markers to ensure safe embedding in HTML comments.
- Updated `updateChatMemory` to handle encoded tool call markers, preserving their integrity during memory storage.
- Enhanced logging to decode tool marker results for better readability while maintaining encoded formats for storage.
- Added comprehensive tests for tool call marker functionality to ensure correct encoding and decoding behavior.
- Refactored related components to integrate the new encoding/decoding logic seamlessly.
* feat: Enhance FullTextEngine search to support low-weight terms and improve scoring
- Updated FullTextEngine to accept low-weight terms in the search method, allowing for better handling of salient terms.
- Implemented downweighting for boolean and numeric tokens in the properties field to reduce noise in search results.
- Added a new test case to validate the downweighting behavior for boolean and numeric queries in FullTextEngine.
- Improved logging for full-text search results to provide clearer insights into the search process.
* feat: Update GraphExpander to enhance search recall with adaptive hop logic
- Implemented guardrails in GraphExpander to adjust hop depth based on the number of grep hits: allows +1 hop for small sets (<5) and limits to 1 hop for large sets (≥50).
- Updated README.md to reflect new guardrail logic and scoring normalization in search results.
- Enhanced test cases to validate the new behavior of skipping co-citations for large result sets and allowing additional hops for smaller sets.
* feat: Filter out background tools during streaming to enhance user experience
- Added logic to determine and filter out background tools, preventing their names from appearing in the tool call display during streaming.
- Implemented a mechanism to include partial tool names only if they meet a specified length threshold, improving clarity in tool call presentations.
* feat: Enhance search tools to support time-based queries and display modified time
- Updated localSearchTool to ensure a healthy cap on max source chunks for time-based queries, improving recall.
- Modified ToolResultFormatter to display actual modified time for time-filtered results, enhancing clarity in search results.
- Adjusted output formatting to differentiate between recency and relevance scores based on query type.
* feat: Introduce semantic search capabilities with Memory Index support
- Added `enableSemanticSearchV3` setting to control the new semantic search feature.
- Implemented `MemoryIndexManager` for managing in-memory vector indexing with JSONL persistence.
- Enhanced `SearchCore` to utilize semantic retrieval based on the new memory index.
- Introduced command to build the semantic memory index, improving search accuracy and retrieval efficiency.
- Updated relevant components to integrate semantic search functionality seamlessly.
* feat: Add unit tests for MemoryIndexManager to validate functionality
- Introduced comprehensive tests for the MemoryIndexManager, covering scenarios such as loading from a file, building a vector store, and indexing vault contents.
- Implemented a mock embeddings API to facilitate testing without external dependencies.
- Added a utility function to reset the MemoryIndexManager state between tests, ensuring isolation and reliability of test outcomes.
* feat: Enhance MemoryIndexManager and related components for improved semantic indexing
- Introduced incremental indexing capabilities in MemoryIndexManager to update the JSONL index with new or modified files, enhancing efficiency.
- Updated commands to utilize the new incremental indexing method, providing users with real-time updates to the semantic memory index.
- Refactored existing indexing logic to support partitioned writes, improving performance and manageability of large datasets.
- Enhanced user notifications during indexing processes to provide better feedback on progress and status.
- Added a setting to enable semantic search, allowing users to blend semantic similarity into search results seamlessly.
* feat: Refine scoring display and enhance search result handling
- Updated SourcesModal to display relevance scores with four decimal places for consistency with SearchCore logs.
- Enhanced TieredLexicalRetriever to include a new `rerank_score` field for better score management and consistency across search results.
- Modified the combineResults method to ensure that title matches retain their original score while incorporating a fused score as `rerank_score`.
- Adjusted formatting in SearchTools to ensure both `score` and `rerank_score` reflect the same final score when present.
- Updated ToolResultFormatter to display scores with four decimal places, improving clarity in search result presentations.
* refactor: Simplify relevant notes retrieval by removing VectorStoreManager dependency
- Removed the use of VectorStoreManager in the relevant notes fetching logic, streamlining the process.
- Integrated MemoryIndexManager for improved handling of in-memory indexing and note retrieval.
- Enhanced error handling to ensure relevant notes are only fetched when the embedding index is available.
- Updated related functions to utilize the new memory index approach, improving performance and reliability.
* refactor: Remove VectorStoreManager dependency and streamline indexing logic
- Eliminated the VectorStoreManager from various components, transitioning to MemoryIndexManager for indexing and retrieval.
- Updated project and chain managers to remove unnecessary dependencies, enhancing code clarity and maintainability.
- Improved error handling and indexing conditions, particularly for mobile users, ensuring a more robust initialization process.
- Refactored settings components to utilize the new memory indexing approach, simplifying the user experience and reducing legacy code.
* chore: Mark legacy components as deprecated in preparation for v3 transition
- Annotated various files including DebugSearchModal, OramaSearchModal, chunkedStorage, dbOperations, hybridRetriever, and vectorStoreManager as deprecated, indicating they are obsolete in v3.
- Updated README.md to reflect the current implementation status and migration notes, emphasizing the removal of Orama-based modules and the transition to MemoryIndexManager for indexing and retrieval.
* feat: Implement file tracking and reindexing for modified files
- Introduced a new FileTrackingState interface to manage the last active file and its modification time.
- Enhanced the CopilotPlugin to opportunistically reindex the previous active file if it was modified while active, contingent on semantic search settings.
- Updated MemoryIndexManager to support reindexing of single modified files, improving efficiency in handling changes.
- Added unit tests for reindexSingleFileIfModified to ensure correct functionality and performance.
* feat: Add graph hops setting for enhanced search result expansion
- Introduced a new `graphHops` setting in `CopilotSettings` to control the number of hops for graph expansion during search, with a default value of 1 and a range of 1-3.
- Updated the `sanitizeSettings` function to validate the `graphHops` value.
- Enhanced the `SearchCore` and `TieredLexicalRetriever` classes to utilize the `graphHops` setting for improved search result relevance.
- Updated documentation in `README.md` to reflect the new feature and its security optimizations.
* refactor: Improve error handling and indexing logic in MemoryIndexManager and related components
- Replaced console error logging with structured logging using logError and logWarn for better error tracking.
- Enhanced the refresh and reindexing functions to ensure proper user notifications and error handling.
- Updated the logic for managing indexed files, including handling exclusions and ensuring accurate reporting of indexed and unindexed files.
- Implemented rate limiting in the MemoryIndexManager to optimize embedding requests and prevent overloading the service.
- Refactored chunk processing to improve efficiency and clarity in the indexing workflow.
* chore: Update README.md to reflect final session completion and key fixes
- Expanded the final session summary with a date and detailed list of completed features and fixes, including UI enhancements and improved logging practices.
- Clarified the status of deferred features and provided migration notes for better user guidance.
- Ensured documentation aligns with the latest implementation changes and optimizations.
* chore: Update memory limits in README.md and MemoryManager.ts for improved performance
- Adjusted memory limits for mobile and desktop platforms in both README.md and MemoryManager.ts to reflect increased capacity (20MB mobile, 100MB desktop).
- Ensured documentation aligns with the latest implementation changes for better clarity on resource management.
* feat: Integrate HyDE document generation into SearchCore for enhanced semantic search
- Implemented a new method to generate hypothetical documents (HyDE) to improve semantic search capabilities.
- Added timeout handling for HyDE generation to ensure graceful error management.
- Updated SearchCore to utilize HyDE documents in search queries, enhancing the relevance of results when semantic search is enabled.
- Introduced unit tests to validate the integration and functionality of HyDE generation within the search process.
* refactor: Enhance MemoryIndexManager with public methods for better indexing management
- Introduced public methods in MemoryIndexManager to streamline access to indexed file paths, check if a file is indexed, retrieve embeddings, and clear the index.
- Updated related components to utilize these new methods, improving code clarity and reducing direct access to internal properties.
- Enhanced error handling and user notifications during memory index operations.
* feat: Implement indexing notification and progress management
- Introduced the IndexingNotificationManager to handle UI notifications during indexing operations, allowing users to pause or stop the process.
- Added the IndexingProgressTracker to track progress across multiple files, enhancing user feedback during indexing.
- Developed the IndexingPipeline to manage the chunking and processing of files into embeddings, improving the overall indexing workflow.
- Created the IndexPersistenceManager for managing the persistence of indexed data, ensuring efficient storage and retrieval of index records.
- Refactored MemoryIndexManager to utilize the new components, streamlining the indexing process and improving code organization.
---------
Co-authored-by: Claude <noreply@anthropic.com>
