mirror of
https://github.com/logancyang/obsidian-copilot.git
synced 2026-07-22 07:50:24 +00:00
* feat: implement chunk-based search architecture and enhance retrieval consistency - Introduced a new `ChunkManager` for managing note chunking, enabling efficient handling of large documents. - Added comprehensive tests for `ChunkManager`, `SearchCore`, and `TieredLexicalRetriever` to ensure consistent chunk retrieval and scoring. - Updated `SearchCore` to utilize chunk-based retrieval, improving performance and relevance in search results. - Enhanced `FullTextEngine` to support chunk indexing, optimizing memory usage and search accuracy. - Refactored scoring mechanisms to maintain chunk order and consistency across search results. - Updated documentation to reflect changes in search architecture and chunking strategies. feat: integrate ChunkManager into SearchCore and related components - Added ChunkManager to SearchCore for improved chunk-based retrieval. - Updated FullTextEngine to accept ChunkManager as an optional parameter, enhancing memory management. - Refactored TieredLexicalRetriever to utilize the shared ChunkManager instance from SearchCore, ensuring consistent chunk handling. - Enhanced tests for TieredLexicalRetriever to mock ChunkManager functionality, validating chunk retrieval behavior. * feat: add enableLexicalBoosts flag to enhance search capabilities. Fix expanded term in ranking bug. - Introduced `enableLexicalBoosts` setting in `CopilotSettings` to control the application of lexical boosts during search operations. - Updated `SearchCore` to utilize the new setting, allowing for conditional application of folder and graph-based relevance boosts. - Enhanced `QueryExpander` and related interfaces to include `expandedTerms`, capturing LLM-generated terms for improved recall. - Added tests to validate the behavior of lexical boosts based on the new setting, ensuring expected functionality in various scenarios. - Updated documentation to reflect the new feature and its implications for search performance and relevance. * feat: enhance SearchCore to conditionally skip lexical and semantic searches based on semantic weight - Updated SearchCore to skip lexical search when semantic weight is 100% and skip semantic search when weight is 0%, improving performance. - Added logging to indicate when searches are skipped, enhancing traceability. - Introduced new tests to validate the behavior of search execution under different semantic weight scenarios, ensuring expected functionality. * feat: enhance ChunkManager and SearchCore with input validation and error handling - Improved input validation in `ChunkManager` to ensure valid note paths and handle edge cases, including empty and excessively long inputs. - Enhanced error handling in `SearchCore` to gracefully manage failures during chunk retrieval and fallback to simple grep results when necessary. - Updated `QueryExpander` to utilize a race condition approach for timeout handling, improving reliability during query expansion. - Removed deprecated tests and added new tests to validate the behavior of input validation and error handling, ensuring robustness in various scenarios. - Updated documentation to reflect changes in input handling and error management strategies. * fix: update semanticWeight assignment to use nullish coalescing operator - Changed the assignment of `semanticWeight` in `TieredLexicalRetriever` to utilize the nullish coalescing operator (`??`) for better handling of undefined values, ensuring a default value of 0.6 is applied when no weight is provided. * feat: enhance CopilotPlusChainRunner and MemoryIndexManager for improved source handling - Added deduplication of sources in CopilotPlusChainRunner to ensure unique results. - Refactored MemoryIndexManager to aggregate scores by chunk ID instead of note path, improving consistency in scoring. - Introduced a new method for aggregating scores per chunk, enhancing the retrieval process. - Updated tests to validate the new behavior in source handling and scoring aggregation. * feat: standardize chunk ID formatting and enhance ChunkManager functionality - Updated chunk ID format to consistently use zero-padding for indices across all components, ensuring uniformity in chunk identification. - Refactored ChunkManager to include automatic cache validation and regeneration for chunk retrieval, improving reliability and performance. - Introduced content hash for chunks to validate content integrity and prevent stale data usage. - Enhanced tests to validate new chunk ID formatting and ensure consistent behavior across various scenarios. - Updated documentation to reflect changes in chunk ID handling and retrieval processes. * feat: streamline index clearing process in registerCommands - Removed redundant file removal logic for clearing local Copilot semantic index. - Introduced IndexPersistenceManager to handle index file clearing, improving code clarity and maintainability. - Updated user notification to reflect the new index clearing process. * feat: add comprehensive tests for FullTextEngine's clear method - Introduced multiple test cases for the clear method in FullTextEngine to ensure safe operation under various conditions, including uninitialized index, multiple clear calls, and handling of index cleanup methods. - Enhanced the clear method to handle scenarios where the index may throw errors during cleanup, ensuring robust state management. - Verified memory management reset during the clear operation, improving overall reliability of the FullTextEngine. * feat: implement TimeoutError and withTimeout utility for enhanced error handling - Introduced a new TimeoutError class for consistent timeout error management across operations. - Added withTimeout utility function to execute operations with a specified timeout, utilizing AbortController for proper cancellation. - Updated QueryExpander and SearchCore to leverage withTimeout for handling LLM requests, improving reliability during long-running operations. - Enhanced tests for withTimeout and TimeoutError to ensure correct behavior under various scenarios, including timeout handling and error propagation. - Updated documentation to reflect the new timeout handling features and their usage. * fix: OOM error for single file index update - Introduced memory safety checks in IndexPersistenceManager to prevent out-of-memory (OOM) errors during record processing. - Implemented batch processing for writing records and updating file records, improving memory efficiency. - Added methods for estimating memory usage and checking memory safety thresholds. - Enhanced MemoryIndexManager to utilize incremental updates for modified files, reducing memory load during reindexing. - Updated tests to validate the new memory-safe operations and ensure correct behavior under high memory usage scenarios. * fix: have semantic search completely independent of grep and lexical search * refactor: update chunk ID formatting to non-padded indices - Changed chunk ID format in ChunkManager and related components to use non-padded indices, improving consistency and simplifying ID generation. - Updated tests to reflect the new ID format and ensure correct behavior across various scenarios. - Removed legacy checks for old padded formats in tests, streamlining the codebase. * feat: add SemanticSearchToggleModal for enabling/disabling semantic search - Introduced SemanticSearchToggleModal to confirm user actions for enabling or disabling semantic search. - Updated QASettings to utilize the new modal for handling semantic search state changes, providing user feedback and guidance on index management. * refactor: streamline embedding model handling in BasicSettings and QASettings - Removed the embedding model selection from BasicSettings, consolidating its functionality within QASettings. - Introduced a new handler for setting the default embedding model in QASettings, ensuring user confirmation via RebuildIndexConfirmModal. - Updated the UI components in QASettings to provide detailed descriptions and tooltips for embedding model selection, enhancing user experience and clarity. * refactor: remove unused components and streamline memory management - Deleted the SemanticReranker and VaultPathValidator classes to simplify the codebase and remove unused functionality. - Removed the index.ts file for v3 search, consolidating exports and reducing clutter. - Updated MemoryIndexManager to utilize a simplified in-memory approach for reindexing, enhancing performance and memory efficiency. - Adjusted tests to reflect changes in the MemoryIndexManager's reindexing strategy, ensuring consistent behavior and improved reliability.
22 lines
672 B
TypeScript
22 lines
672 B
TypeScript
export class CustomError extends Error {
|
|
public code?: string;
|
|
|
|
constructor(message: string, code?: string) {
|
|
super(message);
|
|
this.code = code;
|
|
// This is needed in TypeScript when extending built-in classes
|
|
Object.setPrototypeOf(this, CustomError.prototype);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* TimeoutError class for consistent timeout error handling
|
|
*/
|
|
export class TimeoutError extends Error {
|
|
constructor(operation: string, timeoutMs: number) {
|
|
super(`${operation} timed out after ${timeoutMs}ms`);
|
|
this.name = "TimeoutError";
|
|
// This is needed in TypeScript when extending built-in classes
|
|
Object.setPrototypeOf(this, TimeoutError.prototype);
|
|
}
|
|
}
|