mirror of
https://github.com/logancyang/obsidian-copilot.git
synced 2026-07-22 07:50:24 +00:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Logan Yang
|
c4027fa7a4
|
Update Corpus-in-Context and web search tool guide (#1988)
* Make inline citation permanent and add image guide for local search - Removed the `enableInlineCitations` setting from the configuration and related components to simplify citation management. - Updated citation functions to operate without the setting, ensuring consistent behavior across the codebase. - Adjusted tests and documentation to reflect the removal of inline citation handling logic. - Enhanced the `renderCiCMessage` and `processInlineCitations` functions for improved clarity and functionality. * fix: Update web search tool description for clarity - Revised the description of the `webSearch` tool to specify that it searches the internet for information only when explicitly requested by the user. - Enhanced the instructions for using the `webSearch` tool to clarify when it should be utilized, ensuring users understand the intent required for web searches. - Reintroduced the `createGetTagListTool` import to maintain functionality in the `builtinTools` module. * fix: Enhance response handling in AgentPrompt integration tests - Updated the response processing logic in the AgentPrompt integration tests to ensure that `fullResponse` is always a string, accommodating multimodal and array content. - Implemented a function to extract text parts from arrays and handle various content types, improving the robustness of the tests. * Remove websearch related prompt from modelAdapter * fix: Update time query instructions in builtinTools - Added detailed instructions for handling time queries, including the conversion of city or timezone names to UTC offsets. - Clarified when to use the timezoneOffset parameter and emphasized the need for user clarification if the offset cannot be determined. |
||
|
Logan Yang
|
ed884298ba
|
Integrate ProjectChainRunner and ChatManager with new layered context (#1973)
* Integrate ProjectChainRunner and ChatManager with new layered context - Introduced `ProjectChainRunner` to automatically include project context in L1 via `ChatManager.getSystemPromptForMessage()`, simplifying the class structure by inheriting behavior from `CopilotPlusChainRunner`. - Updated `ChatManager` to build system prompts that append project context when in project mode, ensuring seamless integration without special-case logic. - Implemented null guards for project context to prevent erroneous output when context is unavailable. - Refactored documentation to reflect changes in context handling and the new project chain runner functionality. * Enhance ChatManager tests with project context mocks - Added mocks for project context retrieval and system prompt settings in `ChatManager.test.ts` to facilitate testing without relying on actual project data. - Improved test coverage for context-related functionalities by simulating project context and system prompt retrieval, ensuring robust testing of the ChatManager's behavior in various scenarios. |
||
|
Logan Yang
|
05beb941eb
|
Context revamp (#1971)
* Implement context engine for a layered context * Wire in new context library to LLM chain - Updated the context engine to maintain a cumulative library of all context items (L2) without deduplication, enhancing cache stability. - Introduced smart referencing in the user message (L3) to reference existing items in L2 by ID and include full content for new items. - Improved the LayerToMessagesConverter to format and handle layered context envelopes for better debugging and clarity. - Fixed critical bugs related to context deduplication that previously broke cache stability. - Enhanced tests to validate the new context handling and ensure proper functionality across various scenarios. * Enhance VaultQAChainRunner with tag extraction and inline citation detection - Integrated tag extraction from user queries to improve context-aware retrieval. - Implemented `hasInlineCitations` function to detect inline citation markers in responses. - Updated message construction logic to utilize envelope-based context, enhancing the quality of AI interactions. - Added comprehensive tests for the new inline citation detection functionality, ensuring robustness and accuracy. - Improved logging for better debugging and traceability of the message processing flow. Refactor VaultQAChainRunner to enhance context handling and citation instructions - Integrated LayerToMessagesConverter for improved message construction, ensuring smart referencing and context preservation. - Updated logging to reflect the use of the new converter in envelope-based context construction. - Revised comments for clarity on context preparation and citation instructions. - Minor adjustment in citation utility to improve regex handling for citation updates. * Complete migration to envelope-based context in CopilotPlus and LLM chain runners - Removed all legacy fallback paths, ensuring that context envelopes are now mandatory for operations in CopilotPlusChainRunner and LLMChainRunner. - Enhanced image extraction to only pull from the active note, preventing unintended leaks from attached context notes. - Fixed critical issues with context-in-context (CiC) formatting and added early envelope guards in VaultQAChainRunner to prevent silent failures. - Updated message construction logic to utilize LayerToMessagesConverter for improved smart referencing and context preservation. - Comprehensive tests added to validate the new envelope-based context handling and ensure robustness across all chain runners. * Integrate Plus chain with context envelope * Enhance AutonomousAgentChainRunner with envelope-based context integration - Introduced `LayerToMessagesConverter` for improved message construction, ensuring system and user messages are derived from the context envelope. - Implemented envelope validation to ensure context is available before processing. - Updated message preparation logic to maintain consistency with the envelope-first architecture, preserving tool execution and multimodal support. - Added comprehensive logging for envelope-based context construction to aid debugging. - Documented changes in the context engineering documentation to reflect the new integration and design points. * Support loading saved chat context to L2 context library * Fix ChatPersistenceManager tests for context handling |
||
|
Logan Yang
|
cea72d61b1
|
Support dataview result in active note (#1918) | ||
|
Logan Yang
|
5568b70e0f
|
Merge 3.1.0 preview (#1906)
* Building persistent memory for copilot (#1848) * Implement chat input v3 (#1794) * Fix chat crash React 409 (#1849) * Fix mobile typeahead menu size (#1853) * Enhance progress bar and bug fix (#1814) * feat: Add edit context for progress card. * fix: When adding a model, do not set the key in the single model setting. * fix: fix the width of the popover on mobile. * Refactor Brevilabs API integration to use models base URL (#1855) - Updated constants to switch from BREVILABS_API_BASE_URL to BREVILABS_MODELS_BASE_URL for model-related API calls. - Adjusted ChatModelManager and EmbeddingManager to utilize the new models base URL for configuration settings. * feat: Add new chat history popover. (#1850) * feat: Add new chat history popover. * feat: Add open source file button to chat history popover - Add ArrowUpRight button to each chat history item - Implement openChatSourceFile method in main.ts - Add onOpenSourceFile callback prop chain through components - Open chat files in new Obsidian tabs when clicked - Optimize error handling to prevent duplicate notices * feat: Implement localSearch CiC prompting flow in CopilotPlusChainRunner (#1856) * Properly extract response text in UserMemoryManager (#1857) * feat: Temporarily disable autocomplete features (#1858) * chore: Update version to 3.1.0-preview-250927 in manifest.json * Support space in typeahead trigger and improve search (#1859) * More chat input enhancement (#1864) * Fix dropdown color * Fix badge border in light theme * Increase search result number * support paste image * fix folder context * Enhance ChatPersistenceManager with filename sanitization tests (#1865) - Added tests to ensure proper sanitization of wiki link brackets and illegal characters in filenames when saving chat messages. - Updated filename generation logic to handle empty sanitized topics by defaulting to 'Untitled Chat'. - Refactored imports in ChatPersistenceManager for better organization. * Update ApplyView accept button styles for improved visibility and interaction (#1866) - Adjusted the positioning of the action button to be further from the bottom of the viewport for better accessibility. - Increased the z-index to ensure the button is always on top of other elements. - Added a shadow effect to enhance the button's visibility against the background. * Expose add selection to chat context to free users (#1867) * New chat input improvements (#1868) * Improve search logic in at mention search * Subscribe to file changes * Detect changes for folders and tags * Make context badge tooltip always show * Add full note path in preview * Fix enter being blocked when no search result * Rebuild "Active Note" and more chat input improvement (#1873) * Performance improvement and extend "active note" to note typeahead menu (#1875) * Add better agent prompt logging (#1882) * Add read tool (#1883) * Add readNote tool for reading notes in chunks * Enhance readNote tool to support dynamic note path display and linked note extraction * Refine readNote tool instructions for improved clarity and efficiency in note content retrieval * Add support for readNote tool: integrate emoji, format results, and refine instructions * Add inline citation reminder functionality to user questions * Add an alert when user hits the maxToken limit (#1884) * Refactor AutonomousAgentChainRunner (#1887) * Refactor AutonomousAgentChainRunner to enhance agent workflow, streamline context preparation, and improve response handling * Enhance type safety in addChatHistoryToMessages by specifying message structure * Improve max iteration limit message for clarity and formatting * Refactor tool display name handling for improved clarity and maintainability * Refactor tool call ID generation and visibility handling for improved clarity and uniqueness * Fix agent tool call ID (#1890) * Enhance tool call ID generation and improve readNote handling in prompts Update readNote description for clarity and improve prompt instructions * Update instructions for registerFileTreeTool to clarify usage guidelines * Clarify instructions for readNote tool to improve context inference and handling of partial note titles * Add token counter (#1889) * Fix index rebuild on semantic search toggle (#1891) * Update token counter label (#1892) * Implement tag search v3 (#1893) * feat: Enhance TieredLexicalRetriever to support tag-based retrieval and improve search scoring - Added support for returning all matching tags in TieredLexicalRetriever. - Introduced new options for tag terms and returnAllTags in the retriever. - Updated FullTextEngine to prioritize tag matches and improve scoring for documents with tags. - Enhanced tokenization to handle hierarchical tags and prevent splitting hyphenated tags. - Improved handling of frontmatter tags in documents. - Updated search tools to accommodate new tag handling features. - Added tests to verify the functionality of tag-based retrieval and scoring improvements. * fix: Improve logging for query expansion in SearchCore * feat: Add explanation for non-tag matches in FullTextEngine search results * feat: Enhance QueryExpander to preserve standalone terms in tag handling and improve term validation * feat: Enhance QueryExpander and TieredLexicalRetriever to improve tag handling and standalone term extraction * Update version to 3.1.0-preview-251006 in manifest.json * Normalize tag queries for case-insensitive matching and improve search functionality (#1894) * Implement merge retriever (#1896) * Fix note read (#1897) * Enhance note resolution logic and add tests for wiki-linked notes and basename matching * Implement note resolution outcome types and enhance readNoteTool tests for ambiguous matches * Add deriveReadNoteDisplayName function and enhance readNoteTool tests for edge cases * Fix note tool UI freeze (#1898) * Update how tags work in context (#1895) * Enhance file creation instructions in modelAdapter and update tool usage guidelines in builtinTools (#1901) - Added instructions for confirming folder existence before creating new files in modelAdapter. - Updated custom prompt instructions in builtinTools to clarify the use of getFileTree for folder lookups when creating new notes. * Do not add url context for youtube url (#1899) * Update YouTube Script command and modal title to indicate Plus feature (#1903) * Enhance ChatPersistenceManager to handle file save conflicts (#1904) * Enhance ChatPersistenceManager to handle file save conflicts and improve epoch handling * Fix type checking for existing files in ChatPersistenceManager to prevent errors * Fix verify add (#1905) - Rename "Verify" button to "Test" in Add Model dialog - Make verification not required for adding model in Set Keys - Upgrade chatAnthropic client to fix the Top P -1 error for Claude Opus models * Enhance XML parsing to handle tool calls missing closing tags --------- Co-authored-by: Wenzheng Jiang <jwzh.hi@gmail.com> Co-authored-by: Zero Liu <zero@lumos.com> Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com> |
||
|
Logan Yang
|
92d6f98af2
|
Implement inline citation (#1821)
* Implement inline citation for vault search for both vault QA and Plus chains refactor: enhance citation utilities and improve source handling in chain runners - Introduced new utility functions for citation management, including sanitization and formatting of source catalogs. - Updated CopilotPlusChainRunner and VaultQAChainRunner to utilize the new citation utilities for improved citation handling. - Enhanced ChatSingleMessage component to consolidate duplicate sources and ensure accurate citation numbering. - Added comprehensive tests for citation utilities to ensure functionality and edge case handling. refactor: dedupe chunks for unique titles in search tool sources refactor: enhance citation handling in CopilotPlus and VaultQA chain runners - Introduced a sanitization function to remove pre-existing citation markers from document content, preventing number leakage. - Updated the formatting of retrieved documents in VaultQA to utilize the new sanitization function for cleaner output. - Improved footnote renumbering logic in ChatSingleMessage to prioritize first-mention order, ensuring accurate citation mapping. refactor: streamline source citation handling in ChatSingleMessage component and enhance web search tool instructions - Simplified the processing of source links in ChatSingleMessage to use a more efficient mapping approach. - Updated the web search tool's instruction to clarify the use of footnote-style citations, ensuring compatibility with Markdown formatting. * feat: add inline citation setting toggle and refactor - Introduced a new setting to enable inline citations in AI-generated responses, allowing for footnote-style references within the text. - Updated citation utilities to manage fallback sources and improve citation instructions based on user settings. - Enhanced CopilotPlusChainRunner and VaultQAChainRunner to utilize the new inline citation feature, ensuring accurate source representation. - Modified ChatSingleMessage component to process inline citations based on the new setting, improving the display of sources in chat messages. - Added tests for new citation functionalities to ensure reliability and correctness. * feat: enhance citation utilities and add tests for normalization - Introduced the `normalizeCitations` function to improve citation formatting by removing periods after citations and handling footnote references. - Updated `processInlineCitations` to ensure correct mapping of non-sequential citations and consolidate duplicate sources. - Added comprehensive tests for the new citation functionalities, ensuring accurate citation processing and representation in AI-generated responses. - Modified QASettings description for clarity regarding the inline citations feature. * fix: address critical bug in citation processing for consecutive citations - Enhanced the `normalizeCitations` function to correctly handle consecutive citations without spaces, ensuring accurate mapping and representation in output. - Added a test case to replicate and validate the fix for the identified bug, confirming that both citations are processed and displayed correctly. - Updated existing citation handling logic to improve robustness and prevent unconverted citations in the output. * refactor: improve citation handling and enhance citation utilities - Refactored `CopilotPlusChainRunner` to streamline citation source management, replacing `lastCitationLines` with `lastCitationSources` for better clarity and structure. - Updated the `addFallbackSources` function to handle citation formatting more effectively, ensuring proper display of titles and paths. - Enhanced the `normalizeCitations` function to support multiple citations in brackets, ensuring correct sorting and mapping of citation numbers. - Added new test cases to validate the handling of alternate headings, HTML summaries, and multiple citations, improving overall robustness of citation processing. - Introduced `getWebSearchCitationInstructions` for web-specific citation guidance, enhancing user experience with clear instructions for inline citations. * fix: move documentations * refactor: normalize footnote rendering in ChatSingleMessage component - Introduced `normalizeFootnoteRendering` function to streamline footnote display by removing separators and backreferences, and fixing numbering artifacts. - Updated `ChatSingleMessage` to utilize the new normalization function, enhancing the clarity and presentation of footnotes in chat messages. - Added unit tests for `normalizeFootnoteRendering` to ensure proper functionality and handling of various footnote scenarios. * test: enhance ChatSingleMessage tests and add deduplication tests for search tools - Updated `ChatSingleMessage.test.tsx` to include `TooltipProvider` for improved rendering context. - Refactored mock implementation for `obsidian` to streamline testing setup. - Introduced new test file `SearchTools.dedupe.test.ts` to validate the deduplication logic in search tool sources, ensuring correct handling of duplicate entries while preserving the highest score. - Added tests to verify deduplication behavior for various scenarios, enhancing overall test coverage and reliability. |