Implement autonomous agent (#1689)

* Implement autonomous agent mode and new message architecture

- Implement autonomous agent mode with sequential thinking capabilities
- Migrate from sharedState to new clean message management architecture
- Add ChatManager, MessageRepository, and ContextManager for better separation of concerns
- Implement project-based chat isolation with separate message repositories per project
- Add ChatPersistenceManager for saving/loading chat history
- Refactor chain runner architecture with specialized runners (Autonomous, Project, VaultQA)
- Add XML tool call handling for better compatibility across LLM providers
- Introduce vault and web search toggles in settings
- Enhance model adapters with explicit tool usage instructions
- Add comprehensive test coverage for new architecture components
- Update UI components to use new ChatUIState instead of SharedState

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Enhance context processing and documentation

- Added detailed XML structure for context notes, including metadata (creation and modification times) for notes and selected text.
- Introduced new tests for message lifecycle and XML tag formatting to ensure proper context handling.
- Created TODO.md to track technical debt and future improvements related to context processing and file handling.
- Updated CLAUDE.md to reference the new TODO.md for technical debt awareness.

This commit improves the clarity and functionality of context processing in messages, ensuring rich context is maintained throughout the message lifecycle.

* Fix composer instruction in new message implementation

* Add writeToFile tool to agent (#1632)

* Add writeToFile tool call to agent

* Update src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/tools/ComposerTools.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove unuseful comment

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix tests

* Enhance context processing with new XML tags for notes and selected text (#1633)

* Enhance context processing with new XML tags for notes and selected text

- Introduced new constants for XML tags: SELECTED_TEXT_TAG, VARIABLE_TAG, VARIABLE_NOTE_TAG, EMBEDDED_PDF_TAG, VAULT_NOTE_TAG, and RETRIEVED_DOCUMENT_TAG.
- Updated context processing to format embedded PDFs and notes with XML structure.
- Modified prompt processing to include XML tags for selected text and variables, improving clarity and structure in generated outputs.
- Enhanced tests to validate the new XML formatting in processed prompts.

* Implement XML escaping utility functions and integrate into context processing

- Added `escapeXml` and `escapeXmlAttribute` functions to handle XML special character escaping, enhancing security against XML injection.
- Updated context processing to utilize these functions for escaping note titles, paths, and content, ensuring proper XML formatting.
- Enhanced tests to validate XML escaping functionality across various scenarios, including selected text and variable names.
- Introduced new test files for XML utility functions to ensure comprehensive coverage and reliability.

* Refactor XML utility functions and update imports

- Moved `escapeXml` and `escapeXmlAttribute` functions from `utils/xmlUtils` to `LLMProviders/chainRunner/utils/xmlParsing`, enhancing modularity.
- Updated all relevant imports across the codebase to reflect the new location of XML utility functions.
- Removed obsolete `xmlUtils` file and its associated tests, consolidating XML handling in a single module for better maintainability.

* Add message creation callback in ChatManager for immediate UI updates (#1634)

- Introduced a new method `setOnMessageCreatedCallback` in ChatManager to allow setting a callback that triggers when a message is created.
- Updated ChatUIState to utilize this callback for notifying listeners, ensuring the UI updates immediately upon message creation.
- This enhancement improves the responsiveness of the chat interface by synchronizing message creation events with UI state updates.

* Enhance TimeTools tests for invalid expressions

- Added setup to mock console.warn in tests for invalid time expressions.
- Verified that the appropriate warning message is logged when parsing fails.
- Improved test coverage for handling various invalid time inputs.

* Refactor local search handling and restore sources display

- Unified tool handling in CopilotPlusChainRunner to fix websearch results being ignored when localSearch has results
- All tool outputs now go through the same prepareEnhancedUserMessage method
- Preserved QA format and instruction when only localSearch is used
- Added proper validation for localSearch results (non-empty documents array)
- Maintained time expression support for temporal queries
- Fixed source tracking and display in ChatUIState
- Enhanced error handling for JSON parsing of search results

* Enhance AutonomousAgentChainRunner to track tool calls and results (#1639)

- Introduced a new property to store LLM-formatted messages for memory management.
- Reset LLM messages at the start of each run to ensure fresh context.
- Updated response handling to include LLM-formatted outputs, enhancing memory context for future iterations.
- Modified the base class to accept an optional LLM-formatted output parameter for improved context saving.
- Enhanced the writeToFile tool's response message for clarity on user actions.

* Quick Command (cmd+k) (#1640)

* Quick Command (cmd+k)

* Update custom command modal style

* Improve prompt

* Use submenu for commands

* Fix cursor comment

* Align selected_text variable name

* Fix embedded image wikilinks (#1641)

- Updated the `extractEmbeddedImages` method to accept an optional `sourcePath` parameter for resolving wikilinks.
- Implemented logic to determine the source path from context notes or fallback to the active file.
- Enhanced logging for unresolved images to improve debugging and user feedback.
- Ensured that embedded images are processed correctly based on the new source path handling.

* Fix salient term language issue

* Dedupe source notes (#1642)

* Fix salient term language issue

* Refactor source handling to include path in addition to title and score

- Updated source structures across multiple classes to include a `path` property alongside `title` and `score`.
- Modified methods in `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to accommodate the new source format.
- Enhanced the `deduplicateSources` function to use `path` as the unique key, falling back to `title` if necessary.
- Adjusted the `SourcesModal` to display paths correctly and ensure links open using the appropriate source path.
- Updated type definitions in `message.ts` to reflect the new source structure.

* Update path handling in source mapping for AutonomousAgentChainRunner

- Modified the source mapping logic to ensure the `path` property defaults to an empty string if not provided, enhancing robustness in source data handling.
- This change improves the consistency of source objects by ensuring that all properties are defined, which aids in downstream processing.

* Bump manifest version to v3

* Fix image (#1643)

* Fix salient term language issue

* Bump manifest version to v3

* Fix image passing

- Updated Chat, ChatManager, MessageRepository, and ChatUIState to include an optional `content` parameter in message handling.
- Adjusted method signatures and message creation logic to accommodate the new parameter, improving flexibility in message processing.

* Fix regenerate (#1644)

* Fix image in agent chain (#1646)

* Add YouTube transcription processing to AutonomousAgentChainRunner (#1647)

- Implemented a new method to extract and process YouTube URLs from user messages.
- Fetch transcriptions using the simpleYoutubeTranscriptionTool and handle errors gracefully.
- Updated conversation messages to include fetched transcriptions for improved context in responses.

* Disable quick command when live-mode is not on (#1648)

* Do not call writeToFileTool again for accepted changes as well (#1652)

* Trigger youtube tool only for urls in prompt not context (#1653)

* Stop calling youtube tool in context notes in agent mode

- Added a new instruction for handling YouTube URLs in the DEFAULT_SYSTEM_PROMPT.
- Removed the direct usage of simpleYoutubeTranscriptionTool in the AutonomousAgentChainRunner, as YouTube transcriptions are now automatically processed.
- Updated parameter naming conventions in modelAdapter for clarity and consistency.
- Cleaned up the simpleYoutubeTranscriptionTool definition by removing unnecessary parameters.

* Add agent loop limit message (#1655)

* Add composer toggle and implement auto-preview (#1651)

* Update composer output format to XML and refactor related components

- Changed the output format for composer instructions from JSON to XML in `constants.ts`.
- Updated `CopilotPlusChainRunner` to utilize the new XML format and adjusted streaming logic accordingly.
- Added a toggle for the composer feature in `ChatInput.tsx` and updated UI elements to reflect this change.
- Removed the `ComposerCodeBlock` component as it is no longer needed with the new implementation.
- Cleaned up unused code and comments in `ChatSingleMessage.tsx` related to the previous composer handling.

* Update src/constants.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/constants.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove redundant tests

* Update doc

* Refactor composer handling and implement ActionBlockStreamer

- Replaced `ComposerBlockStreamer` with `ActionBlockStreamer` to handle `writeToFile` blocks more efficiently.
- Updated `constants.ts` to reflect changes in output format for composer instructions.
- Enhanced `ChatSingleMessage.tsx` to process `writeToFile` sections and integrate collapsible UI elements.
- Removed the deprecated `ComposerBlockStreamer` and its associated tests.
- Improved handling of XML codeblocks and unclosed tags during streaming.

* Fix reported bugs

* Refactor ActionBlockStreamer tests and implementation

- Updated `processChunks` to always push content, including null and empty strings, to the output.
- Adjusted test expectations to reflect changes in output handling for chunks with null content.
- Simplified regex in `findCompleteBlock` method by removing XML format handling.

* Fix reported issue

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix tool description generation (#1659)

* Access tool description from zod.shape when zod.properties is missing.

* Remove the access to schema.properties

* Remove unused print

* Also support schema.propertes

* Implement SimpleTool and remove Langchain tool calling (#1662)

* Make web search tool not overly eager

* Use SimpleTool interface instead of langchain tool call

- Replaced LangChain tool definitions with a new SimpleTool interface for better type safety and validation.
- Updated existing tools (e.g., writeToFile, localSearch, webSearch, and YouTube transcription) to utilize the new createTool function.
- Enhanced parameter extraction from Zod schemas for improved tool description generation.
- Cleaned up related code and documentation to reflect the new tool structure and ensure consistency across the codebase.
- Add tests

* Update ChatManager tests to handle undefined context cases

- Modified test cases in ChatManager to include handling of undefined context parameters.
- Ensured that the tests accurately reflect scenarios where no active file exists and when context is undefined.
- Improved overall test coverage for message processing in various contexts.

* Add validation tests for tool schemas and improve schema definitions

- Introduced comprehensive validation tests for tool schemas in `allTools.validation.test.ts` and `SearchTools.schema.test.ts` to ensure adherence to best practices and proper typing.
- Enhanced `SearchTools.ts` by replacing `z.any()` with specific object schemas for `chatHistory` to improve type safety.
- Added tests for various tool patterns, including metadata validation and schema best practices, ensuring tools are correctly defined and validated against expected interfaces.

* Add back example content for canvas file in ComposerTools schema

* Make web search tool not overly eager

* Add license key in agent mode (#1663)

* Add tool execution tests and implement Plus subscription checks (#1664)

- Introduced unit tests for the `executeSequentialToolCall` function to validate tool execution behavior, including handling of Plus-only tools.
- Enhanced `executeSequentialToolCall` to check for Plus subscription requirements before executing tools.
- Updated `createTool` and `SimpleTool` interfaces to include an `isPlusOnly` flag, indicating tools that require a Copilot Plus subscription.
- Marked existing tools (e.g., webSearch and YouTube transcription) as Plus-only where applicable.

* Add timezone conversion tools and enhance current time functionality (#1665)

* Add timezone conversion tools and enhance current time functionality

- Introduced `convertTimeBetweenTimezonesTool` to allow conversion of time between different timezones.
- Updated `getCurrentTimeTool` to accept an optional timezone parameter, improving its functionality to return time in specified timezones.
- Enhanced documentation and added examples for both tools in the codebase.
- Added comprehensive unit tests for the new timezone conversion functionality to ensure accuracy and reliability.

* Enhance timezone functionality and add unit tests

- Updated `getCurrentTime` and `convertToTimeInfo` functions to utilize Luxon's offset for accurate timezone calculations.
- Adjusted timezone offset handling to reflect correct signs.
- Added unit tests for Tokyo and New York timezone offsets to ensure accuracy in timezone calculations.

* Add unit test for past time conversion in timezone tool

- Introduced a new test case to verify that converting a past time (6:00 AM PT) to Tokyo correctly reflects the same day (January 15) without erroneously adding a day.
- Updated the `convertTimeBetweenTimezones` function to ensure accurate handling of past times without unintended date shifts.

* Enhance timezone handling and update related tests

- Added `convertTimeBetweenTimezonesTool` to facilitate time conversion between different UTC offsets.
- Updated `getCurrentTimeTool` to accept timezone offsets instead of names, improving accuracy and flexibility.
- Enhanced error handling for invalid timezone offset formats and added comprehensive unit tests to validate new functionality.
- Updated existing tests to reflect changes in timezone offset handling and ensure consistent behavior across various scenarios.

* Refactor timezone handling in convertTimeBetweenTimezones function

- Updated the creation of DateTime objects to interpret parsed dates as already being in the source timezone, improving clarity and accuracy in timezone conversions.

* Fix image in chat history (#1666)

* Refactor chat history handling to preserve multimodal content

- Removed the deprecated `extractChatHistory` function and replaced it with direct access to raw history from memory, allowing for the preservation of multimodal content.
- Updated `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to utilize `BaseMessage` objects for chat history, ensuring that both user and AI messages maintain their original structure and content.
- Enhanced message processing to accommodate multimodal inputs, storing them appropriately for later use.

* Refactor chat history handling to improve message processing

- Introduced `addChatHistoryToMessages` utility to streamline the addition of chat history, ensuring safe handling of various message formats.
- Updated `AutonomousAgentChainRunner` and `CopilotPlusChainRunner` to utilize the new utility, enhancing clarity and maintainability of the code.
- Removed deprecated manual processing of chat history to improve code efficiency and readability.

* Refactor chat history processing to enhance multimodal support

- Removed the deprecated `extractChatHistory` function and replaced it with `processRawChatHistory` to ensure consistent handling of chat history.
- Introduced `processedMessagesToTextOnly` function to extract text-only content from multimodal messages, improving clarity for question condensing.
- Updated `CopilotPlusChainRunner` to utilize the new processing functions, ensuring that both LLM and question condensing use the same data format.
- Added comprehensive unit tests for `processedMessagesToTextOnly` to validate its functionality across various message formats.

* Refactor memory management in BaseChainRunner to enhance context saving

- Removed the creation of `HumanMessage` and `AIMessage` objects for chat history, simplifying the process.
- Implemented `saveContext` for atomic operations, ensuring proper memory management while preserving input and output data.
- Noted that LangChain's memory now expects text content, which limits the saving of multimodal content.

* Remove multimodal content storage from userMessage in AutonomousAgentChainRunner and CopilotPlusChainRunner to streamline message processing. This change aligns with recent refactors to enhance memory management and improve clarity in handling chat history.

* Enhance ChatInput component with autonomous agent toggle button (#1669)

- Added a toggle for the autonomous agent feature, allowing users to enable or disable it within the ChatInput component.
- Updated the state management to synchronize the autonomous agent toggle with user settings.
- Introduced a button for toggling the autonomous agent mode, which is only visible in Copilot Plus mode.
- Refactored tool call logic to accommodate the new autonomous agent toggle, ensuring proper handling of tool calls based on the toggle state.

* Implement custom topic in conversation history (#1670)

* Refactor LoadChatHistoryModal and ChatPersistenceManager for enhanced topic handling

- Updated LoadChatHistoryModal to prioritize custom topics from file frontmatter, falling back to filename extraction if not present.
- Enhanced ChatPersistenceManager to generate AI topics for new chat files and preserve existing topics when updating files.
- Introduced new methods for finding files by epoch and generating AI topics based on conversation content, improving chat file management and organization.

* Refactor LoadChatHistoryModal and ChatPersistenceManager for improved topic handling

- Updated LoadChatHistoryModal to ensure custom topics are trimmed and validated before use.
- Enhanced ChatPersistenceManager to utilize reduce for efficient conversation summary generation, introducing constants for message and character limits to improve readability and maintainability.

* Implement tool call UI banner (#1671)

* Only add tool call to the user message when agent is off

- Updated the tool call logic to only add tool calls when the autonomous agent toggle is off, ensuring that the autonomous agent manages all tools internally when enabled.
- Improved code clarity by adding comments to explain the new logic.

* Implement tool call collapsible UI element

- Introduced a new utility for managing tool call markers, allowing for structured display and tracking of tool calls during execution.
- Updated the AutonomousAgentChainRunner to utilize the new tool call markers, improving clarity in the streaming response and tool execution process.
- Added a ToolCallBanner component for better visualization of tool calls in the chat interface, including execution status and results.
- Refactored ChatSingleMessage to handle tool call updates dynamically, ensuring that the UI reflects the current state of tool calls.
- Enhanced CSS for tool call containers and added animations for improved user experience.

* Fix disappearing tool call in final response

* Implement tool call result formatting

- Updated ChatSingleMessage to use a stable ID for message roots, enhancing memory management and preventing leaks.
- Implemented cleanup logic for old message roots to optimize performance.
- Introduced ToolResultFormatter to format tool results for better display in the UI, ensuring user-friendly output for various tool types.
- Enhanced ToolCallBanner to utilize the new formatter, improving the presentation of tool call results in the chat interface.

* Add YouTube transcription tool in agent and avoid mistrigger from context

- Replaced the deprecated simpleYoutubeTranscriptionTool with a new youtubeTranscriptionTool that automatically extracts YouTube URLs from user messages.
- Removed the processYouTubeUrls method to streamline the handling of YouTube transcriptions.
- Updated tool execution logic to pass the original user message to tools that require it, enhancing flexibility in tool usage.
- Improved the ToolResultFormatter to support multi-URL responses and provide better error handling for transcription failures.
- Enhanced documentation for YouTube transcription usage in the codebase.

* Enhance ToolCallBanner and ToolResultFormatter for improved user experience

- Added animation constants to ToolCallBanner for better visual feedback.
- Updated ToolResultFormatter to robustly handle JSON parsing, improving error handling for both single objects and arrays.
- Implemented input validation in YouTube transcription tool to prevent excessive input lengths and ensure proper data types.

* Refactor tool execution messages and enhance JSON parsing in ToolResultFormatter

- Updated the confirmation message for the writeToFile tool to be more concise.
- Added a private method in ToolResultFormatter to robustly parse JSON strings, improving error handling and ensuring consistent results.
- Streamlined the handling of search results by separating JSON parsing and regex extraction logic in ToolResultFormatter.
- Enforced required user message content in the YouTube transcription tool to prevent potential errors.

* Truncate long tool results for compact memory (#1672)

* Truncate long tool results for compact memory

- Introduced `processToolResults` utility to format tool results for different contexts, allowing for both truncated and full results.
- Updated `AutonomousAgentChainRunner` to utilize the new utility for improved memory management and clarity in tool result presentation.
- Added unit tests for `toolResultUtils` to ensure proper functionality of truncation and formatting methods, enhancing reliability and maintainability of tool result processing.

* Enhance tool results handling in AutonomousAgentChainRunner

- Added a conditional check to ensure that only non-null tool results are pushed to the llmFormattedMessages array, improving the robustness of message formatting and preventing potential errors from undefined values.

* Add Agent settings section (#1676)

* Add autonomous agent settings and tool configurations

- Introduced settings for enabling the autonomous agent mode, including maximum iterations and available tools.
- Updated the CopilotPlusSettings component to allow users to toggle tools like local search, web search, and YouTube transcription.
- Enhanced the model adapter to utilize the new settings for tool management.

* Enhance tool management by adding always available tools and updating tool usage guidelines in autonomous agent mode

* Make filetree tool always available

* Refactor tool management system to use a centralized ToolRegistry, enabling dynamic tool registration and configuration. Update settings structure to support enabled tool IDs and enhance UI for tool selection in the Copilot settings.

* Initialize built-in tools with vault access and improve tool registration logic

* Enhance documentation for initializeBuiltinTools function to clarify tool registration process and dynamic filtering of user-enabled tools.

* fix: Fix the shouldIndexFile method to return all files when inclusions is empty in project mode. (#1667)

* feat: Display all category items by default in the context-manage-modal. (#1657)

* Remove escapingXML for context data (#1678)

* Add apply command (#1677)

* Implement ReplaceInFile tool (#1661)

* Relevant note improvements (#1681)

* Change message scrolling behavior (#1680)

* Fix timetools and tests (#1683)

* Refactor time tool descriptions for clarity and examples

* Update XML escaping tests to reflect correct behavior and remove unnecessary settings mock

* Strip partial tool call tag (#1682)

* Strip partial tool call tag

* Show a placeholder message

* Remove perserveToolIndicators option

* Add tool dynamic prompt (#1679)

* feat: Enhance tool execution error messages and update custom prompt instructions for built-in tools

* feat: Add unit tests for ModelAdapter and enhance tool instructions for time-based queries

* feat: Implement detailed formatting for replaceInFile tool results

* feat: Enhance GPT model instructions and refine replaceInFile tool usage guidelines

* feat: Expand documentation on schema descriptions and custom instructions for tool usage

* feat: Refine tool documentation by clarifying schema descriptions and custom prompt instructions

* feat: Update GitHub Actions workflow to run on all pull requests regardless of target branch

* Fix tests

* feat: Update integration tests to validate writeToFile blocks and enhance replaceInFile usage example

* Enhance image extraction logic to support both wiki-style and markdown image syntax (#1685)

* Turn off agent when in Projects mode (#1686)

* Remove think and action blocks from copy and insert (#1687)

* Remove think and action blocks from copy, and fix composer for thinking models

* Refactor ToolSettingsSection layout for improved readability and consistency

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wenzhengjiang <jwzh.hi@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Zero Liu <zero@lumos.com>
Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com>
This commit is contained in:
Logan Yang 2025-08-04 09:51:18 -07:00 committed by GitHub
parent b79b8d0a38
commit bc2d7f660a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
98 changed files with 12499 additions and 2025 deletions

View file

@ -7,7 +7,7 @@ on:
push:
branches: ["master", "main"]
pull_request:
branches: ["master", "main"]
# Run on all pull requests regardless of target branch
jobs:
build:

View file

@ -233,6 +233,7 @@ This helps ensure thorough testing and provides documentation for QA.
- Settings are versioned - migrations may be needed
- Local model support available via Ollama/LM Studio
- Rate limiting is implemented for all API calls
- For technical debt and known issues, see [`TODO.md`](./TODO.md)
### Architecture Migration Notes

View file

@ -326,6 +326,81 @@ Return project-specific repository
## Message Lifecycle
### Example: User Message with Context Note
When a user types "Summarize this note" and attaches "meeting-notes.md":
1. **Input**: User text + attached file → Chat component
2. **Storage**: MessageRepository stores displayText: "Summarize this note"
3. **Processing**: ContextManager reads the note and creates processedText with proper XML structure
4. **Memory Sync**: Chain memory receives the processed version for LLM
5. **UI Update**: Shows message with context badge, displays only "Summarize this note"
6. **LLM Processing**: AI receives full context and generates response
### Context XML Format
All context is wrapped in semantic XML tags for clear structure:
#### Note Context
```xml
<note_context>
<title>meeting-notes</title>
<path>docs/meeting-notes.md</path>
<ctime>2024-01-15T10:00:00.000Z</ctime>
<mtime>2024-01-15T14:30:00.000Z</mtime>
<content>
[actual note content here]
</content>
</note_context>
```
#### URL Context
```xml
<url_content>
<url>https://example.com/article</url>
<content>
[fetched content from URL]
</content>
</url_content>
```
#### Selected Text Context
```xml
<selected_text>
<title>Source Note Title</title>
<path>path/to/source.md</path>
<start_line>45</start_line>
<end_line>52</end_line>
<content>
[selected text content]
</content>
</selected_text>
```
#### Error Cases
```xml
<note_context_error>
<title>filename</title>
<path>path/to/file.ext</path>
<error>[Error: Could not process file]</error>
</note_context_error>
```
This separation ensures:
- Clean UI (shows what user typed)
- Rich context for AI (includes note content)
- Reprocessable context on message edits
For detailed examples, see:
- `src/core/MessageLifecycle.test.ts` - Complete lifecycle demonstration with context notes
- `src/core/MessageLifecycle.xmltags.test.ts` - XML tag formatting tests and examples
### 1. Sending a New Message
```
@ -619,4 +694,6 @@ console.log({
- `src/core/MessageRepository.test.ts` - Repository tests
- `src/core/ChatManager.test.ts` - Manager tests
- `src/core/MessageLifecycle.test.ts` - Complete lifecycle examples with context notes
- `src/core/MessageLifecycle.xmltags.test.ts` - XML tag formatting tests and examples
- `src/components/chat-components/MessageContext.test.tsx` - Context display tests

51
TODO.md Normal file
View file

@ -0,0 +1,51 @@
# TODO - Technical Debt & Future Improvements
This document tracks technical debt items and improvements that need to be addressed in the future.
## 1. Docs4LLM SSL Error in Projects Mode
### Issue Description
Document parsing for projects mode is failing with an SSL error (`net::ERR_SSL_BAD_RECORD_MAC_ALERT`) when trying to upload files to the docs4llm API endpoint.
### Technical Details
- **Error Location**: `src/LLMProviders/brevilabsClient.ts:185` in `makeFormDataRequest` method
- **Root Cause**: The method uses native `fetch` API instead of Obsidian's `requestUrl` API
- **Context**: While regular JSON requests were migrated to use `safeFetch` (which uses `requestUrl`) in commit e49aafa to fix CORS issues, the `makeFormDataRequest` method was not updated
### Why Current Approaches Won't Work
1. **Backend Constraint**: The `/docs4llm` endpoint only accepts multipart/form-data format with `files: List[UploadFile]`
2. **Obsidian Limitation**: The existing `safeFetch` function is hardcoded for `application/json` content type
3. **No JSON Alternative**: Unlike `pdf4llm` which accepts base64 JSON, there's no JSON endpoint for `docs4llm`
### Recommended Solution
Create a new `safeFetchFormData` function that:
1. Uses Obsidian's `requestUrl` API with proper multipart/form-data configuration
2. Handles FormData objects correctly
3. Bypasses CORS and SSL restrictions like `safeFetch` does for JSON
### Alternative Solutions
1. **Backend Modification**: Add a new `/docs4llm-base64` endpoint that accepts base64-encoded JSON payloads
2. **Research Obsidian API**: Investigate if newer versions of Obsidian's `requestUrl` support multipart/form-data
3. **SSL Certificate Fix**: Address the underlying SSL certificate issue (temporary workaround)
### Impact
- Users cannot parse non-markdown files (PDFs, Word docs, etc.) in projects mode
- This affects the core functionality of project context loading
- Workaround: Users must ensure their projects only contain markdown files
### References
- Related commit: e49aafa (Brevilabs CORS issue #918)
- Forum discussion: https://forum.obsidian.md/t/holo-how-to-add-a-png-image-or-file-to-formdata-in-obsidian-like-below-this-help/73420
- Backend implementation: `/Users/chaoyang/webapps/brevilabs-api/app/main.py:1039`
---
_Last updated: 2025-07-18_

View file

@ -46,4 +46,47 @@ module.exports = {
read: jest.fn(),
},
})),
ItemView: jest.fn().mockImplementation(function () {
this.containerEl = document.createElement("div");
this.onOpen = jest.fn();
this.onClose = jest.fn();
this.getDisplayText = jest.fn().mockReturnValue("Mock View");
this.getViewType = jest.fn().mockReturnValue("mock-view");
this.getIcon = jest.fn().mockReturnValue("document");
}),
Notice: jest.fn().mockImplementation(function (message) {
this.message = message;
this.noticeEl = document.createElement("div");
this.hide = jest.fn();
}),
TFile: jest.fn().mockImplementation(function (path) {
this.path = path;
this.name = path.split("/").pop();
this.basename = this.name.replace(/\.[^/.]+$/, "");
this.extension = path.split(".").pop();
}),
WorkspaceLeaf: jest.fn().mockImplementation(function () {
this.view = null;
this.setViewState = jest.fn();
this.detach = jest.fn();
this.getViewState = jest.fn().mockReturnValue({});
}),
};
// Mock the global app object
global.app = {
vault: {
getAbstractFileByPath: jest.fn().mockReturnValue({
name: "test-file.md",
path: "test-file.md",
}),
read: jest.fn().mockResolvedValue("test content"),
modify: jest.fn().mockResolvedValue(undefined),
},
workspace: {
getActiveFile: jest.fn().mockReturnValue(null),
getLeaf: jest.fn().mockReturnValue({
openFile: jest.fn().mockResolvedValue(undefined),
}),
},
};

View file

@ -1,7 +1,7 @@
{
"id": "copilot",
"name": "Copilot",
"version": "2.9.4",
"version": "3.0.0-preview-250720",
"minAppVersion": "0.15.0",
"description": "Your AI Copilot: Chat with Your Second Brain, Learn Faster, Work Smarter.",
"author": "Logan Yang",

View file

@ -13,7 +13,8 @@ import {
LLMChainRunner,
ProjectChainRunner,
VaultQAChainRunner,
} from "@/LLMProviders/chainRunner";
AutonomousAgentChainRunner,
} from "@/LLMProviders/chainRunner/index";
import { logError, logInfo } from "@/logger";
import { HybridRetriever } from "@/search/hybridRetriever";
import VectorStoreManager from "@/search/vectorStoreManager";
@ -268,12 +269,18 @@ export default class ChainManager {
private getChainRunner(): ChainRunner {
const chainType = getChainType();
const settings = getSettings();
switch (chainType) {
case ChainType.LLM_CHAIN:
return new LLMChainRunner(this);
case ChainType.VAULT_QA_CHAIN:
return new VaultQAChainRunner(this);
case ChainType.COPILOT_PLUS_CHAIN:
// Use AutonomousAgentChainRunner if the setting is enabled
if (settings.enableAutonomousAgent) {
return new AutonomousAgentChainRunner(this);
}
return new CopilotPlusChainRunner(this);
case ChainType.PROJECT_CHAIN:
return new ProjectChainRunner(this);

View file

@ -1,983 +0,0 @@
import { getCurrentProject } from "@/aiParams";
import { getStandaloneQuestion } from "@/chainUtils";
import {
ABORT_REASON,
AI_SENDER,
EMPTY_INDEX_ERROR_MESSAGE,
LOADING_MESSAGES,
MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT,
ModelCapability,
} from "@/constants";
import {
ImageBatchProcessor,
ImageContent,
ImageProcessingResult,
MessageContent,
} from "@/imageProcessing/imageProcessor";
import { BrevilabsClient } from "@/LLMProviders/brevilabsClient";
import { logError, logInfo, logWarn } from "@/logger";
import { HybridRetriever } from "@/search/hybridRetriever";
import { getSettings, getSystemPrompt } from "@/settings/model";
import { ChatMessage } from "@/types/message";
import { ToolManager } from "@/tools/toolManager";
import {
err2String,
extractChatHistory,
extractUniqueTitlesFromDocs,
extractYoutubeUrl,
formatDateTime,
getApiErrorMessage,
getMessageRole,
withSuppressedTokenWarnings,
} from "@/utils";
import { BaseChatModel } from "@langchain/core/language_models/chat_models";
import { Notice } from "obsidian";
import ChainManager from "./chainManager";
import { COPILOT_TOOL_NAMES, IntentAnalyzer } from "./intentAnalyzer";
import ProjectManager from "./projectManager";
class ThinkBlockStreamer {
private hasOpenThinkBlock = false;
private fullResponse = "";
constructor(private updateCurrentAiMessage: (message: string) => void) {}
private handleClaude37Chunk(content: any[]) {
let textContent = "";
for (const item of content) {
switch (item.type) {
case "text":
textContent += item.text;
break;
case "thinking":
if (!this.hasOpenThinkBlock) {
this.fullResponse += "\n<think>";
this.hasOpenThinkBlock = true;
}
this.fullResponse += item.thinking;
this.updateCurrentAiMessage(this.fullResponse);
return true; // Indicate we handled a thinking chunk
}
}
if (textContent) {
this.fullResponse += textContent;
}
return false; // No thinking chunk handled
}
private handleDeepseekChunk(chunk: any) {
// Handle standard string content
if (typeof chunk.content === "string") {
this.fullResponse += chunk.content;
}
// Handle deepseek reasoning/thinking content
if (chunk.additional_kwargs?.reasoning_content) {
if (!this.hasOpenThinkBlock) {
this.fullResponse += "\n<think>";
this.hasOpenThinkBlock = true;
}
this.fullResponse += chunk.additional_kwargs.reasoning_content;
return true; // Indicate we handled a thinking chunk
}
return false; // No thinking chunk handled
}
processChunk(chunk: any) {
let handledThinking = false;
// Handle Claude 3.7 array-based content
if (Array.isArray(chunk.content)) {
handledThinking = this.handleClaude37Chunk(chunk.content);
} else {
// Handle deepseek format
handledThinking = this.handleDeepseekChunk(chunk);
}
// Close think block if we have one open and didn't handle thinking content
if (this.hasOpenThinkBlock && !handledThinking) {
this.fullResponse += "</think>";
this.hasOpenThinkBlock = false;
}
this.updateCurrentAiMessage(this.fullResponse);
}
close() {
// Make sure to close any open think block at the end
if (this.hasOpenThinkBlock) {
this.fullResponse += "</think>";
this.updateCurrentAiMessage(this.fullResponse);
}
return this.fullResponse;
}
}
export interface ChainRunner {
run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string>;
}
abstract class BaseChainRunner implements ChainRunner {
protected chainManager: ChainManager;
constructor(chainManager: ChainManager) {
this.chainManager = chainManager;
}
abstract run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string>;
protected async handleResponse(
fullAIResponse: string,
userMessage: ChatMessage,
abortController: AbortController,
addMessage: (message: ChatMessage) => void,
updateCurrentAiMessage: (message: string) => void,
sources?: { title: string; score: number }[]
) {
// Save to memory and add message if we have a response
// Skip only if it's a NEW_CHAT abort (clearing everything)
if (
fullAIResponse &&
!(abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT)
) {
await this.chainManager.memoryManager
.getMemory()
.saveContext({ input: userMessage.message }, { output: fullAIResponse });
addMessage({
message: fullAIResponse,
sender: AI_SENDER,
isVisible: true,
timestamp: formatDateTime(new Date()),
sources: sources,
});
// Clear the streaming message since it's now in chat history
updateCurrentAiMessage("");
} else if (abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
// Also clear if it's a new chat
updateCurrentAiMessage("");
}
logInfo(
"==== Chat Memory ====\n",
(this.chainManager.memoryManager.getMemory().chatHistory as any).messages.map(
(m: any) => m.content
)
);
logInfo("==== Final AI Response ====\n", fullAIResponse);
return fullAIResponse;
}
protected async handleError(
error: any,
addMessage?: (message: ChatMessage) => void,
updateCurrentAiMessage?: (message: string) => void
) {
const msg = err2String(error);
logError("Error during LLM invocation:", msg);
const errorData = error?.response?.data?.error || msg;
const errorCode = errorData?.code || msg;
let errorMessage = "";
// Check for specific error messages
if (error?.message?.includes("Invalid license key")) {
errorMessage = "Invalid Copilot Plus license key. Please check your license key in settings.";
} else if (errorCode === "model_not_found") {
errorMessage =
"You do not have access to this model or the model does not exist, please check with your API provider.";
} else {
errorMessage = `${errorCode}`;
}
logError(errorData);
if (addMessage && updateCurrentAiMessage) {
updateCurrentAiMessage("");
// remove langchain troubleshooting URL from error message
const ignoreEndIndex = errorMessage.search("Troubleshooting URL");
errorMessage = ignoreEndIndex !== -1 ? errorMessage.slice(0, ignoreEndIndex) : errorMessage;
// add more user guide for invalid API key
if (msg.search(/401|invalid|not valid/gi) !== -1) {
errorMessage =
"Something went wrong. Please check if you have set your API key." +
"\nPath: Settings > copilot plugin > Basic Tab > Set Keys." +
"\nOr check model config" +
"\nError Details: " +
errorMessage;
}
addMessage({
message: errorMessage,
isErrorMessage: true,
sender: AI_SENDER,
isVisible: true,
timestamp: formatDateTime(new Date()),
});
} else {
// Fallback to Notice if message handlers aren't provided
new Notice(errorMessage);
logError(errorData);
}
}
}
class LLMChainRunner extends BaseChainRunner {
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string> {
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage);
try {
// Get chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
// Create messages array starting with system message
const messages: any[] = [];
// Add system message if available
const systemPrompt = getSystemPrompt();
const chatModel = this.chainManager.chatModelManager.getChatModel();
if (systemPrompt) {
messages.push({
role: getMessageRole(chatModel),
content: systemPrompt,
});
}
// Add chat history
for (const entry of chatHistory) {
messages.push({ role: entry.role, content: entry.content });
}
// Add current user message
messages.push({
role: "user",
content: userMessage.message,
});
logInfo("==== Final Request to AI ====\n", messages);
// Stream with abort signal
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("Stream iteration aborted", { reason: abortController.signal.reason });
break;
}
streamer.processChunk(chunk);
}
} catch (error: any) {
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("Stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Always return the response, even if partial
const response = streamer.close();
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
return this.handleResponse(
response,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
}
class VaultQAChainRunner extends BaseChainRunner {
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string> {
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage);
try {
// Add check for empty index
const indexEmpty = await this.chainManager.vectorStoreManager.isIndexEmpty();
if (indexEmpty) {
return this.handleResponse(
EMPTY_INDEX_ERROR_MESSAGE,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
// Get chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
// Generate standalone question from user message + chat history
// This is similar to what the conversational retrieval chain does
let standaloneQuestion = userMessage.message;
if (chatHistory.length > 0) {
// For simplicity, we'll use the original question directly
// The original chain would rephrase it, but this approach should work for most cases
standaloneQuestion = userMessage.message;
}
// Create retriever (similar to how it's done in chainManager)
const retriever = new HybridRetriever({
minSimilarityScore: 0.01,
maxK: getSettings().maxSourceChunks,
salientTerms: [],
});
// Retrieve relevant documents
const retrievedDocs = await retriever.getRelevantDocuments(standaloneQuestion);
// Store retrieved documents for sources
this.chainManager.storeRetrieverDocuments(retrievedDocs);
// Format documents as context
const context = retrievedDocs.map((doc: any) => doc.pageContent).join("\n\n");
// Create messages array
const messages: any[] = [];
// Add system message with QA instruction
const systemPrompt = getSystemPrompt();
const qaInstructions =
"\n\nAnswer the question with as detailed as possible based only on the following context:\n" +
context;
const fullSystemMessage = systemPrompt + qaInstructions;
const chatModel = this.chainManager.chatModelManager.getChatModel();
if (fullSystemMessage) {
messages.push({
role: getMessageRole(chatModel),
content: fullSystemMessage,
});
}
// Add chat history
for (const entry of chatHistory) {
messages.push({ role: entry.role, content: entry.content });
}
// Add current user question
messages.push({
role: "user",
content: userMessage.message,
});
logInfo("==== Final Request to AI ====\n", messages);
// Stream with abort signal
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("VaultQA stream iteration aborted", { reason: abortController.signal.reason });
break;
}
streamer.processChunk(chunk);
}
} catch (error: any) {
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("VaultQA stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Always get the response, even if partial
let fullAIResponse = streamer.close();
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
// Add sources to the response
fullAIResponse = this.addSourcestoResponse(fullAIResponse);
return this.handleResponse(
fullAIResponse,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
private addSourcestoResponse(response: string): string {
const docTitles = extractUniqueTitlesFromDocs(this.chainManager.getRetrievedDocuments());
if (docTitles.length > 0) {
const links = docTitles.map((title) => `- [[${title}]]`).join("\n");
response += "\n\n#### Sources:\n\n" + links;
}
return response;
}
}
class CopilotPlusChainRunner extends BaseChainRunner {
private isYoutubeOnlyMessage(message: string): boolean {
const trimmedMessage = message.trim();
const hasYoutubeCommand = trimmedMessage.includes("@youtube");
const youtubeUrl = extractYoutubeUrl(trimmedMessage);
// Check if message only contains @youtube command and a valid URL
const words = trimmedMessage
.split(/\s+/)
.filter((word) => word !== "@youtube" && word.length > 0);
return hasYoutubeCommand && youtubeUrl !== null && words.length === 1;
}
private async processImageUrls(urls: string[]): Promise<ImageProcessingResult> {
const failedImages: string[] = [];
const processedImages = await ImageBatchProcessor.processUrlBatch(
urls,
failedImages,
this.chainManager.app.vault
);
ImageBatchProcessor.showFailedImagesNotice(failedImages);
return processedImages;
}
private async processChatInputImages(content: MessageContent[]): Promise<ImageProcessingResult> {
const failedImages: string[] = [];
const processedImages = await ImageBatchProcessor.processChatImageBatch(
content,
failedImages,
this.chainManager.app.vault
);
ImageBatchProcessor.showFailedImagesNotice(failedImages);
return processedImages;
}
private async extractEmbeddedImages(content: string): Promise<string[]> {
const imageRegex = /!\[\[(.*?\.(png|jpg|jpeg|gif|webp|bmp|svg))\]\]/g;
const matches = [...content.matchAll(imageRegex)];
const images = matches.map((match) => match[1]);
return images;
}
private async buildMessageContent(
textContent: string,
userMessage: ChatMessage
): Promise<MessageContent[]> {
const failureMessages: string[] = [];
const successfulImages: ImageContent[] = [];
const settings = getSettings();
// Collect all image sources
const imageSources: { urls: string[]; type: string }[] = [];
// Safely check and add context URLs
const contextUrls = userMessage.context?.urls;
if (contextUrls && contextUrls.length > 0) {
imageSources.push({ urls: contextUrls, type: "context" });
}
// Process embedded images only if setting is enabled
if (settings.passMarkdownImages) {
const embeddedImages = await this.extractEmbeddedImages(textContent);
if (embeddedImages.length > 0) {
imageSources.push({ urls: embeddedImages, type: "embedded" });
}
}
// Process all image sources
for (const source of imageSources) {
const result = await this.processImageUrls(source.urls);
successfulImages.push(...result.successfulImages);
failureMessages.push(...result.failureDescriptions);
}
// Process existing chat content images if present
const existingContent = userMessage.content;
if (existingContent && existingContent.length > 0) {
const result = await this.processChatInputImages(existingContent);
successfulImages.push(...result.successfulImages);
failureMessages.push(...result.failureDescriptions);
}
// Let the LLM know about the image processing failures
let finalText = textContent;
if (failureMessages.length > 0) {
finalText = `${textContent}\n\nNote: \n${failureMessages.join("\n")}\n`;
}
const messageContent: MessageContent[] = [
{
type: "text",
text: finalText,
},
];
// Add successful images after the text content
if (successfulImages.length > 0) {
messageContent.push(...successfulImages);
}
return messageContent;
}
private hasCapability(model: BaseChatModel, capability: ModelCapability): boolean {
const modelName = (model as any).modelName || (model as any).model || "";
const customModel = this.chainManager.chatModelManager.findModelByName(modelName);
return customModel?.capabilities?.includes(capability) ?? false;
}
private isMultimodalModel(model: BaseChatModel): boolean {
return this.hasCapability(model, ModelCapability.VISION);
}
private async streamMultimodalResponse(
textContent: string,
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void
): Promise<string> {
// Get chat history
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
// Create messages array starting with system message
const messages: any[] = [];
// Add system message if available
let fullSystemMessage = await this.getSystemPrompt();
// Add chat history context to system message if exists
if (chatHistory.length > 0) {
fullSystemMessage +=
"\n\nThe following is the relevant conversation history. Use this context to maintain consistency in your responses:";
}
// Get chat model for role determination for O-series models
const chatModel = this.chainManager.chatModelManager.getChatModel();
// Add the combined system message with appropriate role
if (fullSystemMessage) {
messages.push({
role: getMessageRole(chatModel),
content: `${fullSystemMessage}\nIMPORTANT: Maintain consistency with previous responses in the conversation. If you've provided information about a person or topic before, use that same information in follow-up questions.`,
});
}
// Add chat history
for (const entry of chatHistory) {
messages.push({ role: entry.role, content: entry.content });
}
// Get the current chat model
const chatModelCurrent = this.chainManager.chatModelManager.getChatModel();
const isMultimodalCurrent = this.isMultimodalModel(chatModelCurrent);
// Build message content with text and images for multimodal models, or just text for text-only models
const content = isMultimodalCurrent
? await this.buildMessageContent(textContent, userMessage)
: textContent;
// Add current user message
messages.push({
role: "user",
content,
});
const enhancedUserMessage = content instanceof Array ? (content[0] as any).text : content;
logInfo("Enhanced user message: ", enhancedUserMessage);
logInfo("==== Final Request to AI ====\n", messages);
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage);
// Wrap the stream call with warning suppression
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("CopilotPlus multimodal stream iteration aborted", {
reason: abortController.signal.reason,
});
break;
}
streamer.processChunk(chunk);
}
return streamer.close();
}
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
updateLoadingMessage?: (message: string) => void;
}
): Promise<string> {
const { updateLoadingMessage } = options;
let fullAIResponse = "";
let sources: { title: string; score: number }[] = [];
let currentPartialResponse = "";
// Wrapper to track partial response
const trackAndUpdateAiMessage = (message: string) => {
currentPartialResponse = message;
updateCurrentAiMessage(message);
};
try {
// Check if this is a YouTube-only message
if (this.isYoutubeOnlyMessage(userMessage.message)) {
const url = extractYoutubeUrl(userMessage.message);
const failMessage =
"Transcript not available. Only videos with the auto transcript option turned on are supported at the moment.";
if (url) {
try {
const response = await BrevilabsClient.getInstance().youtube4llm(url);
if (response.response.transcript) {
return this.handleResponse(
response.response.transcript,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
return this.handleResponse(
failMessage,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
} catch (error) {
logError("Error processing YouTube video:", error);
return this.handleResponse(
failMessage,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
}
}
logInfo("==== Step 1: Analyzing intent ====");
let toolCalls;
// Use the original message for intent analysis
const messageForAnalysis = userMessage.originalMessage || userMessage.message;
try {
toolCalls = await IntentAnalyzer.analyzeIntent(messageForAnalysis);
} catch (error: any) {
return this.handleResponse(
getApiErrorMessage(error),
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
// Use the same removeAtCommands logic as IntentAnalyzer
const cleanedUserMessage = userMessage.message
.split(" ")
.filter((word) => !COPILOT_TOOL_NAMES.includes(word.toLowerCase()))
.join(" ")
.trim();
const toolOutputs = await this.executeToolCalls(toolCalls, updateLoadingMessage);
const localSearchResult = toolOutputs.find(
(output) => output.tool === "localSearch" && output.output && output.output.length > 0
);
// Format chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
if (localSearchResult) {
logInfo("==== Step 2: Processing local search results ====");
const documents = JSON.parse(localSearchResult.output);
logInfo("==== Step 3: Condensing Question ====");
const standaloneQuestion = await getStandaloneQuestion(cleanedUserMessage, chatHistory);
logInfo("Condensed standalone question: ", standaloneQuestion);
logInfo("==== Step 4: Preparing context ====");
const timeExpression = this.getTimeExpression(toolCalls);
const context = this.prepareLocalSearchResult(documents, timeExpression);
const currentTimeOutputs = toolOutputs.filter((output) => output.tool === "getCurrentTime");
const enhancedQuestion = this.prepareEnhancedUserMessage(
standaloneQuestion,
currentTimeOutputs
);
logInfo(context);
logInfo("==== Step 5: Invoking QA Chain ====");
const qaPrompt = await this.chainManager.promptManager.getQAPrompt({
question: enhancedQuestion,
context,
systemMessage: "", // System prompt is added separately in streamMultimodalResponse
});
fullAIResponse = await this.streamMultimodalResponse(
qaPrompt,
userMessage,
abortController,
trackAndUpdateAiMessage
);
// Append sources to the response
sources = this.getSources(documents);
} else {
// Enhance with tool outputs.
const enhancedUserMessage = this.prepareEnhancedUserMessage(
cleanedUserMessage,
toolOutputs
);
// If no results, default to LLM Chain
logInfo("No local search results. Using standard LLM Chain.");
fullAIResponse = await this.streamMultimodalResponse(
enhancedUserMessage,
userMessage,
abortController,
trackAndUpdateAiMessage
);
}
} catch (error: any) {
// Reset loading message to default
updateLoadingMessage?.(LOADING_MESSAGES.DEFAULT);
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("CopilotPlus stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
// If aborted but not a new chat, use the partial response
if (abortController.signal.aborted && currentPartialResponse) {
fullAIResponse = currentPartialResponse;
}
return this.handleResponse(
fullAIResponse,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage,
sources
);
}
private getSources(documents: any): { title: string; score: number }[] {
if (!documents || !Array.isArray(documents)) {
logWarn("No valid documents provided to getSources");
return [];
}
return this.sortUniqueDocsByScore(documents);
}
private sortUniqueDocsByScore(documents: any[]): any[] {
const uniqueDocs = new Map<string, any>();
// Iterate through all documents
for (const doc of documents) {
if (!doc.title || (!doc?.score && !doc?.rerank_score)) {
logWarn("Invalid document structure:", doc);
continue;
}
const currentDoc = uniqueDocs.get(doc.title);
const isReranked = doc && "rerank_score" in doc;
const docScore = isReranked ? doc.rerank_score : doc.score;
// If the title doesn't exist in the map, or if the new doc has a higher score, update the map
if (!currentDoc || docScore > (currentDoc.score ?? 0)) {
uniqueDocs.set(doc.title, {
title: doc.title,
score: docScore,
isReranked: isReranked,
});
}
}
// Convert the map values back to an array and sort by score in descending order
return Array.from(uniqueDocs.values()).sort((a, b) => (b.score ?? 0) - (a.score ?? 0));
}
private async executeToolCalls(
toolCalls: any[],
updateLoadingMessage?: (message: string) => void
) {
const toolOutputs = [];
for (const toolCall of toolCalls) {
logInfo(`==== Step 2: Calling tool: ${toolCall.tool.name} ====`);
if (toolCall.tool.name === "localSearch") {
updateLoadingMessage?.(LOADING_MESSAGES.READING_FILES);
} else if (toolCall.tool.name === "webSearch") {
updateLoadingMessage?.(LOADING_MESSAGES.SEARCHING_WEB);
} else if (toolCall.tool.name === "getFileTree") {
updateLoadingMessage?.(LOADING_MESSAGES.READING_FILE_TREE);
}
const output = await ToolManager.callTool(toolCall.tool, toolCall.args);
toolOutputs.push({ tool: toolCall.tool.name, output });
}
return toolOutputs;
}
private prepareEnhancedUserMessage(userMessage: string, toolOutputs: any[]) {
let context = "";
if (toolOutputs.length > 0) {
const validOutputs = toolOutputs.filter((output) => output.output != null);
if (validOutputs.length > 0) {
context =
"\n\n# Additional context:\n\n" +
validOutputs
.map(
(output) =>
`<${output.tool}>\n${typeof output.output !== "string" ? JSON.stringify(output.output) : output.output}\n</${output.tool}>`
)
.join("\n\n");
}
}
return `${userMessage}${context}`;
}
private getTimeExpression(toolCalls: any[]): string {
const timeRangeCall = toolCalls.find((call) => call.tool.name === "getTimeRangeMs");
return timeRangeCall ? timeRangeCall.args.timeExpression : "";
}
private prepareLocalSearchResult(documents: any[], timeExpression: string): string {
// First filter documents with includeInContext
const includedDocs = documents.filter((doc) => doc.includeInContext);
// Calculate total content length
const totalLength = includedDocs.reduce((sum, doc) => sum + doc.content.length, 0);
// If total length exceeds threshold, calculate truncation ratio
let truncatedDocs = includedDocs;
if (totalLength > MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT) {
const truncationRatio = MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT / totalLength;
logInfo("Truncating documents to fit context length. Truncation ratio:", truncationRatio);
truncatedDocs = includedDocs.map((doc) => ({
...doc,
content: doc.content.slice(0, Math.floor(doc.content.length * truncationRatio)),
}));
}
const formattedDocs = truncatedDocs
.map((doc: any) => `Note in Vault: ${doc.content}`)
.join("\n\n");
return timeExpression
? `Local Search Result for ${timeExpression}:\n${formattedDocs}`
: `Local Search Result:\n${formattedDocs}`;
}
protected async getSystemPrompt(): Promise<string> {
return getSystemPrompt();
}
}
class ProjectChainRunner extends CopilotPlusChainRunner {
protected async getSystemPrompt(): Promise<string> {
let finalPrompt = getSystemPrompt();
const projectConfig = getCurrentProject();
if (!projectConfig) {
return finalPrompt;
}
// Get context asynchronously
const context = await ProjectManager.instance.getProjectContext(projectConfig.id);
finalPrompt = `${finalPrompt}\n\n<project_system_prompt>\n${projectConfig.systemPrompt}\n</project_system_prompt>`;
// TODO: Move project context out of the system prompt and into the user prompt.
if (context) {
finalPrompt = `${finalPrompt}\n\n <project_context>\n${context}\n</project_context>`;
}
return finalPrompt;
}
}
export { CopilotPlusChainRunner, LLMChainRunner, ProjectChainRunner, VaultQAChainRunner };

View file

@ -0,0 +1,519 @@
import { MessageContent } from "@/imageProcessing/imageProcessor";
import { logError, logInfo, logWarn } from "@/logger";
import { checkIsPlusUser } from "@/plusUtils";
import { getSettings, getSystemPrompt } from "@/settings/model";
import { extractParametersFromZod, SimpleTool } from "@/tools/SimpleTool";
import { ToolRegistry } from "@/tools/ToolRegistry";
import { initializeBuiltinTools } from "@/tools/builtinTools";
import { ChatMessage } from "@/types/message";
import { getMessageRole, withSuppressedTokenWarnings } from "@/utils";
import { processToolResults } from "@/utils/toolResultUtils";
import { CopilotPlusChainRunner } from "./CopilotPlusChainRunner";
import { addChatHistoryToMessages } from "./utils/chatHistoryUtils";
import { messageRequiresTools, ModelAdapter, ModelAdapterFactory } from "./utils/modelAdapter";
import { ThinkBlockStreamer } from "./utils/ThinkBlockStreamer";
import { createToolCallMarker, updateToolCallMarker } from "./utils/toolCallParser";
import {
deduplicateSources,
executeSequentialToolCall,
getToolConfirmtionMessage,
getToolDisplayName,
getToolEmoji,
logToolCall,
logToolResult,
ToolExecutionResult,
} from "./utils/toolExecution";
import { parseXMLToolCalls, stripToolCallXML } from "./utils/xmlParsing";
export class AutonomousAgentChainRunner extends CopilotPlusChainRunner {
private llmFormattedMessages: string[] = []; // Track LLM-formatted messages for memory
private getAvailableTools(): SimpleTool<any, any>[] {
const settings = getSettings();
const registry = ToolRegistry.getInstance();
// Initialize tools if not already done
if (registry.getAllTools().length === 0) {
initializeBuiltinTools(this.chainManager.app?.vault);
}
// Get enabled tool IDs from settings
const enabledToolIds = new Set(settings.autonomousAgentEnabledToolIds || []);
// Get all enabled tools from registry
return registry.getEnabledTools(enabledToolIds, !!this.chainManager.app?.vault);
}
private generateToolDescriptions(): string {
const tools = this.getAvailableTools();
return tools
.map((tool) => {
let params = "";
// All tools now have Zod schema
const parameters = extractParametersFromZod(tool.schema);
if (Object.keys(parameters).length > 0) {
params = Object.entries(parameters)
.map(([key, description]) => `<${key}>${description}</${key}>`)
.join("\n");
}
return `<${tool.name}>
<description>${tool.description}</description>
<parameters>
${params}
</parameters>
</${tool.name}>`;
})
.join("\n\n");
}
private buildIterationDisplay(
iterationHistory: string[],
currentIteration: number,
currentMessage: string
): string {
// Simply join all history without headers or separators
const allParts = [...iterationHistory];
// Add current message if present
if (currentMessage) {
allParts.push(currentMessage);
}
// Join with simple spacing
return allParts.join("\n\n");
}
private generateSystemPrompt(): string {
const basePrompt = getSystemPrompt();
const toolDescriptions = this.generateToolDescriptions();
const availableTools = this.getAvailableTools();
const toolNames = availableTools.map((tool) => tool.name);
// Get tool metadata for custom instructions
const registry = ToolRegistry.getInstance();
const toolMetadata = availableTools
.map((tool) => registry.getToolMetadata(tool.name))
.filter((meta): meta is NonNullable<typeof meta> => meta !== undefined);
// Use model adapter for clean model-specific handling
const chatModel = this.chainManager.chatModelManager.getChatModel();
const adapter = ModelAdapterFactory.createAdapter(chatModel);
return adapter.enhanceSystemPrompt(basePrompt, toolDescriptions, toolNames, toolMetadata);
}
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
updateLoadingMessage?: (message: string) => void;
}
): Promise<string> {
let fullAIResponse = "";
const conversationMessages: any[] = [];
const iterationHistory: string[] = []; // Track all iterations for display
const collectedSources: { title: string; path: string; score: number }[] = []; // Collect sources from localSearch
this.llmFormattedMessages = []; // Reset LLM messages for this run
const isPlusUser = await checkIsPlusUser();
if (!isPlusUser) {
await this.handleError(new Error("Invalid license key"), addMessage, updateCurrentAiMessage);
return "";
}
try {
// Get chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
// Use raw history to preserve multimodal content
const rawHistory = memoryVariables.history || [];
// Build initial conversation messages
const customSystemPrompt = this.generateSystemPrompt();
const chatModel = this.chainManager.chatModelManager.getChatModel();
const adapter = ModelAdapterFactory.createAdapter(chatModel);
if (customSystemPrompt) {
conversationMessages.push({
role: getMessageRole(chatModel),
content: customSystemPrompt,
});
}
// Add chat history - safely handle different message formats
addChatHistoryToMessages(rawHistory, conversationMessages);
// Check if the model supports multimodal (vision) capability
const isMultimodal = this.isMultimodalModel(chatModel);
// Add current user message with model-specific enhancements
const requiresTools = messageRequiresTools(userMessage.message);
const enhancedUserMessage = adapter.enhanceUserMessage(userMessage.message, requiresTools);
// Build message content with images if multimodal, otherwise just use text
const content: string | MessageContent[] = isMultimodal
? await this.buildMessageContent(enhancedUserMessage, userMessage)
: enhancedUserMessage;
conversationMessages.push({
role: "user",
content,
});
// Store original user prompt for tools that need it
const originalUserPrompt = userMessage.originalMessage || userMessage.message;
// Autonomous agent loop
const maxIterations = getSettings().autonomousAgentMaxIterations; // Get from settings
let iteration = 0;
while (iteration < maxIterations) {
if (abortController.signal.aborted) {
break;
}
iteration++;
logInfo(`=== Autonomous Agent Iteration ${iteration} ===`);
// Store tool call messages for this iteration (declared here so it's accessible in the streaming callback)
const currentIterationToolCallMessages: string[] = [];
// Get AI response
const response = await this.streamResponse(
conversationMessages,
abortController,
(message) => {
// Show tool calls as indicators during streaming for clarity, preserve think blocks
const cleanedMessage = stripToolCallXML(message);
// Build display with ALL content including tool calls from history
const displayParts = [];
// Add all iteration history (which includes tool call markers)
displayParts.push(...iterationHistory);
// Add current iteration's tool calls if any
if (currentIterationToolCallMessages.length > 0) {
displayParts.push(currentIterationToolCallMessages.join("\n"));
}
// Add the current streaming message
if (cleanedMessage.trim()) {
displayParts.push(cleanedMessage);
}
const currentDisplay = displayParts.join("\n\n");
updateCurrentAiMessage(currentDisplay);
},
adapter
);
if (!response) break;
// Parse tool calls from the response
const toolCalls = parseXMLToolCalls(response);
// Use model adapter to detect and handle premature responses
const prematureResponseResult = adapter.detectPrematureResponse?.(response);
if (prematureResponseResult?.hasPremature && iteration === 1) {
if (prematureResponseResult.type === "before") {
logWarn("⚠️ Model provided premature response BEFORE tool calls!");
logWarn("Sanitizing response to keep only tool calls for first iteration");
} else if (prematureResponseResult.type === "after") {
logWarn("⚠️ Model provided hallucinated response AFTER tool calls!");
logWarn("Truncating response at last tool call for first iteration");
}
}
if (toolCalls.length === 0) {
// No tool calls, this is the final response
// Strip any tool call XML from final response but preserve think blocks
const cleanedResponse = stripToolCallXML(response);
// Build full response from history (which includes tool call markers) and final response
const allParts = [...iterationHistory];
if (cleanedResponse.trim()) {
allParts.push(cleanedResponse);
}
fullAIResponse = allParts.join("\n\n");
// Add final response to LLM messages
this.llmFormattedMessages.push(response);
break;
}
// Use model adapter to sanitize response if needed
let sanitizedResponse = response;
if (adapter.sanitizeResponse && prematureResponseResult?.hasPremature) {
sanitizedResponse = adapter.sanitizeResponse(response, iteration);
}
// Store this iteration's response (AI reasoning) with tool indicators and think blocks preserved
const responseForHistory: string = stripToolCallXML(sanitizedResponse);
// Only add to history if there's meaningful content
if (responseForHistory.trim()) {
iterationHistory.push(responseForHistory);
}
// Execute tool calls and show progress
const toolResults: ToolExecutionResult[] = [];
const toolCallIdMap = new Map<number, string>(); // Map index to tool call ID
for (let i = 0; i < toolCalls.length; i++) {
const toolCall = toolCalls[i];
if (abortController.signal.aborted) break;
// Log tool call details for debugging
logToolCall(toolCall, iteration);
// Find the tool to check if it's a background tool
const availableTools = this.getAvailableTools();
const tool = availableTools.find((t) => t.name === toolCall.name);
const isBackgroundTool = tool?.isBackground || false;
let toolCallId: string | undefined;
// Only show tool calling message for non-background tools
if (!isBackgroundTool) {
// Create tool calling message with structured marker
const toolEmoji = getToolEmoji(toolCall.name);
const toolDisplayName = getToolDisplayName(toolCall.name);
const confirmationMessage = getToolConfirmtionMessage(toolCall.name);
// Generate unique ID for this tool call
toolCallId = `${toolCall.name}-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
toolCallIdMap.set(i, toolCallId);
// Create structured tool call marker
const toolCallMarker = createToolCallMarker(
toolCallId,
toolCall.name,
toolDisplayName,
toolEmoji,
confirmationMessage || "",
true, // isExecuting
"", // content (empty for now)
"" // result (empty until execution completes)
);
currentIterationToolCallMessages.push(toolCallMarker);
// Show all history plus all tool call messages
const currentDisplay = [...iterationHistory, ...currentIterationToolCallMessages].join(
"\n\n"
);
updateCurrentAiMessage(currentDisplay);
}
const result = await executeSequentialToolCall(
toolCall,
availableTools,
originalUserPrompt
);
toolResults.push(result);
// Update the tool call marker with the result if we have an ID
if (toolCallId && !isBackgroundTool) {
// Update the specific tool call message
const messageIndex = currentIterationToolCallMessages.findIndex((msg) =>
msg.includes(toolCallId)
);
if (messageIndex !== -1) {
currentIterationToolCallMessages[messageIndex] = updateToolCallMarker(
currentIterationToolCallMessages[messageIndex],
toolCallId,
result.result
);
}
// Update the display with the result
const currentDisplay = [...iterationHistory, ...currentIterationToolCallMessages].join(
"\n\n"
);
updateCurrentAiMessage(currentDisplay);
}
// Log tool result
logToolResult(toolCall.name, result);
// Collect sources from localSearch results
if (toolCall.name === "localSearch" && result.success) {
try {
const searchResults = JSON.parse(result.result);
if (Array.isArray(searchResults)) {
const sources = searchResults.map((doc: any) => ({
title: doc.title || doc.path,
path: doc.path || doc.title || "",
score: doc.rerank_score || doc.score || 0,
}));
collectedSources.push(...sources);
}
} catch (e) {
logWarn("Failed to parse localSearch results for sources:", e);
}
}
}
// Add all tool call messages to history so they persist
if (currentIterationToolCallMessages.length > 0) {
const toolCallsString = currentIterationToolCallMessages.join("\n");
iterationHistory.push(toolCallsString);
}
// Don't add tool results to display - they're internal only
// Track LLM-formatted messages for memory
// Add the assistant's response with tool calls
this.llmFormattedMessages.push(response);
// Add tool results in LLM format (truncated for memory)
if (toolResults.length > 0) {
const toolResultsForLLM = processToolResults(toolResults, true); // truncated for memory
if (toolResultsForLLM) {
this.llmFormattedMessages.push(toolResultsForLLM);
}
}
// Add AI response to conversation for next iteration
conversationMessages.push({
role: "assistant",
content: response,
});
// Add tool results as user messages for next iteration (full results for current turn)
const toolResultsForConversation = processToolResults(toolResults, false); // full results
conversationMessages.push({
role: "user",
content: toolResultsForConversation,
});
logInfo("Tool results added to conversation:", toolResultsForConversation);
}
// If we hit max iterations, add a message explaining the limit was reached
if (iteration >= maxIterations && !fullAIResponse) {
logWarn(
`Autonomous agent reached maximum iterations (${maxIterations}) without completing the task`
);
const limitMessage =
`\n\nI've reached the maximum number of iterations (${maxIterations}) for this task. ` +
"I attempted to gather information using various tools but couldn't complete the analysis within the iteration limit. " +
"You may want to try a more specific question or break down your request into smaller parts.";
fullAIResponse = iterationHistory.join("\n\n") + limitMessage;
}
} catch (error: any) {
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("Autonomous agent stream aborted by user", {
reason: abortController.signal.reason,
});
} else {
logError("Autonomous agent failed, falling back to regular Plus mode:", error);
// Fallback to regular CopilotPlusChainRunner
try {
const fallbackRunner = new CopilotPlusChainRunner(this.chainManager);
return await fallbackRunner.run(
userMessage,
abortController,
updateCurrentAiMessage,
addMessage,
options
);
} catch (fallbackError) {
logError("Fallback to regular Plus mode also failed:", fallbackError);
await this.handleError(fallbackError, addMessage, updateCurrentAiMessage);
return "";
}
}
}
// Handle response like the parent class, with sources if we found any
const uniqueSources = deduplicateSources(collectedSources);
// Create LLM-formatted output for memory
const llmFormattedOutput = this.llmFormattedMessages.join("\n\n");
// If we somehow don't have a fullAIResponse but have iteration history, use that
if (!fullAIResponse && iterationHistory.length > 0) {
logWarn("fullAIResponse was empty, using iteration history");
fullAIResponse = iterationHistory.join("\n\n");
}
return this.handleResponse(
fullAIResponse,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage,
uniqueSources.length > 0 ? uniqueSources : undefined,
llmFormattedOutput
);
}
private async streamResponse(
messages: any[],
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
adapter: ModelAdapter
): Promise<string> {
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage, adapter);
const maxRetries = 2;
let retryCount = 0;
while (retryCount <= maxRetries) {
try {
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
break;
}
streamer.processChunk(chunk);
}
return streamer.close();
} catch (error) {
if (error.name === "AbortError" || abortController.signal.aborted) {
return streamer.close();
}
// Check if this is an overloaded error that we should retry
const isOverloadedError =
error?.message?.includes("overloaded") ||
error?.message?.includes("Overloaded") ||
error?.error?.type === "overloaded_error";
if (isOverloadedError && retryCount < maxRetries) {
retryCount++;
logInfo(
`Retrying autonomous agent request (attempt ${retryCount}/${maxRetries + 1}) due to overloaded error`
);
// Wait before retrying (exponential backoff)
await new Promise((resolve) => setTimeout(resolve, 1000 * retryCount));
continue;
}
throw error;
}
}
// This should never be reached, but just in case
return streamer.close();
}
}

View file

@ -0,0 +1,142 @@
import { ABORT_REASON, AI_SENDER } from "@/constants";
import { logError, logInfo } from "@/logger";
import { ChatMessage } from "@/types/message";
import { err2String, formatDateTime } from "@/utils";
import { Notice } from "obsidian";
import ChainManager from "../chainManager";
export interface ChainRunner {
run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string>;
}
export abstract class BaseChainRunner implements ChainRunner {
protected chainManager: ChainManager;
constructor(chainManager: ChainManager) {
this.chainManager = chainManager;
}
abstract run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string>;
protected async handleResponse(
fullAIResponse: string,
userMessage: ChatMessage,
abortController: AbortController,
addMessage: (message: ChatMessage) => void,
updateCurrentAiMessage: (message: string) => void,
sources?: { title: string; path: string; score: number }[],
llmFormattedOutput?: string
) {
// Save to memory and add message if we have a response
// Skip only if it's a NEW_CHAT abort (clearing everything)
if (
fullAIResponse &&
!(abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT)
) {
// Use saveContext for atomic operation and proper memory management
// Note: LangChain's memory expects text content, not multimodal arrays, so multimodal content is not saved
await this.chainManager.memoryManager
.getMemory()
.saveContext(
{ input: userMessage.message },
{ output: llmFormattedOutput || fullAIResponse }
);
addMessage({
message: fullAIResponse,
sender: AI_SENDER,
isVisible: true,
timestamp: formatDateTime(new Date()),
sources: sources,
});
// Clear the streaming message since it's now in chat history
updateCurrentAiMessage("");
} else if (abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
// Also clear if it's a new chat
updateCurrentAiMessage("");
}
logInfo(
"==== Chat Memory ====\n",
(this.chainManager.memoryManager.getMemory().chatHistory as any).messages.map(
(m: any) => m.content
)
);
logInfo("==== Final AI Response ====\n", fullAIResponse);
return fullAIResponse;
}
protected async handleError(
error: any,
addMessage?: (message: ChatMessage) => void,
updateCurrentAiMessage?: (message: string) => void
) {
const msg = err2String(error);
logError("Error during LLM invocation:", msg);
const errorData = error?.response?.data?.error || msg;
const errorCode = errorData?.code || msg;
let errorMessage = "";
// Check for specific error messages
if (error?.message?.includes("Invalid license key")) {
errorMessage = "Invalid Copilot Plus license key. Please check your license key in settings.";
} else if (errorCode === "model_not_found") {
errorMessage =
"You do not have access to this model or the model does not exist, please check with your API provider.";
} else {
errorMessage = `${errorCode}`;
}
logError(errorData);
if (addMessage && updateCurrentAiMessage) {
updateCurrentAiMessage("");
// remove langchain troubleshooting URL from error message
const ignoreEndIndex = errorMessage.search("Troubleshooting URL");
errorMessage = ignoreEndIndex !== -1 ? errorMessage.slice(0, ignoreEndIndex) : errorMessage;
// add more user guide for invalid API key
if (msg.search(/401|invalid|not valid/gi) !== -1) {
errorMessage =
"Something went wrong. Please check if you have set your API key." +
"\nPath: Settings > copilot plugin > Basic Tab > Set Keys." +
"\nOr check model config" +
"\nError Details: " +
errorMessage;
}
addMessage({
message: errorMessage,
isErrorMessage: true,
sender: AI_SENDER,
isVisible: true,
timestamp: formatDateTime(new Date()),
});
} else {
// Fallback to Notice if message handlers aren't provided
new Notice(errorMessage);
logError(errorData);
}
}
}

View file

@ -0,0 +1,683 @@
import { getStandaloneQuestion } from "@/chainUtils";
import {
ABORT_REASON,
COMPOSER_OUTPUT_INSTRUCTIONS,
LOADING_MESSAGES,
MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT,
ModelCapability,
} from "@/constants";
import {
ImageBatchProcessor,
ImageContent,
ImageProcessingResult,
MessageContent,
} from "@/imageProcessing/imageProcessor";
import { BrevilabsClient } from "@/LLMProviders/brevilabsClient";
import { logError, logInfo, logWarn } from "@/logger";
import { getSettings, getSystemPrompt } from "@/settings/model";
import { ToolManager } from "@/tools/toolManager";
import { writeToFileTool } from "@/tools/ComposerTools";
import { ChatMessage } from "@/types/message";
import {
extractYoutubeUrl,
getApiErrorMessage,
getMessageRole,
withSuppressedTokenWarnings,
} from "@/utils";
import { BaseChatModel } from "@langchain/core/language_models/chat_models";
import { COPILOT_TOOL_NAMES, IntentAnalyzer } from "../intentAnalyzer";
import { BaseChainRunner } from "./BaseChainRunner";
import { ActionBlockStreamer } from "./utils/ActionBlockStreamer";
import { ThinkBlockStreamer } from "./utils/ThinkBlockStreamer";
import {
addChatHistoryToMessages,
processRawChatHistory,
processedMessagesToTextOnly,
} from "./utils/chatHistoryUtils";
export class CopilotPlusChainRunner extends BaseChainRunner {
private isYoutubeOnlyMessage(message: string): boolean {
const trimmedMessage = message.trim();
const hasYoutubeCommand = trimmedMessage.includes("@youtube");
const youtubeUrl = extractYoutubeUrl(trimmedMessage);
// Check if message only contains @youtube command and a valid URL
const words = trimmedMessage
.split(/\s+/)
.filter((word) => word !== "@youtube" && word.length > 0);
return hasYoutubeCommand && youtubeUrl !== null && words.length === 1;
}
private async processImageUrls(urls: string[]): Promise<ImageProcessingResult> {
const failedImages: string[] = [];
const processedImages = await ImageBatchProcessor.processUrlBatch(
urls,
failedImages,
this.chainManager.app.vault
);
ImageBatchProcessor.showFailedImagesNotice(failedImages);
return processedImages;
}
private async processChatInputImages(content: MessageContent[]): Promise<ImageProcessingResult> {
const failedImages: string[] = [];
const processedImages = await ImageBatchProcessor.processChatImageBatch(
content,
failedImages,
this.chainManager.app.vault
);
ImageBatchProcessor.showFailedImagesNotice(failedImages);
return processedImages;
}
private async extractEmbeddedImages(content: string, sourcePath?: string): Promise<string[]> {
// Match both wiki-style ![[image.ext]] and standard markdown ![alt](image.ext)
const wikiImageRegex = /!\[\[(.*?\.(png|jpg|jpeg|gif|webp|bmp|svg))\]\]/g;
// Updated regex to handle URLs with or without file extensions
const markdownImageRegex = /!\[.*?\]\(([^)]+)\)/g;
const resolvedImages: string[] = [];
// Process wiki-style images
const wikiMatches = [...content.matchAll(wikiImageRegex)];
for (const match of wikiMatches) {
const imageName = match[1];
// If we have a source path and access to the app, resolve the wikilink
if (sourcePath) {
const resolvedFile = app.metadataCache.getFirstLinkpathDest(imageName, sourcePath);
if (resolvedFile) {
// Use the resolved path
resolvedImages.push(resolvedFile.path);
} else {
// If file not found, log a warning but still include the raw filename
logWarn(`Could not resolve embedded image: ${imageName} from source: ${sourcePath}`);
resolvedImages.push(imageName);
}
} else {
// Fallback to raw filename if no source path available
resolvedImages.push(imageName);
}
}
// Process standard markdown images
const mdMatches = [...content.matchAll(markdownImageRegex)];
for (const match of mdMatches) {
const imagePath = match[1].trim();
// Skip empty paths
if (!imagePath) continue;
// Handle external URLs (http://, https://, etc.)
if (imagePath.match(/^https?:\/\//)) {
// Include external URLs - they will be processed by processImageUrls
// The ImageProcessor will validate if it's actually an image
resolvedImages.push(imagePath);
continue;
}
// For local paths, resolve them using Obsidian's metadata cache
// Let ImageBatchProcessor handle validation of whether it's actually an image
// Clean up the path (remove any leading ./ or /)
const cleanPath = imagePath.replace(/^\.\//, "").replace(/^\//, "");
// If we have a source path and access to the app, resolve the path
if (sourcePath) {
const resolvedFile = app.metadataCache.getFirstLinkpathDest(cleanPath, sourcePath);
if (resolvedFile) {
// Use the resolved path
resolvedImages.push(resolvedFile.path);
} else {
// If file not found, still include the raw path
// Let ImageBatchProcessor handle validation
resolvedImages.push(cleanPath);
}
} else {
// Fallback to raw path if no source path available
resolvedImages.push(cleanPath);
}
}
return resolvedImages;
}
protected async buildMessageContent(
textContent: string,
userMessage: ChatMessage
): Promise<MessageContent[]> {
const failureMessages: string[] = [];
const successfulImages: ImageContent[] = [];
const settings = getSettings();
// Collect all image sources
const imageSources: { urls: string[]; type: string }[] = [];
// Safely check and add context URLs
const contextUrls = userMessage.context?.urls;
if (contextUrls && contextUrls.length > 0) {
imageSources.push({ urls: contextUrls, type: "context" });
}
// Process embedded images only if setting is enabled
if (settings.passMarkdownImages) {
// Determine source path for resolving wikilinks
let sourcePath: string | undefined;
// First, check if we have context notes
if (userMessage.context?.notes && userMessage.context.notes.length > 0) {
// Use the first note in context as the source path
sourcePath = userMessage.context.notes[0].path;
} else {
// Fallback to active file if no context notes
const activeFile = this.chainManager.app?.workspace.getActiveFile();
if (activeFile) {
sourcePath = activeFile.path;
}
}
const embeddedImages = await this.extractEmbeddedImages(textContent, sourcePath);
if (embeddedImages.length > 0) {
imageSources.push({ urls: embeddedImages, type: "embedded" });
}
}
// Process all image sources
for (const source of imageSources) {
const result = await this.processImageUrls(source.urls);
successfulImages.push(...result.successfulImages);
failureMessages.push(...result.failureDescriptions);
}
// Process existing chat content images if present
const existingContent = userMessage.content;
if (existingContent && existingContent.length > 0) {
const result = await this.processChatInputImages(existingContent);
successfulImages.push(...result.successfulImages);
failureMessages.push(...result.failureDescriptions);
}
// Let the LLM know about the image processing failures
let finalText = textContent;
if (failureMessages.length > 0) {
finalText = `${textContent}\n\nNote: \n${failureMessages.join("\n")}\n`;
}
const messageContent: MessageContent[] = [
{
type: "text",
text: finalText,
},
];
// Add successful images after the text content
if (successfulImages.length > 0) {
messageContent.push(...successfulImages);
}
return messageContent;
}
private hasCapability(model: BaseChatModel, capability: ModelCapability): boolean {
const modelName = (model as any).modelName || (model as any).model || "";
const customModel = this.chainManager.chatModelManager.findModelByName(modelName);
return customModel?.capabilities?.includes(capability) ?? false;
}
protected isMultimodalModel(model: BaseChatModel): boolean {
return this.hasCapability(model, ModelCapability.VISION);
}
/**
* If userMessage.message contains '@composer', append COMPOSER_OUTPUT_INSTRUCTIONS to the text content.
* Handles both string and MessageContent[] types.
*/
private appendComposerInstructionsIfNeeded(content: string, userMessage: ChatMessage): string {
if (!userMessage.message || !userMessage.message.includes("@composer")) {
return content;
}
const composerPrompt = `<OUTPUT_FORMAT>\n${COMPOSER_OUTPUT_INSTRUCTIONS}\n</OUTPUT_FORMAT>`;
return `${content}\n\n${composerPrompt}`;
}
private async streamMultimodalResponse(
textContent: string,
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void
): Promise<string> {
// Get chat history
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
// Use the raw history array which contains BaseMessage objects
const rawHistory = memoryVariables.history || [];
// Create messages array starting with system message
const messages: any[] = [];
// Add system message if available
let fullSystemMessage = await this.getSystemPrompt();
// Add chat history context to system message if exists
if (rawHistory.length > 0) {
fullSystemMessage +=
"\n\nThe following is the relevant conversation history. Use this context to maintain consistency in your responses:";
}
// Get chat model for role determination for O-series models
const chatModel = this.chainManager.chatModelManager.getChatModel();
// Add the combined system message with appropriate role
if (fullSystemMessage) {
messages.push({
role: getMessageRole(chatModel),
content: `${fullSystemMessage}\nIMPORTANT: Maintain consistency with previous responses in the conversation. If you've provided information about a person or topic before, use that same information in follow-up questions.`,
});
}
// Add chat history - safely handle different message formats
addChatHistoryToMessages(rawHistory, messages);
// Get the current chat model
const chatModelCurrent = this.chainManager.chatModelManager.getChatModel();
const isMultimodalCurrent = this.isMultimodalModel(chatModelCurrent);
// Build message content with text and images for multimodal models, or just text for text-only models
const content: string | MessageContent[] = isMultimodalCurrent
? await this.buildMessageContent(textContent, userMessage)
: textContent;
// Add current user message
messages.push({
role: "user",
content,
});
const enhancedUserMessage = content instanceof Array ? (content[0] as any).text : content;
logInfo("Enhanced user message: ", enhancedUserMessage);
logInfo("==== Final Request to AI ====\n", messages);
const actionStreamer = new ActionBlockStreamer(ToolManager, writeToFileTool);
const thinkStreamer = new ThinkBlockStreamer(updateCurrentAiMessage);
// Wrap the stream call with warning suppression
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("CopilotPlus multimodal stream iteration aborted", {
reason: abortController.signal.reason,
});
break;
}
for await (const processedChunk of actionStreamer.processChunk(chunk)) {
thinkStreamer.processChunk(processedChunk);
}
}
return thinkStreamer.close();
}
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
updateLoadingMessage?: (message: string) => void;
}
): Promise<string> {
const { updateLoadingMessage } = options;
let fullAIResponse = "";
let sources: { title: string; path: string; score: number }[] = [];
let currentPartialResponse = "";
// Wrapper to track partial response
const trackAndUpdateAiMessage = (message: string) => {
currentPartialResponse = message;
updateCurrentAiMessage(message);
};
try {
// Check if this is a YouTube-only message
if (this.isYoutubeOnlyMessage(userMessage.message)) {
const url = extractYoutubeUrl(userMessage.message);
const failMessage =
"Transcript not available. Only videos with the auto transcript option turned on are supported at the moment.";
if (url) {
try {
const response = await BrevilabsClient.getInstance().youtube4llm(url);
if (response.response.transcript) {
return this.handleResponse(
response.response.transcript,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
return this.handleResponse(
failMessage,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
} catch (error) {
logError("Error processing YouTube video:", error);
return this.handleResponse(
failMessage,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
}
}
logInfo("==== Step 1: Analyzing intent ====");
let toolCalls;
// Use the original message for intent analysis
const messageForAnalysis = userMessage.originalMessage || userMessage.message;
try {
toolCalls = await IntentAnalyzer.analyzeIntent(messageForAnalysis);
} catch (error: any) {
return this.handleResponse(
getApiErrorMessage(error),
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
// Use the same removeAtCommands logic as IntentAnalyzer
const cleanedUserMessage = userMessage.message
.split(" ")
.filter((word) => !COPILOT_TOOL_NAMES.includes(word.toLowerCase()))
.join(" ")
.trim();
const toolOutputs = await this.executeToolCalls(toolCalls, updateLoadingMessage);
// Extract sources from localSearch if present
const localSearchResult = toolOutputs.find(
(output) => output.tool === "localSearch" && output.output != null
);
let hasLocalSearchWithResults = false;
if (localSearchResult) {
try {
const documents = JSON.parse(localSearchResult.output);
if (Array.isArray(documents) && documents.length > 0) {
hasLocalSearchWithResults = true;
sources = this.getSources(documents);
}
} catch (error) {
logWarn("Failed to parse localSearch results for sources:", error);
}
}
// Format chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const rawHistory = memoryVariables.history || [];
// Process history consistently - same data used for both LLM and question condensing
const processedHistory = processRawChatHistory(rawHistory);
const chatHistory = processedMessagesToTextOnly(processedHistory);
// Get standalone question if we have chat history
let questionForEnhancement = cleanedUserMessage;
if (chatHistory.length > 0) {
logInfo("==== Condensing Question ====");
questionForEnhancement = await getStandaloneQuestion(cleanedUserMessage, chatHistory);
logInfo("Condensed standalone question: ", questionForEnhancement);
}
// Enhance with ALL tool outputs including localSearch
let enhancedUserMessage = this.prepareEnhancedUserMessage(
questionForEnhancement,
toolOutputs,
toolCalls
);
// If localSearch has actual results and no other tools, add QA-style instruction to maintain same behavior
const hasOtherTools = toolOutputs.some(
(output) => output.tool !== "localSearch" && output.output != null
);
if (hasLocalSearchWithResults && !hasOtherTools) {
// The QA format is already handled in prepareEnhancedUserMessage, just add the instruction
enhancedUserMessage = `Answer the question with as detailed as possible based only on the following context:\n${enhancedUserMessage}`;
}
// Append composer instruction to the end of text prompt to enhance instruction following.
enhancedUserMessage = this.appendComposerInstructionsIfNeeded(
enhancedUserMessage,
userMessage
);
logInfo("==== Invoking LLM with all tool results ====");
fullAIResponse = await this.streamMultimodalResponse(
enhancedUserMessage,
userMessage,
abortController,
trackAndUpdateAiMessage
);
} catch (error: any) {
// Reset loading message to default
updateLoadingMessage?.(LOADING_MESSAGES.DEFAULT);
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("CopilotPlus stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
// If aborted but not a new chat, use the partial response
if (abortController.signal.aborted && currentPartialResponse) {
fullAIResponse = currentPartialResponse;
}
return this.handleResponse(
fullAIResponse,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage,
sources
);
}
private getSources(documents: any): { title: string; path: string; score: number }[] {
if (!documents || !Array.isArray(documents)) {
logWarn("No valid documents provided to getSources");
return [];
}
return this.sortUniqueDocsByScore(documents);
}
private sortUniqueDocsByScore(documents: any[]): any[] {
const uniqueDocs = new Map<string, any>();
// Iterate through all documents
for (const doc of documents) {
if (!doc.title || (!doc?.score && !doc?.rerank_score)) {
logWarn("Invalid document structure:", doc);
continue;
}
// Use path as the unique key, falling back to title if path is not available
const key = doc.path || doc.title;
const currentDoc = uniqueDocs.get(key);
const isReranked = doc && "rerank_score" in doc;
const docScore = isReranked ? doc.rerank_score : doc.score;
// If the document doesn't exist in the map, or if the new doc has a higher score, update the map
if (!currentDoc || docScore > (currentDoc.score ?? 0)) {
uniqueDocs.set(key, {
title: doc.title,
path: doc.path || doc.title, // Use path if available, otherwise use title
score: docScore,
isReranked: isReranked,
});
}
}
// Convert the map values back to an array and sort by score in descending order
return Array.from(uniqueDocs.values()).sort((a, b) => (b.score ?? 0) - (a.score ?? 0));
}
private async executeToolCalls(
toolCalls: any[],
updateLoadingMessage?: (message: string) => void
) {
const toolOutputs = [];
for (const toolCall of toolCalls) {
logInfo(`==== Step 2: Calling tool: ${toolCall.tool.name} ====`);
if (toolCall.tool.name === "localSearch") {
updateLoadingMessage?.(LOADING_MESSAGES.READING_FILES);
} else if (toolCall.tool.name === "webSearch") {
updateLoadingMessage?.(LOADING_MESSAGES.SEARCHING_WEB);
} else if (toolCall.tool.name === "getFileTree") {
updateLoadingMessage?.(LOADING_MESSAGES.READING_FILE_TREE);
}
const output = await ToolManager.callTool(toolCall.tool, toolCall.args);
toolOutputs.push({ tool: toolCall.tool.name, output });
}
return toolOutputs;
}
private prepareEnhancedUserMessage(userMessage: string, toolOutputs: any[], toolCalls?: any[]) {
let context = "";
let hasLocalSearchWithResults = false;
// Check if localSearch has actual results (non-empty documents array)
const localSearchOutput = toolOutputs.find(
(output) => output.tool === "localSearch" && output.output != null
);
if (localSearchOutput && typeof localSearchOutput.output === "string") {
try {
const documents = JSON.parse(localSearchOutput.output);
if (Array.isArray(documents) && documents.length > 0) {
hasLocalSearchWithResults = true;
}
} catch {
// Invalid JSON or parsing error
}
}
if (toolOutputs.length > 0) {
const validOutputs = toolOutputs.filter((output) => output.output != null);
if (validOutputs.length > 0) {
// Don't add "Additional context" header if only localSearch with results to maintain QA format
const contextHeader =
hasLocalSearchWithResults && validOutputs.length === 1
? ""
: "\n\n# Additional context:\n\n";
context =
contextHeader +
validOutputs
.map((output) => {
let content = output.output;
// Special formatting for localSearch results
if (output.tool === "localSearch" && typeof content === "string") {
try {
const documents = JSON.parse(content);
if (Array.isArray(documents) && documents.length > 0) {
// Get time expression from toolCalls if available
const timeExpression = toolCalls ? this.getTimeExpression(toolCalls) : "";
const formattedContent = this.prepareLocalSearchResult(
documents,
timeExpression
);
content = formattedContent;
}
} catch (error) {
// If parsing fails, use the raw output
logWarn("Failed to parse localSearch output for formatting:", error);
}
}
// Ensure content is string
if (typeof content !== "string") {
content = JSON.stringify(content);
}
// Only wrap in XML tags if there are multiple tools
if (validOutputs.length > 1) {
return `<${output.tool}>\n${content}\n</${output.tool}>`;
} else if (output.tool === "localSearch" && hasLocalSearchWithResults) {
// For localSearch with results only, don't wrap in XML to maintain QA format
return content;
} else {
return `<${output.tool}>\n${content}\n</${output.tool}>`;
}
})
.join("\n\n");
}
}
// For QA format when only localSearch with results is present
if (hasLocalSearchWithResults && toolOutputs.filter((o) => o.output != null).length === 1) {
return `${context}\n\nQuestion: ${userMessage}`;
}
return `${userMessage}${context}`;
}
private getTimeExpression(toolCalls: any[]): string {
const timeRangeCall = toolCalls.find((call) => call.tool.name === "getTimeRangeMs");
return timeRangeCall ? timeRangeCall.args.timeExpression : "";
}
private prepareLocalSearchResult(documents: any[], timeExpression: string): string {
// First filter documents with includeInContext
const includedDocs = documents.filter((doc) => doc.includeInContext);
// Calculate total content length
const totalLength = includedDocs.reduce((sum, doc) => sum + doc.content.length, 0);
// If total length exceeds threshold, calculate truncation ratio
let truncatedDocs = includedDocs;
if (totalLength > MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT) {
const truncationRatio = MAX_CHARS_FOR_LOCAL_SEARCH_CONTEXT / totalLength;
logInfo("Truncating documents to fit context length. Truncation ratio:", truncationRatio);
truncatedDocs = includedDocs.map((doc) => ({
...doc,
content: doc.content.slice(0, Math.floor(doc.content.length * truncationRatio)),
}));
}
const formattedDocs = truncatedDocs
.map((doc: any) => `Note in Vault: ${doc.content}`)
.join("\n\n");
return timeExpression
? `Local Search Result for ${timeExpression}:\n${formattedDocs}`
: `Local Search Result:\n${formattedDocs}`;
}
protected async getSystemPrompt(): Promise<string> {
return getSystemPrompt();
}
}

View file

@ -0,0 +1,97 @@
import { ABORT_REASON } from "@/constants";
import { logInfo } from "@/logger";
import { getSystemPrompt } from "@/settings/model";
import { ChatMessage } from "@/types/message";
import { extractChatHistory, getMessageRole, withSuppressedTokenWarnings } from "@/utils";
import { BaseChainRunner } from "./BaseChainRunner";
import { ThinkBlockStreamer } from "./utils/ThinkBlockStreamer";
export class LLMChainRunner extends BaseChainRunner {
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string> {
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage);
try {
// Get chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
// Create messages array starting with system message
const messages: any[] = [];
// Add system message if available
const systemPrompt = getSystemPrompt();
const chatModel = this.chainManager.chatModelManager.getChatModel();
if (systemPrompt) {
messages.push({
role: getMessageRole(chatModel),
content: systemPrompt,
});
}
// Add chat history
for (const entry of chatHistory) {
messages.push({ role: entry.role, content: entry.content });
}
// Add current user message
messages.push({
role: "user",
content: userMessage.message,
});
logInfo("==== Final Request to AI ====\n", messages);
// Stream with abort signal
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("Stream iteration aborted", { reason: abortController.signal.reason });
break;
}
streamer.processChunk(chunk);
}
} catch (error: any) {
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("Stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Always return the response, even if partial
const response = streamer.close();
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
return this.handleResponse(
response,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
}

View file

@ -0,0 +1,25 @@
import { getCurrentProject } from "@/aiParams";
import { getSystemPrompt } from "@/settings/model";
import ProjectManager from "../projectManager";
import { CopilotPlusChainRunner } from "./CopilotPlusChainRunner";
export class ProjectChainRunner extends CopilotPlusChainRunner {
protected async getSystemPrompt(): Promise<string> {
let finalPrompt = getSystemPrompt();
const projectConfig = getCurrentProject();
if (!projectConfig) {
return finalPrompt;
}
// Get context asynchronously
const context = await ProjectManager.instance.getProjectContext(projectConfig.id);
finalPrompt = `${finalPrompt}\n\n<project_system_prompt>\n${projectConfig.systemPrompt}\n</project_system_prompt>`;
// TODO: Move project context out of the system prompt and into the user prompt.
if (context) {
finalPrompt = `${finalPrompt}\n\n <project_context>\n${context}\n</project_context>`;
}
return finalPrompt;
}
}

View file

@ -0,0 +1,640 @@
# Chain Runner Architecture & Tool Calling System
This directory contains the refactored chain runner system for Obsidian Copilot, providing multiple chain execution strategies with different tool calling approaches.
## Overview
The chain runner system provides two distinct tool calling approaches:
1. **Legacy Tool Calling** (CopilotPlusChainRunner) - Uses Brevilabs API for intent analysis
2. **Autonomous Agent** (AutonomousAgentChainRunner) - Uses XML-based tool calling
## Architecture
```
chainRunner/
├── BaseChainRunner.ts # Abstract base class with shared functionality
├── LLMChainRunner.ts # Basic LLM interaction (no tools)
├── VaultQAChainRunner.ts # Vault-only Q&A with retrieval
├── CopilotPlusChainRunner.ts # Legacy tool calling system
├── ProjectChainRunner.ts # Project-aware extension of Plus
├── AutonomousAgentChainRunner.ts # XML-based autonomous agent tool calling
├── index.ts # Main exports
└── utils/
├── ThinkBlockStreamer.ts # Handles thinking content from models
├── xmlParsing.ts # XML tool call parsing utilities
├── toolExecution.ts # Tool execution helpers
└── modelAdapter.ts # Model-specific adaptations
```
## Tool Calling Systems Comparison
### 1. Legacy Tool Calling (CopilotPlusChainRunner)
**How it works:**
- Uses Brevilabs API (`IntentAnalyzer.analyzeIntent()`) to analyze user intent
- Determines which tools to call based on the analysis
- Executes tools synchronously before sending to LLM
- Enhances user message with tool outputs as context
**Flow:**
```
User Message → Intent Analysis → Tool Execution → Enhanced Prompt → LLM Response
```
**Example:**
```typescript
// 1. Analyze intent
const toolCalls = await IntentAnalyzer.analyzeIntent(message);
// 2. Execute tools
const toolOutputs = await this.executeToolCalls(toolCalls);
// 3. Enhance message with context
const enhancedMessage = this.prepareEnhancedUserMessage(message, toolOutputs);
// 4. Send to LLM
const response = await this.streamMultimodalResponse(enhancedMessage, ...);
```
**Tools Available:**
- `localSearch` - Search vault content
- `webSearch` - Search the web
- `getCurrentTime` - Get current time
- `getFileTree` - Get file structure
- `pomodoroTool` - Pomodoro timer
- `youtubeTranscription` - YouTube video transcription
### 2. Autonomous Agent (AutonomousAgentChainRunner)
**How it works:**
- **No LangChain dependency** - Uses simple tool interface with XML-based tool calling
- AI decides autonomously which tools to use via structured XML format
- Iterative loop where AI can call multiple tools in sequence
- Each tool result informs the next decision
**Flow:**
```
User Message → AI Reasoning → XML Tool Call → Tool Execution →
AI Analysis → More Tools? → Final Response
```
**XML Tool Call Format:**
```xml
<use_tool>
<name>localSearch</name>
<args>
{
"query": "machine learning notes",
"salientTerms": ["machine", "learning", "AI", "algorithms"]
}
</args>
</use_tool>
```
**Sequential Loop:**
```typescript
while (iteration < maxIterations) {
// 1. Get AI response
const response = await this.streamResponse(messages);
// 2. Parse XML tool calls
const toolCalls = parseXMLToolCalls(response);
if (toolCalls.length === 0) {
// No tools needed - final response
break;
}
// 3. Execute each tool
for (const toolCall of toolCalls) {
const result = await executeSequentialToolCall(toolCall, availableTools);
toolResults.push(result);
}
// 4. Add results to conversation for next iteration
messages.push({ role: "user", content: toolResultsForConversation });
}
```
## Key Differences
| Aspect | Legacy (Plus) | Autonomous Agent |
| ------------------ | ----------------------- | ------------------------------------- |
| **Tool Decision** | Brevilabs API analysis | AI decides autonomously |
| **Tool Execution** | Pre-LLM, synchronous | During conversation, iterative |
| **Tool Format** | SimpleTool interface | XML-based structured format |
| **Reasoning** | Intent analysis → tools | AI reasoning → tools → more reasoning |
| **Iterations** | Single pass | Up to 4 iterations |
| **Tool Chaining** | Limited | Full chaining support |
## SimpleTool Interface
### Overview
The SimpleTool interface provides a clean, type-safe way to define tools with Zod validation:
```typescript
interface SimpleTool<TSchema extends z.ZodType = z.ZodVoid> {
name: string;
description: string;
schema: TSchema;
call: (args: z.infer<TSchema>) => Promise<any>;
timeoutMs?: number;
isBackground?: boolean;
}
```
### Creating Tools
All tools are created using the unified `createTool` function with Zod schemas:
#### Tool with No Parameters
```typescript
const indexTool = createTool({
name: "indexVault",
description: "Index the vault to the Copilot index",
schema: z.void(), // No parameters
handler: async () => {
// Tool implementation
return "Indexing complete";
},
isBackground: true, // Optional: hide from user
});
```
#### Tool with Parameters
```typescript
// Define schema with validation rules
const searchSchema = z.object({
query: z.string().min(1).describe("The search query"),
salientTerms: z.array(z.string()).min(1).describe("Key terms extracted from query"),
timeRange: z
.object({
startTime: z.any(),
endTime: z.any(),
})
.optional()
.describe("Time range for search"),
});
// Create tool with automatic validation
const searchTool = createTool({
name: "localSearch",
description: "Search for notes based on query and time range",
schema: searchSchema,
handler: async ({ query, salientTerms, timeRange }) => {
// Handler receives fully typed and validated arguments
// TypeScript knows the exact types from the schema
return performSearch(query, salientTerms, timeRange);
},
timeoutMs: 30000, // Optional: custom timeout
});
```
### Benefits of Unified Zod Approach
1. **Type Safety**: Full TypeScript type inference from schemas
2. **Runtime Validation**: All inputs validated before reaching handler
3. **Consistent Interface**: One way to create all tools
4. **Better Error Messages**: Zod provides detailed validation errors
5. **No Any Types**: Everything is properly typed
6. **Simpler Codebase**: No need to maintain multiple tool creation methods
### Advanced Zod Patterns
#### Complex Validation
```typescript
const emailToolSchema = z.object({
to: z.string().email().describe("Recipient email"),
subject: z.string().min(1).max(100).describe("Email subject"),
body: z.string().min(1).describe("Email content"),
cc: z.array(z.string().email()).optional().describe("CC recipients"),
});
```
#### Union Types for Actions
```typescript
const actionSchema = z.discriminatedUnion("type", [
z.object({
type: z.literal("search"),
query: z.string().min(1),
}),
z.object({
type: z.literal("create"),
content: z.string().min(1),
tags: z.array(z.string()).default([]),
}),
z.object({
type: z.literal("delete"),
id: z.string().uuid(),
}),
]);
const actionTool = createTool({
name: "performAction",
description: "Perform various actions",
schema: actionSchema,
handler: async (action) => {
// TypeScript knows exactly which type based on discriminator
switch (action.type) {
case "search":
return search(action.query);
case "create":
return create(action.content, action.tags);
case "delete":
return deleteItem(action.id);
}
},
});
```
#### Custom Validation
```typescript
const filePathSchema = z
.string()
.refine((val) => val.endsWith(".md") || val.endsWith(".canvas"), {
message: "File must be .md or .canvas",
})
.refine((val) => !val.includes(".."), { message: "Path traversal not allowed" })
.describe("Path to markdown or canvas file");
```
#### Transformations
```typescript
const dateToolSchema = z.object({
date: z
.string()
.describe("Date in ISO format or natural language")
.transform((str) => new Date(str)),
timezone: z.string().default("UTC").describe("Timezone identifier"),
});
```
### Schema Composition
```typescript
// Base schemas that can be reused
const timeRangeSchema = z
.object({
startTime: z.date(),
endTime: z.date(),
})
.refine((data) => data.endTime > data.startTime, {
message: "End time must be after start time",
});
const paginationSchema = z.object({
page: z.number().int().positive().default(1),
pageSize: z.number().int().positive().max(100).default(20),
});
// Compose into larger schemas
const searchWithPaginationSchema = z.object({
query: z.string().min(1).describe("Search query"),
filters: z.record(z.string()).optional().describe("Additional filters"),
timeRange: timeRangeSchema.optional(),
pagination: paginationSchema,
});
```
### Default Values
```typescript
const configSchema = z.object({
temperature: z.number().min(0).max(2).default(0.7),
maxTokens: z.number().int().positive().default(1000),
model: z.enum(["gpt-4", "gpt-3.5-turbo"]).default("gpt-4"),
});
// Handler receives object with defaults applied
const configTool = createTool({
name: "updateConfig",
schema: configSchema,
handler: async (config) => {
// config.temperature is always defined (0.7 if not provided)
// config.maxTokens is always defined (1000 if not provided)
// config.model is always defined ("gpt-4" if not provided)
return updateConfiguration(config);
},
});
```
### Handling Validation Errors with Retry
When AI-generated parameters fail Zod validation, the tool execution will return a formatted error. The autonomous agent automatically handles this through its iterative loop:
```typescript
// Example tool with strict validation
const searchToolWithValidation = createTool({
name: "searchNotes",
description: "Search notes with specific criteria",
schema: z.object({
query: z.string().min(2, "Query must be at least 2 characters"),
limit: z.number().int().min(1).max(100),
sortBy: z.enum(["relevance", "date", "title"]),
}),
handler: async ({ query, limit, sortBy }) => {
return performSearch(query, limit, sortBy);
},
});
// When the AI provides invalid parameters:
// Input: { query: "a", limit: 200, sortBy: "random" }
//
// The flow:
// 1. Tool execution catches Zod validation error
// 2. Returns: "Tool searchNotes validation failed: query: Query must be at least 2 characters,
// limit: Number must be less than or equal to 100, sortBy: Invalid enum value"
// 3. This error is added to the conversation as a user message
// 4. The AI sees the error in the next iteration and can retry with corrected parameters
// 5. The autonomous agent continues up to 4 iterations, allowing multiple retry attempts
// Example conversation flow:
// Iteration 1: AI calls tool with invalid params → receives error
// Iteration 2: AI understands error and retries with { query: "search term", limit: 50, sortBy: "date" } → success
```
The validation errors are automatically formatted to be clear and actionable, helping the AI self-correct. The autonomous agent's iterative design naturally provides retry capability with the AI learning from each error.
## XML Tool Calling Details
### Tool Call Parsing (`xmlParsing.ts`)
```typescript
// Parse XML tool calls from AI response
function parseXMLToolCalls(text: string): ToolCall[] {
const regex = /<use_tool>([\s\S]*?)<\/use_tool>/g;
// Extracts name and args from XML structure
}
// Strip tool calls from display
function stripToolCallXML(text: string): string {
// Removes XML tool blocks and code blocks for clean display
}
```
### Tool Execution (`toolExecution.ts`)
```typescript
// Execute individual tool with timeout and error handling
async function executeSequentialToolCall(
toolCall: ToolCall,
availableTools: any[]
): Promise<ToolExecutionResult> {
// 30-second timeout per tool
// Error handling and validation
// Result formatting
}
```
### Available Tools in Sequential Mode
All tools from the legacy system plus autonomous decision-making:
- **localSearch** - Vault content search with salient terms
- **webSearch** - Web search with chat history context
- **getFileTree** - File structure exploration
- **getCurrentTime** - Time-based queries
- **pomodoroTool** - Productivity timer
- **indexTool** - Vault indexing operations
- **youtubeTranscription** - Video content analysis
### System Prompt Engineering
The Autonomous Agent mode uses a comprehensive system prompt that:
1. **Explains the XML format** with exact examples
2. **Provides tool descriptions** with parameter details
3. **Sets expectations** for reasoning and tool chaining
4. **Includes critical requirements** (e.g., salientTerms for localSearch)
Example system prompt section:
```
When you need to use a tool, format it EXACTLY like this:
<use_tool>
<name>localSearch</name>
<args>
{
"query": "piano learning",
"salientTerms": ["piano", "learning", "practice", "music"]
}
</args>
</use_tool>
CRITICAL: For localSearch, you MUST always provide both "query" (string) and "salientTerms" (array of strings).
```
## Benefits of Autonomous Agent
1. **Autonomous Tool Selection** - AI decides what tools to use without pre-analysis
2. **Tool Chaining** - Can use results from one tool to inform the next
3. **Complex Workflows** - Multi-step reasoning with tool support
4. **Model Agnostic** - Works with any LLM that can follow XML format
5. **No External Dependencies** - No Brevilabs API required
6. **Transparency** - User can see the AI's reasoning process
## Usage
### Enable Autonomous Agent
```typescript
// In settings
settings.enableAutonomousAgent = true;
// ChainManager automatically selects the appropriate runner
const runner = chainManager.getChainRunner(); // Returns AutonomousAgentChainRunner
```
### Example Query Flow
**User Input:** "Find my notes about machine learning and research current best practices"
**Autonomous Agent Process:**
1. **Iteration 1**: AI reasons about the task → calls `localSearch` for ML notes
2. **Iteration 2**: Analyzes vault results → calls `webSearch` for current practices
3. **Iteration 3**: Synthesizes both sources → provides comprehensive response
**Legacy Process:**
1. Intent analysis determines both tools needed
2. Executes both tools
3. Single LLM call with all context
## Error Handling & Fallbacks
### Autonomous Agent Fallbacks
```typescript
try {
// Sequential thinking execution
} catch (error) {
// Automatic fallback to CopilotPlusChainRunner
const fallbackRunner = new CopilotPlusChainRunner(this.chainManager);
return await fallbackRunner.run(/* same parameters */);
}
```
### Tool Execution Safeguards
- 30-second timeout per tool
- Graceful error handling with descriptive messages
- Tool availability validation
- Result validation and formatting
## Model Adapter Pattern
### Overview
The Model Adapter pattern handles model-specific quirks and requirements cleanly, keeping the core logic model-agnostic.
### Architecture
```typescript
interface ModelAdapter {
enhanceSystemPrompt(basePrompt: string, toolDescriptions: string): string;
enhanceUserMessage(message: string, requiresTools: boolean): string;
parseToolCalls?(response: string): any[]; // Future extension
needsSpecialHandling(): boolean;
sanitizeResponse?(response: string, iteration: number): string;
shouldTruncateStreaming?(partialResponse: string): boolean;
detectPrematureResponse?(response: string): {
hasPremature: boolean;
type: "before" | "after" | null;
};
}
```
### Current Adapters
1. **BaseModelAdapter** - Default behavior for well-behaved models
2. **GPTModelAdapter** - Aggressive prompting for GPT models that often skip tool calls
3. **ClaudeModelAdapter** - Specialized handling for Claude thinking models (3.7 Sonnet, Claude 4)
4. **GeminiModelAdapter** - Ready for Gemini-specific handling
### Adding a New Model
```typescript
class NewModelAdapter extends BaseModelAdapter {
enhanceSystemPrompt(basePrompt: string, toolDescriptions: string): string {
const base = super.enhanceSystemPrompt(basePrompt, toolDescriptions);
return base + "\n\n[Model-specific instructions here]";
}
enhanceUserMessage(message: string, requiresTools: boolean): string {
// Add model-specific hints if needed
return requiresTools ? `${message}\n[Model-specific hint]` : message;
}
}
// Register in ModelAdapterFactory
if (modelName.includes("newmodel")) {
return new NewModelAdapter(modelName);
}
```
### Claude Model Adapter Features
The `ClaudeModelAdapter` includes specialized handling for Claude thinking models:
#### Thinking Model Support
- **Claude 3.7 Sonnet** and **Claude 4** - Automatic thinking mode configuration
- **Think Block Preservation** - Maintains valuable reasoning context in responses
- **Temperature Control** - Disables temperature for thinking models (as required by API)
#### Claude 4 Hallucination Prevention
Claude 4 has a tendency to write complete responses immediately after tool calls instead of waiting for results. The adapter addresses this with:
```typescript
// Enhanced prompting with explicit autonomous agent pattern
enhanceSystemPrompt(basePrompt: string, toolDescriptions: string): string {
if (this.isClaudeSonnet4()) {
// Add specific instructions for Claude 4:
// - Brief sentence + tool calls + STOP pattern
// - Explicit warnings about premature responses
// - Clear autonomous agent iteration guidance
}
}
// Detection of premature responses
detectPrematureResponse(response: string): {
hasPremature: boolean;
type: "before" | "after" | null;
} {
// Allows brief sentences before tool calls (up to 2 sentences, 200 chars)
// Detects substantial content after tool calls (forbidden)
// Uses threshold-based detection for generalizability
}
// Response sanitization
sanitizeResponse(response: string, iteration: number): string {
// Preserves ALL think blocks
// Removes substantial non-thinking content after tool calls
// Only applies to first iteration when hallucination occurs
}
// Streaming truncation
shouldTruncateStreaming(partialResponse: string): boolean {
// Prevents streaming of hallucinated content to users
// Truncates at last complete tool call when threshold exceeded
}
```
#### Flow Improvement
The adapter creates a better conversational flow by allowing brief explanatory sentences before tool calls:
```
[Think block]
I'll search your vault and web for piano practice information.
🔍 Calling vault search...
[Think block]
Let me gather more specific information about practice routines.
🌐 Calling web search...
[Think block]
[final answer]
```
### Benefits
1. **Separation of Concerns** - Model quirks isolated from core logic
2. **Maintainability** - Easy to find and update model-specific code
3. **Extensibility** - Simple to add support for new models
4. **Testing** - Model adapters can be unit tested independently
5. **Clean Core** - Autonomous agent logic remains model-agnostic
6. **Hallucination Prevention** - Specialized handling for problematic models
7. **Streaming Protection** - Prevents bad content from reaching users
8. **Generalizable Solutions** - Uses threshold-based detection over regex patterns
## Future Considerations
1. **Tool Discovery** - Dynamic tool registration
2. **Custom Tools** - User-defined tool capabilities
3. **Parallel Execution** - Multiple tools simultaneously
4. **Tool Result Caching** - Avoid redundant calls
5. **Advanced Reasoning** - More sophisticated decision trees
6. **Tool Permissions** - User control over tool access
7. **Alternative Parsing** - Model adapters could handle non-XML formats
8. **Response Validation** - Adapters could validate model outputs
9. **Model-Specific Optimizations** - Expand adapter capabilities for emerging models
10. **Hallucination Detection** - More sophisticated premature response detection
The autonomous agent approach represents a significant evolution from traditional tool calling, enabling more sophisticated AI reasoning and autonomous task completion.

View file

@ -0,0 +1,161 @@
import { ABORT_REASON, EMPTY_INDEX_ERROR_MESSAGE, RETRIEVED_DOCUMENT_TAG } from "@/constants";
import { logInfo } from "@/logger";
import { HybridRetriever } from "@/search/hybridRetriever";
import { getSettings, getSystemPrompt } from "@/settings/model";
import { ChatMessage } from "@/types/message";
import {
extractChatHistory,
extractUniqueTitlesFromDocs,
getMessageRole,
withSuppressedTokenWarnings,
} from "@/utils";
import { BaseChainRunner } from "./BaseChainRunner";
import { ThinkBlockStreamer } from "./utils/ThinkBlockStreamer";
export class VaultQAChainRunner extends BaseChainRunner {
async run(
userMessage: ChatMessage,
abortController: AbortController,
updateCurrentAiMessage: (message: string) => void,
addMessage: (message: ChatMessage) => void,
options: {
debug?: boolean;
ignoreSystemMessage?: boolean;
updateLoading?: (loading: boolean) => void;
}
): Promise<string> {
const streamer = new ThinkBlockStreamer(updateCurrentAiMessage);
try {
// Add check for empty index
const indexEmpty = await this.chainManager.vectorStoreManager.isIndexEmpty();
if (indexEmpty) {
return this.handleResponse(
EMPTY_INDEX_ERROR_MESSAGE,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
// Get chat history from memory
const memory = this.chainManager.memoryManager.getMemory();
const memoryVariables = await memory.loadMemoryVariables({});
const chatHistory = extractChatHistory(memoryVariables);
// Generate standalone question from user message + chat history
// This is similar to what the conversational retrieval chain does
let standaloneQuestion = userMessage.message;
if (chatHistory.length > 0) {
// For simplicity, we'll use the original question directly
// The original chain would rephrase it, but this approach should work for most cases
standaloneQuestion = userMessage.message;
}
// Create retriever (similar to how it's done in chainManager)
const retriever = new HybridRetriever({
minSimilarityScore: 0.01,
maxK: getSettings().maxSourceChunks,
salientTerms: [],
});
// Retrieve relevant documents
const retrievedDocs = await retriever.getRelevantDocuments(standaloneQuestion);
// Store retrieved documents for sources
this.chainManager.storeRetrieverDocuments(retrievedDocs);
// Format documents as context with XML tags
const context = retrievedDocs
.map(
(doc: any) =>
`<${RETRIEVED_DOCUMENT_TAG}>\n${doc.pageContent}\n</${RETRIEVED_DOCUMENT_TAG}>`
)
.join("\n\n");
// Create messages array
const messages: any[] = [];
// Add system message with QA instruction
const systemPrompt = getSystemPrompt();
const qaInstructions =
"\n\nAnswer the question with as detailed as possible based only on the following context:\n" +
context;
const fullSystemMessage = systemPrompt + qaInstructions;
const chatModel = this.chainManager.chatModelManager.getChatModel();
if (fullSystemMessage) {
messages.push({
role: getMessageRole(chatModel),
content: fullSystemMessage,
});
}
// Add chat history
for (const entry of chatHistory) {
messages.push({ role: entry.role, content: entry.content });
}
// Add current user question
messages.push({
role: "user",
content: userMessage.message,
});
logInfo("==== Final Request to AI ====\n", messages);
// Stream with abort signal
const chatStream = await withSuppressedTokenWarnings(() =>
this.chainManager.chatModelManager.getChatModel().stream(messages, {
signal: abortController.signal,
})
);
for await (const chunk of chatStream) {
if (abortController.signal.aborted) {
logInfo("VaultQA stream iteration aborted", { reason: abortController.signal.reason });
break;
}
streamer.processChunk(chunk);
}
} catch (error: any) {
// Check if the error is due to abort signal
if (error.name === "AbortError" || abortController.signal.aborted) {
logInfo("VaultQA stream aborted by user", { reason: abortController.signal.reason });
// Don't show error message for user-initiated aborts
} else {
await this.handleError(error, addMessage, updateCurrentAiMessage);
}
}
// Always get the response, even if partial
let fullAIResponse = streamer.close();
// Only skip saving if it's a new chat (clearing everything)
if (abortController.signal.aborted && abortController.signal.reason === ABORT_REASON.NEW_CHAT) {
updateCurrentAiMessage("");
return "";
}
// Add sources to the response
fullAIResponse = this.addSourcestoResponse(fullAIResponse);
return this.handleResponse(
fullAIResponse,
userMessage,
abortController,
addMessage,
updateCurrentAiMessage
);
}
private addSourcestoResponse(response: string): string {
const docTitles = extractUniqueTitlesFromDocs(this.chainManager.getRetrievedDocuments());
if (docTitles.length > 0) {
const links = docTitles.map((title) => `- [[${title}]]`).join("\n");
response += "\n\n#### Sources:\n\n" + links;
}
return response;
}
}

View file

@ -0,0 +1,22 @@
// Main exports for chain runners
export type { ChainRunner } from "./BaseChainRunner";
export { BaseChainRunner } from "./BaseChainRunner";
export { LLMChainRunner } from "./LLMChainRunner";
export { VaultQAChainRunner } from "./VaultQAChainRunner";
export { CopilotPlusChainRunner } from "./CopilotPlusChainRunner";
export { ProjectChainRunner } from "./ProjectChainRunner";
export { AutonomousAgentChainRunner } from "./AutonomousAgentChainRunner";
// Utility exports (for internal use or testing)
export { ThinkBlockStreamer } from "./utils/ThinkBlockStreamer";
export { parseXMLToolCalls, stripToolCallXML } from "./utils/xmlParsing";
export type { ToolCall } from "./utils/xmlParsing";
export {
executeSequentialToolCall,
getToolDisplayName,
getToolEmoji,
logToolCall,
logToolResult,
deduplicateSources,
} from "./utils/toolExecution";
export type { ToolExecutionResult } from "./utils/toolExecution";

View file

@ -0,0 +1,232 @@
import { ActionBlockStreamer } from "./ActionBlockStreamer";
import { ToolManager } from "@/tools/toolManager";
import { ToolResultFormatter } from "@/tools/ToolResultFormatter";
// Mock the ToolManager and ToolResultFormatter
jest.mock("@/tools/toolManager");
jest.mock("@/tools/ToolResultFormatter");
const MockedToolManager = ToolManager as jest.Mocked<typeof ToolManager>;
const MockedToolResultFormatter = ToolResultFormatter as jest.Mocked<typeof ToolResultFormatter>;
describe("ActionBlockStreamer", () => {
let writeToFileTool: any;
let streamer: ActionBlockStreamer;
beforeEach(() => {
writeToFileTool = { name: "writeToFile" };
MockedToolManager.callTool.mockClear();
// Mock ToolResultFormatter to return the raw result without "File change result: " prefix
MockedToolResultFormatter.format = jest.fn((_toolName, result) => result);
streamer = new ActionBlockStreamer(MockedToolManager, writeToFileTool);
});
// Helper function to process chunks and collect results
async function processChunks(chunks: { content: string | null }[]) {
const outputContents: any[] = [];
for (const chunk of chunks) {
for await (const result of streamer.processChunk(chunk)) {
// Always push the content, even if it's null, undefined, or empty string
outputContents.push(result.content);
}
}
return outputContents;
}
it("should pass through chunks without writeToFile tags unchanged", async () => {
const chunks = [{ content: "Hello " }, { content: "world, this is " }, { content: "a test." }];
const output = await processChunks(chunks);
// All chunks should be yielded as-is
expect(output).toEqual(["Hello ", "world, this is ", "a test."]);
// No tool calls should be made
expect(MockedToolManager.callTool).not.toHaveBeenCalled();
});
it("should handle a complete writeToFile block in a single chunk", async () => {
MockedToolManager.callTool.mockResolvedValue("File written successfully.");
const chunks = [
{
content:
"Some text before <writeToFile><path>file.txt</path><content>content</content></writeToFile> and some text after.",
},
];
const output = await processChunks(chunks);
// Should yield original chunk plus tool result
expect(output).toEqual([
"Some text before <writeToFile><path>file.txt</path><content>content</content></writeToFile> and some text after.",
"\nFile written successfully.\n",
]);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "file.txt",
content: "content",
});
});
it("should handle a complete xml-wrapped writeToFile block", async () => {
MockedToolManager.callTool.mockResolvedValue("XML file written.");
const chunks = [
{
content:
"```xml\n<writeToFile><path>file.xml</path><content>xml content</content></writeToFile>\n```",
},
];
const output = await processChunks(chunks);
expect(output).toEqual([
"```xml\n<writeToFile><path>file.xml</path><content>xml content</content></writeToFile>\n```",
"\nXML file written.\n",
]);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "file.xml",
content: "xml content",
});
});
it("should handle a writeToFile block split across multiple chunks", async () => {
MockedToolManager.callTool.mockResolvedValue("Split file written.");
const chunks = [
{ content: "Here is a file <writeToFile><path>split.txt</path>" },
{ content: "<content>split content</content>" },
{ content: "</writeToFile> That was it." },
];
const output = await processChunks(chunks);
// All chunks should be yielded as-is, plus tool result when complete block is detected
expect(output).toEqual([
"Here is a file <writeToFile><path>split.txt</path>",
"<content>split content</content>",
"</writeToFile> That was it.",
"\nSplit file written.\n",
]);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "split.txt",
content: "split content",
});
});
it("should handle multiple writeToFile blocks in the stream", async () => {
MockedToolManager.callTool
.mockResolvedValueOnce("File 1 written.")
.mockResolvedValueOnce("File 2 written.");
const chunks = [
{
content:
"<writeToFile><path>f1.txt</path><content>c1</content></writeToFile>Some text<writeToFile><path>f2.txt</path><content>c2</content></writeToFile>",
},
];
const output = await processChunks(chunks);
// Should yield original chunk plus both tool results
expect(output).toEqual([
"<writeToFile><path>f1.txt</path><content>c1</content></writeToFile>Some text<writeToFile><path>f2.txt</path><content>c2</content></writeToFile>",
"\nFile 1 written.\n",
"\nFile 2 written.\n",
]);
expect(MockedToolManager.callTool).toHaveBeenCalledTimes(2);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "f1.txt",
content: "c1",
});
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "f2.txt",
content: "c2",
});
});
it("should handle unclosed tags without calling tools", async () => {
const chunks = [
{ content: "Starting... <writeToFile><path>unclosed.txt</path>" },
{ content: "<content>this will not be closed" },
];
const output = await processChunks(chunks);
// Should yield all chunks as-is
expect(output).toEqual([
"Starting... <writeToFile><path>unclosed.txt</path>",
"<content>this will not be closed",
]);
// No tool calls should be made for incomplete blocks
expect(MockedToolManager.callTool).not.toHaveBeenCalled();
});
it("should handle tool call errors gracefully", async () => {
MockedToolManager.callTool.mockRejectedValue(new Error("Tool error"));
const chunks = [
{
content: "<writeToFile><path>error.txt</path><content>content</content></writeToFile>",
},
];
const output = await processChunks(chunks);
expect(output).toEqual([
"<writeToFile><path>error.txt</path><content>content</content></writeToFile>",
"\nError: Tool error\n",
]);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "error.txt",
content: "content",
});
});
it("should handle chunks with different content types", async () => {
const chunks: any[] = [
{ content: "Hello" },
{ content: null },
{ content: "" },
{ content: " World" },
];
const output = await processChunks(chunks);
// Should yield all chunks as-is, null content is yielded but not added to buffer
expect(output).toEqual(["Hello", null, "", " World"]);
});
it("should handle whitespace in path and content", async () => {
MockedToolManager.callTool.mockResolvedValue("Whitespace handled.");
const chunks = [
{
content:
"<writeToFile><path> spaced.txt </path><content> content with spaces </content></writeToFile>",
},
];
await processChunks(chunks);
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "spaced.txt",
content: "content with spaces",
});
});
it("should handle malformed blocks gracefully", async () => {
MockedToolManager.callTool.mockResolvedValue("Malformed handled.");
const chunks = [
{
content: "<writeToFile><path>missing-content.txt</path></writeToFile>",
},
];
const output = await processChunks(chunks);
// Should yield chunk as-is plus tool result
expect(output).toEqual([
"<writeToFile><path>missing-content.txt</path></writeToFile>",
"\nMalformed handled.\n",
]);
// Tool should be called with undefined content
expect(MockedToolManager.callTool).toHaveBeenCalledWith(writeToFileTool, {
path: "missing-content.txt",
content: undefined,
});
});
});

View file

@ -0,0 +1,93 @@
import { ToolManager } from "@/tools/toolManager";
import { ToolResultFormatter } from "@/tools/ToolResultFormatter";
/**
* ActionBlockStreamer processes streaming chunks to detect and handle writeToFile blocks.
*
* 1. Accumulates chunks in a buffer
* 2. Detects complete writeToFile blocks
* 3. Calls the writeToFile tool when a complete block is found
* 4. Returns chunks as-is otherwise
*/
export class ActionBlockStreamer {
private buffer = "";
constructor(
private toolManager: typeof ToolManager,
private writeToFileTool: any
) {}
private findCompleteBlock(str: string) {
// Regex for both formats
const regex = /<writeToFile>[\s\S]*?<\/writeToFile>/;
const match = str.match(regex);
if (!match || match.index === undefined) {
return null;
}
return {
block: match[0],
endIdx: match.index + match[0].length,
};
}
async *processChunk(chunk: any): AsyncGenerator<any, void, unknown> {
// Handle different chunk formats
let chunkContent = "";
// Handle Claude thinking model array-based content
if (Array.isArray(chunk.content)) {
for (const item of chunk.content) {
if (item.type === "text" && item.text != null) {
chunkContent += item.text;
}
}
}
// Handle standard string content
else if (chunk.content != null) {
chunkContent = chunk.content;
}
// Add to buffer
if (chunkContent) {
this.buffer += chunkContent;
}
// Yield the original chunk as-is
yield chunk;
// Process all complete blocks in the buffer
let blockInfo = this.findCompleteBlock(this.buffer);
while (blockInfo) {
const { block, endIdx } = blockInfo;
// Extract content from the block
const pathMatch = block.match(/<path>([\s\S]*?)<\/path>/);
const contentMatch = block.match(/<content>([\s\S]*?)<\/content>/);
const filePath = pathMatch ? pathMatch[1].trim() : undefined;
const fileContent = contentMatch ? contentMatch[1].trim() : undefined;
// Call the tool
try {
const result = await this.toolManager.callTool(this.writeToFileTool, {
path: filePath,
content: fileContent,
});
// Format tool result using ToolResultFormatter for consistency with agent mode
const formattedResult = ToolResultFormatter.format("writeToFile", result);
yield { ...chunk, content: `\n${formattedResult}\n` };
} catch (err: any) {
yield { ...chunk, content: `\nError: ${err?.message || err}\n` };
}
// Remove processed block from buffer
this.buffer = this.buffer.substring(endIdx);
// Check for another complete block in the remaining buffer
blockInfo = this.findCompleteBlock(this.buffer);
}
}
}

View file

@ -0,0 +1,124 @@
import { ModelAdapter } from "./modelAdapter";
/**
* ThinkBlockStreamer handles streaming content from various LLM providers
* that support thinking/reasoning modes (like Claude and Deepseek).
*/
export class ThinkBlockStreamer {
private hasOpenThinkBlock = false;
private fullResponse = "";
private shouldTruncate = false;
constructor(
private updateCurrentAiMessage: (message: string) => void,
private modelAdapter?: ModelAdapter
) {}
private handleClaude37Chunk(content: any[]) {
let textContent = "";
for (const item of content) {
switch (item.type) {
case "text":
textContent += item.text;
break;
case "thinking":
if (!this.hasOpenThinkBlock) {
this.fullResponse += "\n<think>";
this.hasOpenThinkBlock = true;
}
// Guard against undefined thinking content
if (item.thinking !== undefined) {
this.fullResponse += item.thinking;
}
this.updateCurrentAiMessage(this.fullResponse);
return true; // Indicate we handled a thinking chunk
}
}
if (textContent) {
this.fullResponse += textContent;
}
return false; // No thinking chunk handled
}
private handleDeepseekChunk(chunk: any) {
// Handle standard string content
if (typeof chunk.content === "string") {
this.fullResponse += chunk.content;
}
// Handle deepseek reasoning/thinking content
if (chunk.additional_kwargs?.reasoning_content) {
if (!this.hasOpenThinkBlock) {
this.fullResponse += "\n<think>";
this.hasOpenThinkBlock = true;
}
// Guard against undefined reasoning content
if (chunk.additional_kwargs.reasoning_content !== undefined) {
this.fullResponse += chunk.additional_kwargs.reasoning_content;
}
return true; // Indicate we handled a thinking chunk
}
return false; // No thinking chunk handled
}
processChunk(chunk: any) {
// If we've already decided to truncate, don't process more chunks
if (this.shouldTruncate) {
return;
}
let handledThinking = false;
// Handle Claude 3.7 array-based content
if (Array.isArray(chunk.content)) {
handledThinking = this.handleClaude37Chunk(chunk.content);
} else {
// Handle deepseek format
handledThinking = this.handleDeepseekChunk(chunk);
}
// Close think block if we have one open and didn't handle thinking content
if (this.hasOpenThinkBlock && !handledThinking) {
this.fullResponse += "</think>";
this.hasOpenThinkBlock = false;
}
// Check if we should truncate streaming based on model adapter
if (this.modelAdapter?.shouldTruncateStreaming?.(this.fullResponse)) {
this.shouldTruncate = true;
// Find the last complete tool call to truncate cleanly
this.fullResponse = this.truncateToLastCompleteToolCall(this.fullResponse);
}
this.updateCurrentAiMessage(this.fullResponse);
}
private truncateToLastCompleteToolCall(response: string): string {
// Find the last complete </use_tool> tag
const lastCompleteToolEnd = response.lastIndexOf("</use_tool>");
if (lastCompleteToolEnd === -1) {
// No complete tool calls found, return original response
return response;
}
// Truncate to after the last complete tool call
const truncated = response.substring(0, lastCompleteToolEnd + "</use_tool>".length);
// Use model adapter to sanitize if available
if (this.modelAdapter?.sanitizeResponse) {
return this.modelAdapter.sanitizeResponse(truncated, 1);
}
return truncated;
}
close() {
// Make sure to close any open think block at the end
if (this.hasOpenThinkBlock) {
this.fullResponse += "</think>";
this.updateCurrentAiMessage(this.fullResponse);
}
return this.fullResponse;
}
}

View file

@ -0,0 +1,245 @@
import {
processRawChatHistory,
addChatHistoryToMessages,
processedMessagesToTextOnly,
} from "./chatHistoryUtils";
describe("chatHistoryUtils", () => {
describe("processRawChatHistory", () => {
it("should process BaseMessage objects correctly", () => {
const rawHistory = [
{
_getType: () => "human",
content: "Hello",
},
{
_getType: () => "ai",
content: "Hi there!",
},
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([
{ role: "user", content: "Hello" },
{ role: "assistant", content: "Hi there!" },
]);
});
it("should handle multimodal content in BaseMessage objects", () => {
const multimodalContent = [
{ type: "text", text: "What is this?" },
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,..." } },
];
const rawHistory = [
{
_getType: () => "human",
content: multimodalContent,
},
{
_getType: () => "ai",
content: "This is an image of a cat.",
},
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([
{ role: "user", content: multimodalContent },
{ role: "assistant", content: "This is an image of a cat." },
]);
});
it("should skip system messages", () => {
const rawHistory = [
{
_getType: () => "system",
content: "You are a helpful assistant",
},
{
_getType: () => "human",
content: "Hello",
},
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([{ role: "user", content: "Hello" }]);
});
it("should handle legacy message formats", () => {
const rawHistory = [
{
role: "human",
content: "Hello",
},
{
role: "ai",
content: "Hi!",
},
{
sender: "user",
content: "How are you?",
},
{
sender: "AI",
content: "I am doing well!",
},
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([
{ role: "user", content: "Hello" },
{ role: "assistant", content: "Hi!" },
{ role: "user", content: "How are you?" },
{ role: "assistant", content: "I am doing well!" },
]);
});
it("should handle null and undefined messages", () => {
const rawHistory = [
null,
undefined,
{
_getType: () => "human",
content: "Hello",
},
{},
{ content: undefined },
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([{ role: "user", content: "Hello" }]);
});
it("should handle messages with unknown types", () => {
const rawHistory = [
{
_getType: () => "unknown",
content: "Some content",
},
{
role: "unknown",
content: "Other content",
},
];
const result = processRawChatHistory(rawHistory);
expect(result).toEqual([]);
});
});
describe("addChatHistoryToMessages", () => {
it("should add processed messages to target array", () => {
const rawHistory = [
{
_getType: () => "human",
content: "Hello",
},
{
_getType: () => "ai",
content: "Hi there!",
},
];
const messages: any[] = [];
addChatHistoryToMessages(rawHistory, messages);
expect(messages).toEqual([
{ role: "user", content: "Hello" },
{ role: "assistant", content: "Hi there!" },
]);
});
it("should append to existing messages", () => {
const rawHistory = [
{
_getType: () => "human",
content: "Hello",
},
];
const messages = [{ role: "system", content: "You are helpful" }];
addChatHistoryToMessages(rawHistory, messages);
expect(messages).toEqual([
{ role: "system", content: "You are helpful" },
{ role: "user", content: "Hello" },
]);
});
});
describe("processedMessagesToTextOnly", () => {
it("should handle string content", () => {
const processedMessages = [
{ role: "user" as const, content: "Hello" },
{ role: "assistant" as const, content: "Hi there!" },
];
const result = processedMessagesToTextOnly(processedMessages);
expect(result).toEqual([
{ role: "user", content: "Hello" },
{ role: "assistant", content: "Hi there!" },
]);
});
it("should extract text from multimodal content", () => {
const processedMessages = [
{
role: "user" as const,
content: [
{ type: "text", text: "What is this?" },
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,..." } },
],
},
{
role: "assistant" as const,
content: "This is a cat.",
},
];
const result = processedMessagesToTextOnly(processedMessages);
expect(result).toEqual([
{ role: "user", content: "What is this?" },
{ role: "assistant", content: "This is a cat." },
]);
});
it("should handle multiple text parts in multimodal content", () => {
const processedMessages = [
{
role: "user" as const,
content: [
{ type: "text", text: "First part." },
{ type: "image_url", image_url: { url: "data:image/jpeg;base64,..." } },
{ type: "text", text: "Second part." },
],
},
];
const result = processedMessagesToTextOnly(processedMessages);
expect(result).toEqual([{ role: "user", content: "First part. Second part." }]);
});
it("should handle image-only content", () => {
const processedMessages = [
{
role: "user" as const,
content: [{ type: "image_url", image_url: { url: "data:image/jpeg;base64,..." } }],
},
];
const result = processedMessagesToTextOnly(processedMessages);
expect(result).toEqual([{ role: "user", content: "[Image content]" }]);
});
});
});

View file

@ -0,0 +1,112 @@
/**
* Utility functions for safely processing chat history from LangChain memory
*/
export interface ProcessedMessage {
role: "user" | "assistant";
content: any; // string or MessageContent[]
}
/**
* Safely process raw history from LangChain memory, handling both BaseMessage
* objects and legacy formats while preserving multimodal content
*
* @param rawHistory Array of messages from memory.loadMemoryVariables()
* @returns Array of processed messages safe for LLM consumption
*/
export function processRawChatHistory(rawHistory: any[]): ProcessedMessage[] {
const messages: ProcessedMessage[] = [];
for (const message of rawHistory) {
if (!message) continue;
// Check if this is a BaseMessage with _getType method
if (typeof message._getType === "function") {
const messageType = message._getType();
// Only process human and AI messages
if (messageType === "human") {
messages.push({ role: "user", content: message.content });
} else if (messageType === "ai") {
messages.push({ role: "assistant", content: message.content });
}
// Skip system messages and unknown types
} else if (message.content !== undefined) {
// Fallback for other message formats - try to infer role
const role = inferMessageRole(message);
if (role) {
messages.push({ role, content: message.content });
}
}
}
return messages;
}
/**
* Try to infer the role from various message format properties
* @returns 'user' | 'assistant' | null
*/
function inferMessageRole(message: any): "user" | "assistant" | null {
// Check various properties that might indicate the role
if (message.role === "human" || message.role === "user" || message.sender === "user") {
return "user";
} else if (message.role === "ai" || message.role === "assistant" || message.sender === "AI") {
return "assistant";
}
// Can't determine role
return null;
}
/**
* Add processed chat history to messages array for LLM consumption
* This is a convenience function that combines processing and adding
*
* @param rawHistory Raw history from memory
* @param messages Target messages array to add to
*/
export function addChatHistoryToMessages(rawHistory: any[], messages: any[]): void {
const processedHistory = processRawChatHistory(rawHistory);
for (const msg of processedHistory) {
messages.push({ role: msg.role, content: msg.content });
}
}
export interface ChatHistoryEntry {
role: "user" | "assistant";
content: string;
}
/**
* Convert processed messages to text-only format for question condensing
* This extracts just the text content from potentially multimodal messages
*
* @param processedMessages Messages processed by processRawChatHistory
* @returns Array of text-only chat history entries
*/
export function processedMessagesToTextOnly(
processedMessages: ProcessedMessage[]
): ChatHistoryEntry[] {
return processedMessages.map((msg) => {
let textContent: string;
if (typeof msg.content === "string") {
textContent = msg.content;
} else if (Array.isArray(msg.content)) {
// Extract text from multimodal content
const textParts = msg.content
.filter((item: any) => item.type === "text")
.map((item: any) => item.text || "")
.join(" ");
textContent = textParts || "[Image content]";
} else {
textContent = String(msg.content || "");
}
return {
role: msg.role,
content: textContent,
};
});
}

View file

@ -0,0 +1,342 @@
// Test for the image extraction logic
describe("Image extraction from content", () => {
// Mock the global app object
const mockApp = {
metadataCache: {
getFirstLinkpathDest: jest.fn(),
},
};
(global as any).app = mockApp;
// Mock logger (removed since we're no longer logging warnings)
beforeEach(() => {
jest.clearAllMocks();
});
// Helper function that replicates the extractEmbeddedImages logic
async function extractEmbeddedImages(content: string, sourcePath?: string): Promise<string[]> {
// Match both wiki-style ![[image.ext]] and standard markdown ![alt](image.ext)
const wikiImageRegex = /!\[\[(.*?\.(png|jpg|jpeg|gif|webp|bmp|svg))\]\]/g;
// Updated regex to handle URLs with or without file extensions
const markdownImageRegex = /!\[.*?\]\(([^)]+)\)/g;
const resolvedImages: string[] = [];
// Process wiki-style images
const wikiMatches = [...content.matchAll(wikiImageRegex)];
for (const match of wikiMatches) {
const imageName = match[1];
// If we have a source path and access to the app, resolve the wikilink
if (sourcePath) {
const resolvedFile = mockApp.metadataCache.getFirstLinkpathDest(imageName, sourcePath);
if (resolvedFile) {
// Use the resolved path
resolvedImages.push(resolvedFile.path);
} else {
// If file not found, still include the raw filename
resolvedImages.push(imageName);
}
} else {
// Fallback to raw filename if no source path available
resolvedImages.push(imageName);
}
}
// Process standard markdown images
const mdMatches = [...content.matchAll(markdownImageRegex)];
for (const match of mdMatches) {
const imagePath = match[1].trim();
// Skip empty paths
if (!imagePath) continue;
// Handle external URLs (http://, https://, etc.)
if (imagePath.match(/^https?:\/\//)) {
// Include external URLs - they will be processed by processImageUrls
// The ImageProcessor will validate if it's actually an image
resolvedImages.push(imagePath);
continue;
}
// For local paths, resolve them using Obsidian's metadata cache
// Let ImageBatchProcessor handle validation of whether it's actually an image
// Clean up the path (remove any leading ./ or /)
const cleanPath = imagePath.replace(/^\.\//, "").replace(/^\//, "");
// If we have a source path and access to the app, resolve the path
if (sourcePath) {
const resolvedFile = mockApp.metadataCache.getFirstLinkpathDest(cleanPath, sourcePath);
if (resolvedFile) {
// Use the resolved path
resolvedImages.push(resolvedFile.path);
} else {
// If file not found, still include the raw path
// Let ImageBatchProcessor handle validation
resolvedImages.push(cleanPath);
}
} else {
// Fallback to raw path if no source path available
resolvedImages.push(cleanPath);
}
}
return resolvedImages;
}
describe("Wiki-style image syntax ![[image]]", () => {
it("should extract wiki-style images with extensions", async () => {
const content = "Here is an image ![[screenshot.png]] and another ![[diagram.jpg]]";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "attachments/screenshot.png" })
.mockReturnValueOnce({ path: "images/diagram.jpg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["attachments/screenshot.png", "images/diagram.jpg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"screenshot.png",
sourcePath
);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"diagram.jpg",
sourcePath
);
});
it("should handle wiki-style images with paths", async () => {
const content = "Image with path ![[folder/image.png]]";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest.mockReturnValueOnce({
path: "resolved/folder/image.png",
});
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["resolved/folder/image.png"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"folder/image.png",
sourcePath
);
});
it("should fallback to raw filename when file not found", async () => {
const content = "Missing image ![[missing.png]]";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest.mockReturnValueOnce(null);
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["missing.png"]);
});
});
describe("Markdown image syntax ![](image)", () => {
it("should extract markdown images with simple paths", async () => {
const content = "Here is ![alt text](image.png) and ![](photo.jpg)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "attachments/image.png" })
.mockReturnValueOnce({ path: "attachments/photo.jpg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["attachments/image.png", "attachments/photo.jpg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"image.png",
sourcePath
);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"photo.jpg",
sourcePath
);
});
it("should handle markdown images with relative paths", async () => {
const content = "Relative paths ![](./images/test.png) and ![](../assets/icon.svg)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "images/test.png" })
.mockReturnValueOnce({ path: "assets/icon.svg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["images/test.png", "assets/icon.svg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"images/test.png",
sourcePath
);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"../assets/icon.svg",
sourcePath
);
});
it("should include external URLs", async () => {
const content =
"External images ![](https://example.com/image.png) and ![](http://site.com/pic.jpg)";
const sourcePath = "notes/test.md";
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["https://example.com/image.png", "http://site.com/pic.jpg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).not.toHaveBeenCalled();
});
it("should include external URLs with query parameters", async () => {
const content =
"Unsplash image ![](https://images.unsplash.com/photo-1746555697990-3a405a5152b9?q=80&w=1587&auto=format&fit=crop&ixlib=rb-4.1.0&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D)";
const sourcePath = "notes/test.md";
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual([
"https://images.unsplash.com/photo-1746555697990-3a405a5152b9?q=80&w=1587&auto=format&fit=crop&ixlib=rb-4.1.0&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D",
]);
expect(mockApp.metadataCache.getFirstLinkpathDest).not.toHaveBeenCalled();
});
it("should clean leading slashes from paths", async () => {
const content = "Absolute path ![](/images/absolute.png)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest.mockReturnValueOnce({
path: "images/absolute.png",
});
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["images/absolute.png"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledWith(
"images/absolute.png",
sourcePath
);
});
});
describe("Mixed syntaxes", () => {
it("should extract both wiki and markdown images", async () => {
const content = "Wiki ![[wiki.png]] and markdown ![](markdown.jpg) together";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "attachments/wiki.png" })
.mockReturnValueOnce({ path: "attachments/markdown.jpg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["attachments/wiki.png", "attachments/markdown.jpg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledTimes(2);
});
it("should handle multiple images of both types", async () => {
const content = `
![[first.png]] some text ![](second.jpg)
More text ![[third.gif]] and ![alt](fourth.svg)
`;
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "imgs/first.png" })
.mockReturnValueOnce({ path: "imgs/third.gif" })
.mockReturnValueOnce({ path: "imgs/second.jpg" })
.mockReturnValueOnce({ path: "imgs/fourth.svg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual([
"imgs/first.png",
"imgs/third.gif",
"imgs/second.jpg",
"imgs/fourth.svg",
]);
});
});
describe("Edge cases", () => {
it("should extract all markdown links regardless of extension", async () => {
const content = "Valid ![[image.png]] but not ![[document.pdf]] or ![](script.js)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "attachments/image.png" })
.mockReturnValueOnce(null); // script.js not found
const result = await extractEmbeddedImages(content, sourcePath);
// Wiki-style ![[document.pdf]] is skipped because it doesn't have image extension
// But markdown ![](script.js) is included - ImageBatchProcessor will validate
expect(result).toEqual(["attachments/image.png", "script.js"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledTimes(2);
});
it("should handle content with no images", async () => {
const content = "Just text with no images";
const sourcePath = "notes/test.md";
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual([]);
expect(mockApp.metadataCache.getFirstLinkpathDest).not.toHaveBeenCalled();
});
it("should handle no source path", async () => {
const content = "Image ![[test.png]] and ![](other.jpg)";
const result = await extractEmbeddedImages(content);
expect(result).toEqual(["test.png", "other.jpg"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).not.toHaveBeenCalled();
});
it("should handle malformed markdown syntax", async () => {
const content = "Malformed ![](no-extension) and ![]() empty";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest.mockReturnValueOnce(null); // no-extension not found
const result = await extractEmbeddedImages(content, sourcePath);
// no-extension is included, empty path is skipped
expect(result).toEqual(["no-extension"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledTimes(1);
});
it("should include all markdown links for local paths (validation happens in ImageBatchProcessor)", async () => {
const content = "Document ![](document.pdf) and text ![](notes.md)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce(null) // document.pdf not found
.mockReturnValueOnce(null); // notes.md not found
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["document.pdf", "notes.md"]);
expect(mockApp.metadataCache.getFirstLinkpathDest).toHaveBeenCalledTimes(2);
});
it("should handle special characters in filenames", async () => {
const content = "Special chars ![[image (1).png]] and ![](file-name_2.jpg)";
const sourcePath = "notes/test.md";
mockApp.metadataCache.getFirstLinkpathDest
.mockReturnValueOnce({ path: "attachments/image (1).png" })
.mockReturnValueOnce({ path: "attachments/file-name_2.jpg" });
const result = await extractEmbeddedImages(content, sourcePath);
expect(result).toEqual(["attachments/image (1).png", "attachments/file-name_2.jpg"]);
});
});
});

View file

@ -0,0 +1,155 @@
import { ModelAdapterFactory } from "./modelAdapter";
import { ToolMetadata } from "@/tools/ToolRegistry";
describe("ModelAdapter", () => {
describe("enhanceSystemPrompt", () => {
const basePrompt = "You are a helpful assistant.";
const toolDescriptions = "Tool descriptions here";
const createToolMetadata = (id: string, instructions: string): ToolMetadata => ({
id,
displayName: `${id} Display`,
description: `${id} description`,
category: "custom",
customPromptInstructions: instructions,
});
it("should include tool instructions when tools are enabled", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const toolMetadata: ToolMetadata[] = [
createToolMetadata("localSearch", "LocalSearch specific instructions"),
createToolMetadata("webSearch", "WebSearch specific instructions"),
createToolMetadata("writeToFile", "WriteToFile specific instructions"),
];
const enhancedPrompt = adapter.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
["localSearch", "webSearch", "writeToFile"],
toolMetadata
);
// Check that tool instructions are included
expect(enhancedPrompt).toContain("LocalSearch specific instructions");
expect(enhancedPrompt).toContain("WebSearch specific instructions");
expect(enhancedPrompt).toContain("WriteToFile specific instructions");
});
it("should only include instructions for enabled tools", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const toolMetadata: ToolMetadata[] = [
createToolMetadata("localSearch", "LocalSearch specific instructions"),
createToolMetadata("webSearch", "WebSearch specific instructions"),
createToolMetadata("writeToFile", "WriteToFile specific instructions"),
];
// Only pass localSearch as enabled
const enhancedPrompt = adapter.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
["localSearch"],
[toolMetadata[0]] // Only localSearch metadata
);
// Should include localSearch instructions
expect(enhancedPrompt).toContain("LocalSearch specific instructions");
// Should NOT include other tool instructions
expect(enhancedPrompt).not.toContain("WebSearch specific instructions");
expect(enhancedPrompt).not.toContain("WriteToFile specific instructions");
});
it("should include base structure elements", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(basePrompt, toolDescriptions, [], []);
// Check base sections exist
expect(enhancedPrompt).toContain("# Autonomous Agent Mode");
expect(enhancedPrompt).toContain("## Time-based Queries");
expect(enhancedPrompt).toContain("## General Guidelines");
});
it("should handle GPT-specific enhancements", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(basePrompt, toolDescriptions, [], []);
// Check GPT-specific sections
expect(enhancedPrompt).toContain("CRITICAL FOR GPT MODELS");
expect(enhancedPrompt).toContain("FINAL REMINDER FOR GPT MODELS");
});
it("should handle Claude-specific enhancements", () => {
const mockModel = { modelName: "claude-3-7-sonnet" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(basePrompt, toolDescriptions, [], []);
// Check Claude-specific sections
expect(enhancedPrompt).toContain("IMPORTANT FOR CLAUDE THINKING MODELS");
});
it("should handle Gemini-specific enhancements", () => {
const mockModel = { modelName: "gemini-pro" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(basePrompt, toolDescriptions, [], []);
// Check Gemini-specific sections
expect(enhancedPrompt).toContain("CRITICAL INSTRUCTIONS FOR GEMINI");
});
it("should exclude instructions when no metadata provided", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
["localSearch", "webSearch"],
[] // No metadata
);
// Should not include any tool-specific instructions
expect(enhancedPrompt).not.toContain("LocalSearch specific instructions");
expect(enhancedPrompt).not.toContain("WebSearch specific instructions");
});
it("should include composer-specific examples for GPT when file tools are enabled", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const enhancedPrompt = adapter.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
["replaceInFile", "writeToFile"],
[]
);
// Check for composer-specific GPT instructions
expect(enhancedPrompt).toContain("FILE EDITING WITH COMPOSER TOOLS");
expect(enhancedPrompt).toContain("fix the typo");
expect(enhancedPrompt).toContain("add item 4 to the list");
expect(enhancedPrompt).toContain("diff parameter MUST contain");
});
it("should enhance file editing messages for GPT", () => {
const mockModel = { modelName: "gpt-4" } as any;
const adapter = ModelAdapterFactory.createAdapter(mockModel);
const editMessage = "fix the typo in my note";
const enhanced = adapter.enhanceUserMessage(editMessage, true);
expect(enhanced).toContain("GPT REMINDER");
expect(enhanced).toContain("replaceInFile");
expect(enhanced).toContain("SEARCH/REPLACE blocks");
});
});
});

View file

@ -0,0 +1,697 @@
import { BaseChatModel } from "@langchain/core/language_models/chat_models";
import { logInfo } from "@/logger";
import { ToolMetadata } from "@/tools/ToolRegistry";
/**
* Model-specific adaptations for autonomous agent
* Handles quirks and requirements of different LLM providers
*/
export interface ModelAdapter {
/**
* Enhance system prompt with model-specific instructions
* @param basePrompt - The base system prompt to enhance
* @param toolDescriptions - Available tool descriptions to include
* @param availableToolNames - List of enabled tool names (for backward compatibility)
* @param toolMetadata - Tool metadata including custom instructions
* @returns The enhanced system prompt
*/
enhanceSystemPrompt(
basePrompt: string,
toolDescriptions: string,
availableToolNames?: string[],
toolMetadata?: ToolMetadata[]
): string;
/**
* Enhance user message if needed for specific models
* @param message - The user's message to enhance
* @param requiresTools - Whether the message likely requires tool usage
* @returns The enhanced user message
*/
enhanceUserMessage(message: string, requiresTools: boolean): string;
/**
* Parse tool calls from model response (future: handle different formats)
* @param response - The model's response text containing tool calls
* @returns Array of parsed tool calls
*/
parseToolCalls?(response: string): any[];
/**
* Check if model needs special handling
* @returns True if the model requires special handling beyond base behavior
*/
needsSpecialHandling(): boolean;
/**
* Sanitize response for autonomous agent mode (e.g., remove premature content)
* @param response - The model's response text to sanitize
* @param iteration - The current iteration number in the autonomous agent loop
* @returns The sanitized response text
*/
sanitizeResponse?(response: string, iteration: number): string;
/**
* Check if streaming should be truncated early for this model
* @param partialResponse - The partial response text received during streaming
* @returns True if streaming should be truncated at this point
*/
shouldTruncateStreaming?(partialResponse: string): boolean;
/**
* Detect premature responses for this model
* @param response - The model's response text to analyze
* @returns Object indicating if premature content was detected and its type
*/
detectPrematureResponse?(response: string): {
hasPremature: boolean;
type: "before" | "after" | null;
};
}
/**
* Base adapter with default behavior (no modifications)
*/
class BaseModelAdapter implements ModelAdapter {
constructor(protected modelName: string) {}
private buildToolSpecificInstructions(toolMetadata: ToolMetadata[]): string {
const instructions: string[] = [];
// Collect all custom instructions from tool metadata
for (const meta of toolMetadata) {
if (meta.customPromptInstructions) {
instructions.push(meta.customPromptInstructions);
}
}
return instructions.length > 0 ? instructions.join("\n\n") : "";
}
enhanceSystemPrompt(
basePrompt: string,
toolDescriptions: string,
availableToolNames?: string[],
toolMetadata?: ToolMetadata[]
): string {
const metadata = toolMetadata || [];
// Build tool-specific instructions from metadata
const toolSpecificInstructions = this.buildToolSpecificInstructions(metadata);
return `${basePrompt}
# Autonomous Agent Mode
You are now in autonomous agent mode. You can use tools to gather information and complete tasks step by step.
When you need to use a tool, format it EXACTLY like this:
<use_tool>
<name>tool_name_here</name>
<parameter_name>value</parameter_name>
<another_parameter>["array", "values"]</another_parameter>
</use_tool>
IMPORTANT: Use the EXACT parameter names as shown in the tool descriptions below. Do NOT use generic names like "param1" or "param".
Available tools:
${toolDescriptions}
# Tool Usage Guidelines
## Time-based Queries
When users ask about temporal periods (e.g., "what did I do last month", "show me notes from last week"), you MUST:
1. First call getTimeRangeMs to convert the time expression to a proper time range
2. Then use localSearch with the timeRange parameter from step 1
3. For salientTerms, ONLY use words that exist in the user's original query (excluding time expressions)
Example for "what did I do last month":
1. Call getTimeRangeMs with timeExpression: "last month"
2. Use localSearch with query matching the user's question
3. salientTerms: [] - empty because "what", "I", "do" are not meaningful search terms
Example for "meetings about project X last week":
1. Call getTimeRangeMs with timeExpression: "last week"
2. Use localSearch with query "meetings about project X"
3. salientTerms: ["meetings", "project", "X"] - these words exist in the original query
${toolSpecificInstructions ? toolSpecificInstructions + "\n\n" : ""}## General Guidelines
- NEVER mention tool names like "localSearch", "webSearch", etc. in your responses. Use natural language like "searching your vault", "searching the web", etc.
You can use multiple tools in sequence. After each tool execution, you'll receive the results and can decide whether to use more tools or provide your final response.
Always explain your reasoning before using tools. Be conversational and clear about what you're doing.
When you've gathered enough information, provide your final response without any tool calls.
IMPORTANT: Do not include any code blocks (\`\`\`) or tool_code blocks in your responses. Only use the <use_tool> format for tool calls.
NOTE: Use individual XML parameter tags. For arrays, use JSON format like ["item1", "item2"].`;
}
enhanceUserMessage(message: string, requiresTools: boolean): string {
return message;
}
needsSpecialHandling(): boolean {
return false;
}
}
/**
* GPT-specific adapter with aggressive prompting
*/
class GPTModelAdapter extends BaseModelAdapter {
enhanceSystemPrompt(
basePrompt: string,
toolDescriptions: string,
availableToolNames?: string[],
toolMetadata?: ToolMetadata[]
): string {
const baseSystemPrompt = super.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
availableToolNames,
toolMetadata
);
const tools = availableToolNames || [];
const hasComposerTools = tools.includes("writeToFile") || tools.includes("replaceInFile");
// Insert GPT-specific instructions after the base prompt
let gptSpecificSection = `
CRITICAL FOR GPT MODELS: You MUST ALWAYS include XML tool calls in your response. Do not just describe what you plan to do - you MUST include the actual XML tool call blocks.`;
if (hasComposerTools) {
gptSpecificSection += `
🚨 FILE EDITING WITH COMPOSER TOOLS - GPT SPECIFIC EXAMPLES 🚨
When user asks you to edit or modify a file, you MUST:
1. Determine if it's a small edit (use replaceInFile) or major rewrite (use writeToFile)
2. Include the tool call immediately in your response
EXAMPLE 1 - User says "fix the typo 'teh' to 'the' in my note":
CORRECT RESPONSE:
"I'll fix the typo in your note.
<use_tool>
<name>replaceInFile</name>
<path>path/to/note.md</path>
<diff>\`\`\`
------- SEARCH
teh
=======
the
+++++++ REPLACE
\`\`\`</diff>
</use_tool>"
EXAMPLE 2 - User says "add item 4 to the list":
CORRECT RESPONSE:
"I'll add item 4 to your list.
<use_tool>
<name>replaceInFile</name>
<path>path/to/file.md</path>
<diff>\`\`\`
------- SEARCH
- Item 1
- Item 2
- Item 3
=======
- Item 1
- Item 2
- Item 3
- Item 4
+++++++ REPLACE
\`\`\`</diff>
</use_tool>"
WRONG (DO NOT DO THIS):
"I'll help you add item 4 to the list. Let me update that for you."
[No tool call = FAILURE]
CRITICAL: The diff parameter MUST contain the SEARCH/REPLACE blocks wrapped in triple backticks EXACTLY as shown above.`;
}
gptSpecificSection += `
FINAL REMINDER FOR GPT MODELS: If the user asks you to search, find, edit, or modify anything, you MUST include the appropriate <use_tool> XML block in your very next response. Do not wait for another turn.`;
return baseSystemPrompt + gptSpecificSection;
}
enhanceUserMessage(message: string, requiresTools: boolean): string {
if (requiresTools) {
const lowerMessage = message.toLowerCase();
const requiresSearch =
lowerMessage.includes("find") ||
lowerMessage.includes("search") ||
lowerMessage.includes("my notes");
const requiresFileEdit =
lowerMessage.includes("edit") ||
lowerMessage.includes("modify") ||
lowerMessage.includes("update") ||
lowerMessage.includes("change") ||
lowerMessage.includes("fix") ||
lowerMessage.includes("add") ||
lowerMessage.includes("typo");
if (requiresSearch) {
return `${message}\n\nREMINDER: Use the <use_tool> XML format to call the localSearch tool.`;
}
if (requiresFileEdit) {
return `${message}\n\n🚨 GPT REMINDER: Use replaceInFile for small edits (with SEARCH/REPLACE blocks in diff parameter). The diff parameter MUST contain triple backticks around the SEARCH/REPLACE blocks. Check the examples in your system prompt.`;
}
}
return message;
}
needsSpecialHandling(): boolean {
return true;
}
}
/**
* Claude adapter with special handling for thinking models
*/
class ClaudeModelAdapter extends BaseModelAdapter {
/**
* Check if this is a Claude thinking model (3.7 Sonnet or Claude 4)
* @returns True if the model supports thinking/reasoning modes
*/
private isThinkingModel(): boolean {
return (
this.modelName.includes("claude-3-7-sonnet") ||
this.modelName.includes("claude-sonnet-4") ||
this.modelName.includes("claude-3.7-sonnet") ||
this.modelName.includes("claude-4-sonnet")
);
}
/**
* Check if this is specifically Claude Sonnet 4 (has hallucination issues)
* @returns True if the model is Claude Sonnet 4 variants
*/
private isClaudeSonnet4(): boolean {
return (
this.modelName.includes("claude-sonnet-4") ||
this.modelName.includes("claude-4-sonnet") ||
this.modelName.includes("claude-sonnet-4-20250514")
);
}
enhanceSystemPrompt(
basePrompt: string,
toolDescriptions: string,
availableToolNames?: string[],
toolMetadata?: ToolMetadata[]
): string {
const baseSystemPrompt = super.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
availableToolNames,
toolMetadata
);
// Add specific instructions for thinking models
if (this.isThinkingModel()) {
let thinkingModelSection = `
IMPORTANT FOR CLAUDE THINKING MODELS:
- You are a thinking model with internal reasoning capability
- Your thinking process will be automatically wrapped in <think> tags - do not manually add thinking tags
- Place ALL tool calls AFTER your thinking is complete, in the main response body
- Tool calls must be in the main response body, NOT inside thinking sections
- Format tool calls exactly as shown in the examples above
- Do not provide final answers before using tools - use tools first, then provide your response based on the results
- If you need to use tools, include them immediately after your thinking, before any final response
CORRECT FLOW FOR THINKING MODELS:
1. Think through the problem (this happens automatically)
2. Use tools to gather information (place tool calls in main response)
3. Wait for tool results
4. Provide final response based on gathered information
INCORRECT: Providing a final answer before using tools
CORRECT: Using tools first, then providing answer based on results`;
// Add even stronger instructions specifically for Claude Sonnet 4
if (this.isClaudeSonnet4()) {
thinkingModelSection += `
🚨 CRITICAL INSTRUCTIONS FOR CLAUDE SONNET 4 - AUTONOMOUS AGENT MODE 🚨
WARNING: You have a specific tendency to write complete responses immediately after tool calls. This BREAKS the autonomous agent pattern!
🔄 CORRECT AUTONOMOUS AGENT ITERATION PATTERN:
1. User asks question
2. Brief sentence about what you'll do (1 sentence max)
3. Use tools to gather information: <use_tool>...</use_tool>
4. STOP after tool calls - Do not write anything else
5. Wait for tool results (system provides them)
6. Evaluate results and either: Use more tools OR provide final answer
IDEAL RESPONSE FLOW:
- Brief action statement (1 sentence)
- Tool calls
- STOP (wait for results)
- Brief transition statement (1 sentence)
- More tool calls OR final answer
🎯 CORRECT FIRST RESPONSE PATTERN (when tools needed):
I'll search your vault for piano practice information.
<use_tool>
<name>localSearch</name>
<query>piano practice</query>
<salientTerms>["piano", "practice"]</salientTerms>
</use_tool>
🌐 MULTILINGUAL EXAMPLE (PRESERVE ORIGINAL LANGUAGE):
<use_tool>
<name>localSearch</name>
<query></query>
<salientTerms>["ピアノ", "練習", "方法"]</salientTerms>
</use_tool>
<use_tool>
<name>webSearch</name>
<query>piano techniques</query>
<chatHistory>[]</chatHistory>
</use_tool>
[RESPONSE ENDS HERE - NO MORE TEXT]
🎯 CORRECT FOLLOW-UP RESPONSE PATTERN:
Let me gather more specific information about practice schedules.
<use_tool>
<name>localSearch</name>
<query>practice schedule</query>
<salientTerms>["practice", "schedule"]</salientTerms>
</use_tool>
[RESPONSE ENDS HERE - NO MORE TEXT]
WRONG PATTERN (DO NOT DO THIS):
<use_tool>...</use_tool>
Based on the search results, here's a complete practice plan...
[This is FORBIDDEN - you haven't received results yet!]
🔑 KEY UNDERSTANDING FOR CLAUDE 4:
- Brief 1-sentence explanations BEFORE tool calls are good
- Each response is ONE STEP in a multi-step process
- After tool calls, STOP and wait for the system to provide results
- Your thinking is automatically handled in <think> blocks
AUTONOMOUS AGENT RULES FOR CLAUDE 4:
1. If you need tools: Brief sentence + tool calls, then STOP
2. If you receive tool results: Evaluate if you need more tools
3. If you need more tools: Brief sentence + more tool calls, then STOP
4. If you have enough info: THEN provide your final response
REMEMBER: One brief sentence before tools is perfect. Nothing after tool calls.`;
}
return baseSystemPrompt + thinkingModelSection;
}
return baseSystemPrompt;
}
needsSpecialHandling(): boolean {
return this.isThinkingModel();
}
/**
* Detect premature responses in Claude models, especially Claude 4
* Checks for content before tool calls (allowed in small amounts) and after tool calls (not allowed)
* @param response - The model's response text to analyze
* @returns Object indicating if premature content was detected and its type
*/
detectPrematureResponse(response: string): {
hasPremature: boolean;
type: "before" | "after" | null;
} {
const firstToolCallIndex = response.indexOf("<use_tool>");
if (firstToolCallIndex === -1) {
// No tool calls at all, so it's either a final response or a problem
return { hasPremature: false, type: null };
}
// Check content before tool calls - allow brief sentences (1-2 sentences max)
const contentBeforeTools = response.substring(0, firstToolCallIndex).trim();
const contentBeforeWithoutThinking = contentBeforeTools
.replace(/<think>[\s\S]*?<\/think>/g, "")
.trim();
// Allow brief action statements (up to 2 sentences, 200 characters)
const sentences = contentBeforeWithoutThinking
.split(/[.!?]+/)
.filter((s) => s.trim().length > 0);
const BRIEF_STATEMENT_THRESHOLD = 200; // characters
if (sentences.length > 2 || contentBeforeWithoutThinking.length > BRIEF_STATEMENT_THRESHOLD) {
return { hasPremature: true, type: "before" };
}
// Check content after last tool call (main Claude 4 issue)
const lastToolCallEndIndex = response.lastIndexOf("</use_tool>");
if (lastToolCallEndIndex !== -1) {
const contentAfterTools = response
.substring(lastToolCallEndIndex + "</use_tool>".length)
.trim();
// Remove think blocks to analyze only non-thinking content
const contentAfterWithoutThinking = contentAfterTools
.replace(/<think>[\s\S]*?<\/think>/g, "")
.trim();
// Simple threshold: any substantial non-thinking content after tools is premature
const SUBSTANTIAL_CONTENT_THRESHOLD = 100; // characters
if (contentAfterWithoutThinking.length > SUBSTANTIAL_CONTENT_THRESHOLD) {
return { hasPremature: true, type: "after" };
}
}
return { hasPremature: false, type: null };
}
/**
* Sanitize Claude 4 responses by removing premature content after tool calls
* Preserves all think blocks while removing substantial non-thinking content
* @param response - The model's response text to sanitize
* @param iteration - The current iteration number (only sanitizes first iteration)
* @returns The sanitized response text
*/
sanitizeResponse(response: string, iteration: number): string {
if (!this.isClaudeSonnet4() || iteration !== 1) {
return response;
}
const prematureResult = this.detectPrematureResponse(response);
if (!prematureResult.hasPremature) {
return response;
}
if (prematureResult.type === "after") {
// Simple approach: preserve ALL think blocks, remove substantial non-thinking content
const lastToolCallEndIndex = response.lastIndexOf("</use_tool>");
if (lastToolCallEndIndex !== -1) {
const baseResponse = response.substring(0, lastToolCallEndIndex + "</use_tool>".length);
const contentAfterTools = response.substring(lastToolCallEndIndex + "</use_tool>".length);
// Extract and preserve ALL think blocks
const thinkBlockRegex = /<think>[\s\S]*?<\/think>/g;
const thinkBlocks = contentAfterTools.match(thinkBlockRegex) || [];
// Return base response + all think blocks (remove everything else)
return baseResponse + (thinkBlocks.length > 0 ? "\n" + thinkBlocks.join("\n") : "");
}
}
return response;
}
/**
* Check if Claude 4 streaming should be truncated to prevent hallucinated content
* @param partialResponse - The partial response text received during streaming
* @returns True if streaming should be truncated at this point
*/
shouldTruncateStreaming(partialResponse: string): boolean {
if (!this.isClaudeSonnet4()) {
return false;
}
// Check if we have tool calls and substantial non-thinking content after them
const lastToolCallEndIndex = partialResponse.lastIndexOf("</use_tool>");
if (lastToolCallEndIndex === -1) {
return false;
}
const contentAfterTools = partialResponse
.substring(lastToolCallEndIndex + "</use_tool>".length)
.trim();
// Remove think blocks to analyze only non-thinking content
const contentAfterWithoutThinking = contentAfterTools
.replace(/<think>[\s\S]*?<\/think>/g, "")
.trim();
// Simple threshold: if there's substantial non-thinking content, truncate
const STREAMING_TRUNCATE_THRESHOLD = 50;
return contentAfterWithoutThinking.length > STREAMING_TRUNCATE_THRESHOLD;
}
}
/**
* Gemini adapter with aggressive tool calling prompts
*/
class GeminiModelAdapter extends BaseModelAdapter {
enhanceSystemPrompt(
basePrompt: string,
toolDescriptions: string,
availableToolNames?: string[],
toolMetadata?: ToolMetadata[]
): string {
const baseSystemPrompt = super.enhanceSystemPrompt(
basePrompt,
toolDescriptions,
availableToolNames,
toolMetadata
);
// Gemini needs very explicit instructions about tool usage
const tools = availableToolNames || [];
const hasLocalSearch = tools.includes("localSearch");
const geminiSpecificSection = `
🚨 CRITICAL INSTRUCTIONS FOR GEMINI - AUTONOMOUS AGENT MODE 🚨
You MUST use tools to complete tasks. DO NOT ask the user questions about how to proceed.
${
hasLocalSearch
? `
When the user mentions "my notes" or "my vault", use the localSearch tool.
WRONG:
"Let's start by searching your notes. What kind of information should I look for?"
CORRECT:
<use_tool>
<name>localSearch</name>
<query>piano</query>
<salientTerms>["piano"]</salientTerms>
</use_tool>
`
: ""
}
GEMINI SPECIFIC RULES:
1. When user mentions "my notes" about X use localSearch with query "X"
2. DO NOT ask clarifying questions about search terms
3. DO NOT wait for permission to use tools
4. Use tools based on the user's request
PATTERN FOR MULTI-STEP REQUESTS:
User: "based on my project roadmap notes and create summary"
Your response:
<use_tool>
<name>localSearch</name>
<query>project roadmap</query>
<salientTerms>["project", "roadmap"]</salientTerms>
</use_tool>
Remember: The user has already told you what to do. Execute it NOW with the available tools.`;
return baseSystemPrompt + geminiSpecificSection;
}
enhanceUserMessage(message: string, requiresTools: boolean): string {
if (requiresTools) {
// Add explicit reminder for Gemini
return `${message}\n\nREMINDER: Use the tools immediately. Do not ask questions. For "my notes", use localSearch.`;
}
return message;
}
needsSpecialHandling(): boolean {
return true;
}
}
/**
* Factory to create appropriate adapter based on model
*/
export class ModelAdapterFactory {
static createAdapter(model: BaseChatModel): ModelAdapter {
const modelName = ((model as any).modelName || (model as any).model || "").toLowerCase();
logInfo(`Creating model adapter for: ${modelName}`);
// GPT models need special handling
if (modelName.includes("gpt")) {
logInfo("Using GPTModelAdapter");
return new GPTModelAdapter(modelName);
}
// Claude models
if (modelName.includes("claude")) {
logInfo("Using ClaudeModelAdapter");
return new ClaudeModelAdapter(modelName);
}
// Gemini models (check for both "gemini" and "google" prefixes)
if (modelName.includes("gemini") || modelName.includes("google/gemini")) {
logInfo("Using GeminiModelAdapter");
return new GeminiModelAdapter(modelName);
}
// Copilot Plus models
if (modelName.includes("copilot-plus")) {
logInfo("Using BaseModelAdapter for Copilot Plus");
return new BaseModelAdapter(modelName);
}
// Default adapter for unknown models
logInfo("Using BaseModelAdapter (default)");
return new BaseModelAdapter(modelName);
}
}
/**
* Helper to detect if user message likely requires tools
* @param message - The user's message to analyze
* @returns True if the message likely requires tool usage
*/
export function messageRequiresTools(message: string): boolean {
const toolIndicators = [
"find",
"search",
"look for",
"look up",
"my notes",
"in my vault",
"from my vault",
"check the web",
"search online",
"from the internet",
"current time",
"what time",
"timer",
"youtube",
"video",
"transcript",
];
const lowerMessage = message.toLowerCase();
return toolIndicators.some((indicator) => lowerMessage.includes(indicator));
}

View file

@ -0,0 +1,122 @@
export interface ToolCallMarker {
id: string;
toolName: string;
displayName: string;
emoji: string;
confirmationMessage?: string;
isExecuting: boolean;
result?: string;
startIndex: number;
endIndex: number;
}
export interface ParsedMessage {
segments: Array<{
type: "text" | "toolCall";
content: string;
toolCall?: ToolCallMarker;
}>;
}
/**
* Parse tool call markers from a message
* Format: <!--TOOL_CALL_START:id:toolName:displayName:emoji:confirmationMessage:isExecuting-->content<!--TOOL_CALL_END:id:result-->
*/
export function parseToolCallMarkers(message: string): ParsedMessage {
const segments: ParsedMessage["segments"] = [];
// Use [\s\S] instead of . with 's' flag for compatibility with ES6
const toolCallRegex =
/<!--TOOL_CALL_START:([^:]+):([^:]+):([^:]+):([^:]+):([^:]*):([^:]+)-->([\s\S]*?)<!--TOOL_CALL_END:\1:([\s\S]*?)-->/g;
let lastIndex = 0;
let match;
while ((match = toolCallRegex.exec(message)) !== null) {
// Add text before the tool call
if (match.index > lastIndex) {
segments.push({
type: "text",
content: message.slice(lastIndex, match.index),
});
}
// Parse the tool call
const [
fullMatch,
id,
toolName,
displayName,
emoji,
confirmationMessage,
isExecuting,
content,
result,
] = match;
segments.push({
type: "toolCall",
content: content,
toolCall: {
id,
toolName,
displayName,
emoji,
confirmationMessage: confirmationMessage || undefined,
isExecuting: isExecuting === "true",
result: result || undefined,
startIndex: match.index,
endIndex: match.index + fullMatch.length,
},
});
lastIndex = match.index + fullMatch.length;
}
// Add any remaining text
if (lastIndex < message.length) {
segments.push({
type: "text",
content: message.slice(lastIndex),
});
}
// If no tool calls found, return the entire message as text
if (segments.length === 0) {
segments.push({
type: "text",
content: message,
});
}
return { segments };
}
/**
* Create a tool call marker
*/
export function createToolCallMarker(
id: string,
toolName: string,
displayName: string,
emoji: string,
confirmationMessage: string = "",
isExecuting: boolean = true,
content: string = "",
result: string = ""
): string {
return `<!--TOOL_CALL_START:${id}:${toolName}:${displayName}:${emoji}:${confirmationMessage}:${isExecuting}-->${content}<!--TOOL_CALL_END:${id}:${result}-->`;
}
/**
* Update a tool call marker with result
*/
export function updateToolCallMarker(message: string, id: string, result: string): string {
// Escape the id to prevent regex injection
const escapedId = id.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const regex = new RegExp(
`(<!--TOOL_CALL_START:${escapedId}:[^:]+:[^:]+:[^:]+:[^:]*:)true(-->[\\s\\S]*?<!--TOOL_CALL_END:${escapedId}:)[\\s\\S]*?-->`,
"g"
);
return message.replace(regex, `$1false$2${result}-->`);
}

View file

@ -0,0 +1,122 @@
import { executeSequentialToolCall } from "./toolExecution";
import { createTool } from "@/tools/SimpleTool";
import { z } from "zod";
// Mock dependencies
jest.mock("@/plusUtils", () => ({
checkIsPlusUser: jest.fn(),
}));
jest.mock("@/logger", () => ({
logError: jest.fn(),
logInfo: jest.fn(),
logWarn: jest.fn(),
}));
jest.mock("@/tools/toolManager", () => ({
ToolManager: {
callTool: jest.fn(),
},
}));
import { checkIsPlusUser } from "@/plusUtils";
import { ToolManager } from "@/tools/toolManager";
describe("toolExecution", () => {
const mockCheckIsPlusUser = checkIsPlusUser as jest.MockedFunction<typeof checkIsPlusUser>;
const mockCallTool = ToolManager.callTool as jest.MockedFunction<typeof ToolManager.callTool>;
beforeEach(() => {
jest.clearAllMocks();
});
describe("executeSequentialToolCall", () => {
it("should execute tools without isPlusOnly flag", async () => {
const testTool = createTool({
name: "testTool",
description: "Test tool",
schema: z.object({ input: z.string() }),
handler: async ({ input }) => `Result: ${input}`,
});
mockCallTool.mockResolvedValueOnce("Tool executed successfully");
const result = await executeSequentialToolCall(
{ name: "testTool", args: { input: "test" } },
[testTool]
);
expect(result).toEqual({
toolName: "testTool",
result: "Tool executed successfully",
success: true,
});
expect(mockCheckIsPlusUser).not.toHaveBeenCalled();
});
it("should block plus-only tools for non-plus users", async () => {
const plusTool = createTool({
name: "plusTool",
description: "Plus-only tool",
schema: z.void(),
handler: async () => "Should not execute",
isPlusOnly: true,
});
mockCheckIsPlusUser.mockResolvedValueOnce(false);
const result = await executeSequentialToolCall({ name: "plusTool", args: {} }, [plusTool]);
expect(result).toEqual({
toolName: "plusTool",
result: "Error: plusTool requires a Copilot Plus subscription",
success: false,
});
expect(mockCallTool).not.toHaveBeenCalled();
});
it("should allow plus-only tools for plus users", async () => {
const plusTool = createTool({
name: "plusTool",
description: "Plus-only tool",
schema: z.void(),
handler: async () => "Plus tool executed",
isPlusOnly: true,
});
mockCheckIsPlusUser.mockResolvedValueOnce(true);
mockCallTool.mockResolvedValueOnce("Plus tool executed");
const result = await executeSequentialToolCall({ name: "plusTool", args: {} }, [plusTool]);
expect(result).toEqual({
toolName: "plusTool",
result: "Plus tool executed",
success: true,
});
expect(mockCheckIsPlusUser).toHaveBeenCalled();
expect(mockCallTool).toHaveBeenCalled();
});
it("should handle tool not found", async () => {
const result = await executeSequentialToolCall({ name: "unknownTool", args: {} }, []);
expect(result).toEqual({
toolName: "unknownTool",
result:
"Error: Tool 'unknownTool' not found. Available tools: . Make sure you have the tool enabled in the Agent settings.",
success: false,
});
});
it("should handle invalid tool call", async () => {
const result = await executeSequentialToolCall(null as any, []);
expect(result).toEqual({
toolName: "unknown",
result: "Error: Invalid tool call - missing tool name",
success: false,
});
});
});
});

View file

@ -0,0 +1,231 @@
import { logError, logInfo, logWarn } from "@/logger";
import { checkIsPlusUser } from "@/plusUtils";
import { ToolManager } from "@/tools/toolManager";
import { err2String } from "@/utils";
import { ToolCall } from "./xmlParsing";
export interface ToolExecutionResult {
toolName: string;
result: string;
success: boolean;
}
/**
* Executes a single tool call with timeout and error handling
*/
export async function executeSequentialToolCall(
toolCall: ToolCall,
availableTools: any[],
originalUserMessage?: string
): Promise<ToolExecutionResult> {
const DEFAULT_TOOL_TIMEOUT = 30000; // 30 seconds timeout per tool
try {
// Validate tool call
if (!toolCall || !toolCall.name) {
return {
toolName: toolCall?.name || "unknown",
result: "Error: Invalid tool call - missing tool name",
success: false,
};
}
// Find the tool in the existing tool registry
const tool = availableTools.find((t) => t.name === toolCall.name);
if (!tool) {
const availableToolNames = availableTools.map((t) => t.name).join(", ");
return {
toolName: toolCall.name,
result: `Error: Tool '${toolCall.name}' not found. Available tools: ${availableToolNames}. Make sure you have the tool enabled in the Agent settings.`,
success: false,
};
}
// Check if tool requires Plus subscription
if (tool.isPlusOnly) {
const isPlusUser = await checkIsPlusUser();
if (!isPlusUser) {
return {
toolName: toolCall.name,
result: `Error: ${getToolDisplayName(toolCall.name)} requires a Copilot Plus subscription`,
success: false,
};
}
}
// Prepare tool arguments
const toolArgs = { ...toolCall.args };
// If tool requires user message content and it's provided, inject it
if (tool.requiresUserMessageContent && originalUserMessage) {
toolArgs._userMessageContent = originalUserMessage;
}
// Determine timeout for this tool
let timeout = DEFAULT_TOOL_TIMEOUT;
if (typeof tool.timeoutMs === "number") {
timeout = tool.timeoutMs;
}
let result;
if (!timeout || timeout === Infinity) {
// No timeout for this tool
result = await ToolManager.callTool(tool, toolArgs);
} else {
// Use timeout
result = await Promise.race([
ToolManager.callTool(tool, toolArgs),
new Promise((_, reject) =>
setTimeout(
() => reject(new Error(`Tool execution timed out after ${timeout}ms`)),
timeout
)
),
]);
}
// Validate result
if (result === null || result === undefined) {
logWarn(`Tool ${toolCall.name} returned null/undefined result`);
return {
toolName: toolCall.name,
result: "Tool executed but returned no result",
success: true,
};
}
return {
toolName: toolCall.name,
result: typeof result === "string" ? result : JSON.stringify(result),
success: true,
};
} catch (error) {
logError(`Error executing tool ${toolCall.name}:`, error);
return {
toolName: toolCall.name,
result: `Error: ${err2String(error)}`,
success: false,
};
}
}
/**
* Get display name for tool (user-friendly version)
*/
export function getToolDisplayName(toolName: string): string {
const displayNameMap: Record<string, string> = {
localSearch: "vault search",
webSearch: "web search",
getFileTree: "file tree",
getCurrentTime: "current time",
getTimeRangeMs: "time range",
getTimeInfoByEpoch: "time info",
convertTimeBetweenTimezones: "timezone converter",
startPomodoro: "pomodoro timer",
pomodoroTool: "pomodoro timer",
simpleYoutubeTranscriptionTool: "YouTube transcription",
youtubeTranscription: "YouTube transcription",
indexVault: "vault indexing",
indexTool: "index",
writeToFile: "file editor",
replaceInFile: "file editor",
};
return displayNameMap[toolName] || toolName;
}
/**
* Get emoji for tool display
*/
export function getToolEmoji(toolName: string): string {
const emojiMap: Record<string, string> = {
localSearch: "🔍",
webSearch: "🌐",
getFileTree: "📁",
getCurrentTime: "🕒",
getTimeRangeMs: "📅",
getTimeInfoByEpoch: "🕰️",
convertTimeBetweenTimezones: "🌍",
startPomodoro: "⏱️",
pomodoroTool: "⏱️",
simpleYoutubeTranscriptionTool: "📺",
youtubeTranscription: "📺",
indexVault: "📚",
indexTool: "📚",
writeToFile: "✏️",
replaceInFile: "🔄",
};
return emojiMap[toolName] || "🔧";
}
/**
* Get user confirmation message for tool call
*/
export function getToolConfirmtionMessage(toolName: string): string | null {
if (toolName == "writeToFile" || toolName == "replaceInFile") {
return "Accept / reject in the Preview";
}
return null;
}
/**
* Log tool call details for debugging
*/
export function logToolCall(toolCall: ToolCall, iteration: number): void {
const displayName = getToolDisplayName(toolCall.name);
const emoji = getToolEmoji(toolCall.name);
// Create clean parameter display
const paramDisplay =
Object.keys(toolCall.args).length > 0
? JSON.stringify(toolCall.args, null, 2)
: "(no parameters)";
logInfo(`${emoji} [Iteration ${iteration}] ${displayName.toUpperCase()}`);
logInfo(`Parameters:`, paramDisplay);
logInfo("---");
}
/**
* Log tool execution result
*/
export function logToolResult(toolName: string, result: ToolExecutionResult): void {
const displayName = getToolDisplayName(toolName);
const emoji = getToolEmoji(toolName);
const status = result.success ? "✅ SUCCESS" : "❌ FAILED";
logInfo(`${emoji} ${displayName.toUpperCase()} RESULT: ${status}`);
// Log abbreviated result for readability
if (result.result.length > 500) {
logInfo(
`Result: ${result.result.substring(0, 500)}... (truncated, ${result.result.length} chars total)`
);
} else {
logInfo(`Result:`, result.result);
}
logInfo("---");
}
/**
* Deduplicate sources by path, keeping highest score
* If path is not available, falls back to title
*/
export function deduplicateSources(
sources: { title: string; path: string; score: number }[]
): { title: string; path: string; score: number }[] {
const uniqueSources = new Map<string, { title: string; path: string; score: number }>();
for (const source of sources) {
// Use path as the unique key, falling back to title if path is not available
const key = source.path || source.title;
const existing = uniqueSources.get(key);
if (!existing || source.score > existing.score) {
uniqueSources.set(key, source);
}
}
return Array.from(uniqueSources.values()).sort((a, b) => b.score - a.score);
}

View file

@ -0,0 +1,514 @@
import { parseXMLToolCalls, escapeXml, escapeXmlAttribute, stripToolCallXML } from "./xmlParsing";
describe("parseXMLToolCalls", () => {
it("should parse hybrid XML tool calls with JSON arrays", () => {
const text = `
I'll search your vault for piano learning notes.
<use_tool>
<name>localSearch</name>
<query>piano learning</query>
<salientTerms>["piano", "learning", "practice", "music"]</salientTerms>
</use_tool>
Let me also search the web.
<use_tool>
<name>webSearch</name>
<query>piano techniques</query>
<chatHistory>[]</chatHistory>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(2);
// First tool call
expect(toolCalls[0].name).toBe("localSearch");
expect(toolCalls[0].args.query).toBe("piano learning");
expect(toolCalls[0].args.salientTerms).toEqual(["piano", "learning", "practice", "music"]);
// Second tool call
expect(toolCalls[1].name).toBe("webSearch");
expect(toolCalls[1].args.query).toBe("piano techniques");
expect(toolCalls[1].args.chatHistory).toEqual([]); // JSON array format
});
it("should handle tool calls with no parameters", () => {
const text = `
<use_tool>
<name>getFileTree</name>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("getFileTree");
expect(toolCalls[0].args).toEqual({});
});
it("should handle string parameters without JSON parsing", () => {
const text = `
<use_tool>
<name>simpleYoutubeTranscription</name>
<url>https://youtube.com/watch?v=123</url>
<language>en</language>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("simpleYoutubeTranscription");
expect(toolCalls[0].args.url).toBe("https://youtube.com/watch?v=123");
expect(toolCalls[0].args.language).toBe("en");
});
it("should handle hybrid approach with both JSON and XML formats", () => {
const text = `
<use_tool>
<name>hybridTool</name>
<stringParam>simple string</stringParam>
<jsonArray>["item1", "item2"]</jsonArray>
<jsonObject>{"key": "value", "number": 42}</jsonObject>
<xmlArray>
<item>xml1</item>
<item>xml2</item>
</xmlArray>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("hybridTool");
expect(toolCalls[0].args.stringParam).toBe("simple string");
expect(toolCalls[0].args.jsonArray).toEqual(["item1", "item2"]); // JSON parsed
expect(toolCalls[0].args.jsonObject).toEqual({ key: "value", number: 42 }); // JSON parsed
expect(toolCalls[0].args.xmlArray).toEqual(["xml1", "xml2"]); // XML parsed
});
it("should handle nested item tags", () => {
const text = `
<use_tool>
<name>nestedTool</name>
<simpleParam>working string</simpleParam>
<nestedArray>
<item>
<title>Item 1</title>
<value>Value 1</value>
</item>
<item>
<title>Item 2</title>
<value>Value 2</value>
</item>
</nestedArray>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("nestedTool");
expect(toolCalls[0].args.simpleParam).toBe("working string");
expect(toolCalls[0].args.nestedArray).toEqual([
{ title: "Item 1", value: "Value 1" },
{ title: "Item 2", value: "Value 2" },
]);
});
it("should ignore tool calls with empty names", () => {
const text = `
<use_tool>
<name></name>
<param>value</param>
</use_tool>
<use_tool>
<name>validTool</name>
<param>value</param>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("validTool");
});
it("should handle whitespace and formatting", () => {
const text = `
<use_tool>
<name> localSearch </name>
<query> search query </query>
<salientTerms> ["term1", "term2"] </salientTerms>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("localSearch");
expect(toolCalls[0].args.query).toBe("search query");
expect(toolCalls[0].args.salientTerms).toEqual(["term1", "term2"]);
});
it("should return empty array for text with no tool calls", () => {
const text = "This is just regular text with no tool calls.";
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(0);
});
it("should handle webSearch tool with hybrid chatHistory formats", () => {
const text = `
<use_tool>
<name>webSearch</name>
<query>piano practice tips</query>
<chatHistory>[]</chatHistory>
</use_tool>
<use_tool>
<name>webSearch</name>
<query>follow-up search</query>
<chatHistory>
<item>
<role>user</role>
<content>Tell me about piano</content>
</item>
<item>
<role>assistant</role>
<content>Piano is a musical instrument</content>
</item>
</chatHistory>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(2);
// First tool call - JSON empty array
expect(toolCalls[0].name).toBe("webSearch");
expect(toolCalls[0].args.query).toBe("piano practice tips");
expect(toolCalls[0].args.chatHistory).toEqual([]);
// Second tool call - XML item format
expect(toolCalls[1].name).toBe("webSearch");
expect(toolCalls[1].args.query).toBe("follow-up search");
expect(toolCalls[1].args.chatHistory).toEqual([
{ role: "user", content: "Tell me about piano" },
{ role: "assistant", content: "Piano is a musical instrument" },
]);
});
it("should handle malformed JSON gracefully", () => {
const text = `
<use_tool>
<name>testTool</name>
<goodParam>working string</goodParam>
<badJson>[invalid json</badJson>
<goodJson>["valid", "array"]</goodJson>
</use_tool>
`;
const toolCalls = parseXMLToolCalls(text);
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe("testTool");
expect(toolCalls[0].args.goodParam).toBe("working string");
expect(toolCalls[0].args.badJson).toBe("[invalid json"); // Falls back to string
expect(toolCalls[0].args.goodJson).toEqual(["valid", "array"]); // JSON parsed
});
});
describe("XML Escaping Utilities", () => {
describe("escapeXml", () => {
it("should escape ampersands", () => {
expect(escapeXml("foo & bar")).toBe("foo &amp; bar");
});
it("should escape less than signs", () => {
expect(escapeXml("foo < bar")).toBe("foo &lt; bar");
});
it("should escape greater than signs", () => {
expect(escapeXml("foo > bar")).toBe("foo &gt; bar");
});
it("should escape double quotes", () => {
expect(escapeXml('foo "bar" baz')).toBe("foo &quot;bar&quot; baz");
});
it("should escape single quotes", () => {
expect(escapeXml("foo 'bar' baz")).toBe("foo &apos;bar&apos; baz");
});
it("should escape multiple special characters", () => {
expect(escapeXml('<tag attr="value">content & more</tag>')).toBe(
"&lt;tag attr=&quot;value&quot;&gt;content &amp; more&lt;/tag&gt;"
);
});
it("should handle empty strings", () => {
expect(escapeXml("")).toBe("");
});
it("should handle strings with no special characters", () => {
expect(escapeXml("hello world")).toBe("hello world");
});
it("should handle non-string inputs", () => {
expect(escapeXml(null as any)).toBe("");
expect(escapeXml(undefined as any)).toBe("");
expect(escapeXml(123 as any)).toBe("");
});
it("should escape XML entity references", () => {
expect(escapeXml("&lt;&gt;&quot;&apos;&amp;")).toBe(
"&amp;lt;&amp;gt;&amp;quot;&amp;apos;&amp;amp;"
);
});
});
describe("escapeXmlAttribute", () => {
it("should escape attribute values the same as escapeXml", () => {
const testString = '<tag attr="value">content & more</tag>';
expect(escapeXmlAttribute(testString)).toBe(escapeXml(testString));
});
it("should handle variable names with special characters", () => {
expect(escapeXmlAttribute('my"variable')).toBe("my&quot;variable");
expect(escapeXmlAttribute("my'variable")).toBe("my&apos;variable");
expect(escapeXmlAttribute("my<variable>")).toBe("my&lt;variable&gt;");
});
});
});
describe("stripToolCallXML", () => {
describe("complete tool calls", () => {
it("should remove complete tool call blocks", () => {
const text = `Before tool call.
<use_tool>
<name>localSearch</name>
<query>test query</query>
</use_tool>
After tool call.`;
const result = stripToolCallXML(text);
expect(result).toBe("Before tool call.\n\nAfter tool call.");
});
it("should remove multiple complete tool call blocks", () => {
const text = `First text.
<use_tool>
<name>localSearch</name>
<query>first search</query>
</use_tool>
Middle text.
<use_tool>
<name>webSearch</name>
<query>second search</query>
</use_tool>
Final text.`;
const result = stripToolCallXML(text);
expect(result).toBe("First text.\n\nMiddle text.\n\nFinal text.");
});
});
describe("partial tool calls", () => {
it("should show calling message for partial tool call at end of text", () => {
const text = `Some text before.
<use_tool>
<name>localSearch</name>
<query>incomplete`;
const result = stripToolCallXML(text);
expect(result).toBe("Some text before.\n\nCalling vault search...");
});
it("should show generic calling message for partial tool call with only opening tag", () => {
const text = `Some text before.
<use_tool>`;
const result = stripToolCallXML(text);
expect(result).toBe("Some text before.\n\nCalling tool...");
});
it("should show calling message for partial tool call with incomplete parameters", () => {
const text = `Some text before.
<use_tool>
<name>webSearch</name>
<query>incomplete query
<someParam>value`;
const result = stripToolCallXML(text);
expect(result).toBe("Some text before.\n\nCalling web search...");
});
it("should handle mixed complete and partial tool calls", () => {
const text = `Start text.
<use_tool>
<name>localSearch</name>
<query>complete search</query>
</use_tool>
Middle text.
<use_tool>
<name>webSearch</name>
<query>incomplete`;
const result = stripToolCallXML(text);
expect(result).toBe("Start text.\n\nMiddle text.\n\nCalling web search...");
});
it("should show calling message for partial tool call in middle when followed by text", () => {
const text = `Before text.
<use_tool>
<name>localSearch</name>
<query>incomplete query without closing
This text should remain.`;
const result = stripToolCallXML(text);
expect(result).toBe("Before text.\n\nCalling vault search...");
});
it("should show generic calling message when tool name is not yet available", () => {
const text = `Some text before.
<use_tool>
<name>`;
const result = stripToolCallXML(text);
expect(result).toBe("Some text before.\n\nCalling tool...");
});
});
describe("edge cases", () => {
it("should handle text with no tool calls", () => {
const text = "Just regular text with no tool calls.";
const result = stripToolCallXML(text);
expect(result).toBe("Just regular text with no tool calls.");
});
it("should handle empty string", () => {
const result = stripToolCallXML("");
expect(result).toBe("");
});
it("should handle text with just whitespace", () => {
const result = stripToolCallXML(" \n \n ");
expect(result).toBe("");
});
it("should handle multiple partial tool calls", () => {
const text = `Text before.
<use_tool>
<name>localSearch</name>
More text.
<use_tool>
<name>webSearch</name>
<param>value`;
const result = stripToolCallXML(text);
expect(result).toBe("Text before.\n\nCalling vault search...");
});
it("should preserve non-tool XML tags", () => {
const text = `Some text with <strong>bold</strong> and <em>italic</em> tags.
<use_tool>
<name>localSearch</name>
<query>search</query>
</use_tool>
More <div>HTML-like</div> content.`;
const result = stripToolCallXML(text);
expect(result).toBe(
"Some text with <strong>bold</strong> and <em>italic</em> tags.\n\nMore <div>HTML-like</div> content."
);
});
it("should handle tool calls with complex nested content", () => {
const text = `Before tool.
<use_tool>
<name>complexTool</name>
<nested>
<item>value1</item>
<item>value2</item>
</nested>
<jsonParam>["item1", "item2"]</jsonParam>
</use_tool>
After tool.`;
const result = stripToolCallXML(text);
expect(result).toBe("Before tool.\n\nAfter tool.");
});
});
describe("code block removal", () => {
it("should remove empty code blocks", () => {
const text = `Some text.
\`\`\`
\`\`\`
More text.
\`\`\`javascript
\`\`\``;
const result = stripToolCallXML(text);
expect(result).toBe("Some text.\n\nMore text.");
});
it("should remove tool_code blocks", () => {
const text = `Some text.
\`\`\`tool_code
some tool code here
\`\`\`
More text.`;
const result = stripToolCallXML(text);
expect(result).toBe("Some text.\n\nMore text.");
});
it("should clean up excessive whitespace", () => {
const text = `Text with
multiple
newlines.`;
const result = stripToolCallXML(text);
expect(result).toBe("Text with\n\nmultiple\n\nnewlines.");
});
});
});

View file

@ -0,0 +1,208 @@
import { logError, logWarn } from "@/logger";
import { getToolDisplayName } from "./toolExecution";
/**
* Escapes special XML characters in a string to prevent XML injection
* @param str - The string to escape
* @returns The escaped string safe for XML content
*/
export function escapeXml(str: string): string {
if (typeof str !== "string") {
return "";
}
return str
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&apos;");
}
/**
* Escapes special XML characters for use in XML attributes
* @param str - The string to escape for attribute use
* @returns The escaped string safe for XML attributes
*/
export function escapeXmlAttribute(str: string): string {
return escapeXml(str);
}
export interface ToolCall {
name: string;
args: any;
}
/**
* Parses XML tool call blocks from AI responses using pure XML format
* Format: <use_tool><name>toolName</name><param1>value1</param1><arrayParam><item>val1</item><item>val2</item></arrayParam></use_tool>
*/
export function parseXMLToolCalls(text: string): ToolCall[] {
const toolCalls: ToolCall[] = [];
try {
const regex = /<use_tool>([\s\S]*?)<\/use_tool>/g;
let match;
while ((match = regex.exec(text)) !== null) {
const content = match[1];
const nameMatch = content.match(/<name>([\s\S]*?)<\/name>/);
if (nameMatch) {
const name = nameMatch[1].trim();
// Validate tool name
if (!name || name.length === 0) {
logWarn("Skipping tool call with empty name");
continue;
}
// Parse individual XML parameters using pure XML approach
const args: any = {};
// Remove the name tag from content to avoid matching it
const contentWithoutName = content.replace(/<name>[\s\S]*?<\/name>/, "");
// Find all parameter tags
const paramRegex = /<([^>]+)>([\s\S]*?)<\/\1>/g;
let paramMatch;
while ((paramMatch = paramRegex.exec(contentWithoutName)) !== null) {
const paramName = paramMatch[1].trim();
const paramContent = paramMatch[2].trim();
// Skip empty parameter names
if (!paramName) continue;
// Parse parameter content as pure XML
args[paramName] = parseParameterContent(paramContent, paramName);
}
toolCalls.push({ name, args });
}
}
} catch (error) {
logError("Error parsing XML tool calls:", error);
// Return empty array if parsing fails completely
return [];
}
return toolCalls;
}
/**
* Parses parameter content using hybrid approach
* - JSON arrays/objects: ["item1", "item2"] or {"key": "value"}
* - Pure XML arrays: <item>value1</item><item>value2</item>
* - Pure XML objects: <key1>value1</key1><key2>value2</key2>
* - Simple strings: just the text value
* - Empty content: returns appropriate empty value based on parameter name
*/
function parseParameterContent(content: string, parameterName?: string): any {
if (!content) {
// Special handling for known array parameters that should default to empty arrays
if (parameterName === "chatHistory" || parameterName === "salientTerms") {
return [];
}
return "";
}
// Check if content contains XML tags
const hasXmlTags = /<[^>]+>/.test(content);
if (!hasXmlTags) {
// Try to parse as JSON if it looks like a JSON array/object
if (
(content.startsWith("[") && content.endsWith("]")) ||
(content.startsWith("{") && content.endsWith("}"))
) {
try {
return JSON.parse(content);
} catch {
// If JSON parsing fails, use as string
return content;
}
}
// Simple string value
return content;
}
// Check if it's an array format with <item> tags
const itemMatches = content.match(/<item>([\s\S]*?)<\/item>/g);
if (itemMatches) {
return itemMatches.map((match) => {
const itemContent = match.replace(/<\/?item>/g, "").trim();
return parseParameterContent(itemContent); // Recursive for nested structures
});
}
// Check if it's an object format with key-value pairs
const objectRegex = /<([^>]+)>([\s\S]*?)<\/\1>/g;
const objectEntries: [string, any][] = [];
let objectMatch;
while ((objectMatch = objectRegex.exec(content)) !== null) {
const key = objectMatch[1].trim();
const value = objectMatch[2].trim();
objectEntries.push([key, parseParameterContent(value)]);
}
if (objectEntries.length > 0) {
return Object.fromEntries(objectEntries);
}
// Fallback to string if we can't parse as structured data
return content;
}
/**
* Extracts tool name from a partial tool call block
*/
function extractToolNameFromPartialBlock(partialContent: string): string | null {
const nameMatch = partialContent.match(/<name>([\s\S]*?)<\/name>/);
if (nameMatch) {
const name = nameMatch[1].trim();
return name || null;
}
return null;
}
/**
* Strips XML tool call blocks, thinking blocks, and various code blocks from text.
* @param text - The text to clean
* @returns The cleaned text
*/
export function stripToolCallXML(text: string): string {
let cleaned = text;
// Remove all complete <use_tool>...</use_tool> blocks
cleaned = cleaned.replace(/<use_tool>[\s\S]*?<\/use_tool>/g, "");
// Replace partial tool calls with calling message
cleaned = cleaned.replace(/<use_tool>([\s\S]*)$/g, (match, partialContent) => {
const toolName = extractToolNameFromPartialBlock(partialContent);
if (toolName) {
const displayName = getToolDisplayName(toolName);
return `Calling ${displayName}...`;
} else {
return `Calling tool...`;
}
});
// Keep thinking blocks in autonomous agent mode as they provide valuable context
// They are only removed in other contexts where they add noise
// Remove empty code blocks that might appear
cleaned = cleaned.replace(/```\w*\s*```/g, "");
// Remove tool_code blocks (both empty and with content)
cleaned = cleaned.replace(/```tool_code[\s\S]*?```/g, "");
// Remove any remaining empty code blocks with various languages
cleaned = cleaned.replace(/```[\w]*[\s\n]*```/g, "");
// Clean up excessive whitespace and trim
cleaned = cleaned.replace(/\n\s*\n\s*\n/g, "\n\n").trim();
return cleaned;
}

View file

@ -140,7 +140,7 @@ export default class ChatModelManager {
fetch: customModel.enableCors ? safeFetch : undefined,
},
...(isThinkingEnabled && {
thinking: { type: "enabled", budget_tokens: 1024 },
thinking: { type: "enabled", budget_tokens: 2048 },
}),
},
[ChatModelProviders.AZURE_OPENAI]: {

View file

@ -3,6 +3,7 @@ import { isProjectMode } from "@/aiParams";
import { createGetFileTreeTool } from "@/tools/FileTreeTools";
import { indexTool, localSearchTool, webSearchTool } from "@/tools/SearchTools";
import {
convertTimeBetweenTimezonesTool,
getCurrentTimeTool,
getTimeInfoByEpochTool,
getTimeRangeMsTool,
@ -30,6 +31,7 @@ export class IntentAnalyzer {
if (this.tools.length === 0) {
this.tools = [
getCurrentTimeTool,
convertTimeBetweenTimezonesTool,
getTimeInfoByEpochTool,
getTimeRangeMsTool,
localSearchTool,
@ -104,7 +106,7 @@ export class IntentAnalyzer {
const { timeRange, salientTerms } = context;
// Handle @vault command
if (message.includes("@vault") && (salientTerms.length > 0 || timeRange)) {
if (message.includes("@vault")) {
// Remove all @commands from the query
const cleanQuery = this.removeAtCommands(originalMessage);

View file

@ -569,7 +569,9 @@ ${contextParts.join("\n\n")}
// Note: We're only processing markdown files here, other file types
// are handled by FileParserManager and stored in the file cache
const files = this.app.vault.getFiles().filter((file) => {
return file.extension === "md" && shouldIndexFile(file, inclusionPatterns, exclusionPatterns);
return (
file.extension === "md" && shouldIndexFile(file, inclusionPatterns, exclusionPatterns, true)
);
});
logInfo(`Found ${files.length} markdown files to process for project context`);

View file

@ -99,7 +99,7 @@ export class ProjectContextCache {
isProject: true,
});
if (shouldIndexFile(file, inclusions, exclusions)) {
if (shouldIndexFile(file, inclusions, exclusions, true)) {
// Only invalidate markdown context, keep other contexts
await this.invalidateMarkdownContext(project);
logInfo(
@ -525,7 +525,7 @@ export class ProjectContextCache {
const file = this.vault.getAbstractFileByPath(filePath);
// If file no longer exists or doesn't match patterns, remove its reference
if (!(file instanceof TFile) || !shouldIndexFile(file, inclusions, exclusions)) {
if (!(file instanceof TFile) || !shouldIndexFile(file, inclusions, exclusions, true)) {
// Note: We don't remove from fileCache to preserve content for future use
removedCount++;
} else {
@ -571,7 +571,7 @@ export class ProjectContextCache {
let addedCount = 0;
for (const file of allFiles) {
if (shouldIndexFile(file, inclusions, exclusions)) {
if (shouldIndexFile(file, inclusions, exclusions, true)) {
if (contextCacheToUpdate.fileContexts[file.path]) {
continue;
}

View file

@ -21,7 +21,7 @@ import { CustomCommand } from "@/commands/type";
import { useSettingsValue } from "@/settings/model";
// Custom hook for managing chat chain
function useChatChain(selectedModel: CustomModel) {
function useChatChain(selectedModel: CustomModel, systemPrompt?: string) {
const [chatMemory] = useState<BaseChatMemory>(
new BufferMemory({ returnMessages: true, memoryKey: "history" })
);
@ -32,10 +32,10 @@ function useChatChain(selectedModel: CustomModel) {
async function initChatChain() {
const chatModel = await ChatModelManager.getInstance().createModelInstance(selectedModel);
const defaultSystemPrompt =
"You are a helpful assistant. You'll help the user with their content editing needs.";
const chatPrompt = ChatPromptTemplate.fromMessages([
SystemMessagePromptTemplate.fromTemplate(
"You are a helpful assistant. You'll help the user with their content editing needs."
),
SystemMessagePromptTemplate.fromTemplate(systemPrompt || defaultSystemPrompt),
new MessagesPlaceholder("history"),
HumanMessagePromptTemplate.fromTemplate("{input}"),
]);
@ -57,7 +57,7 @@ function useChatChain(selectedModel: CustomModel) {
}
initChatChain();
}, [selectedModel, chatMemory]);
}, [selectedModel, chatMemory, systemPrompt]);
return { chatChain, chatMemory };
}
@ -67,6 +67,7 @@ interface CustomCommandChatModalContentProps {
command: CustomCommand;
onInsert: (message: string) => void;
onReplace: (message: string) => void;
systemPrompt?: string;
}
function CustomCommandChatModalContent({
@ -74,6 +75,7 @@ function CustomCommandChatModalContent({
command,
onInsert,
onReplace,
systemPrompt,
}: CustomCommandChatModalContentProps) {
const [aiCurrentMessage, setAiCurrentMessage] = useState<string | null>(null);
const [processedMessage, setProcessedMessage] = useState<string | null>(null);
@ -88,7 +90,7 @@ function CustomCommandChatModalContent({
[command.modelKey, modelKey, settings.activeModels]
);
const { chatChain, chatMemory } = useChatChain(selectedModel);
const { chatChain, chatMemory } = useChatChain(selectedModel, systemPrompt);
const commandTitle = command.title;
@ -298,12 +300,16 @@ function CustomCommandChatModalContent({
<div className="tw-flex tw-gap-2">
{generating ? (
// When generating, show Stop button
<Button variant="secondary" onClick={handleStopGeneration}>
<Button size="sm" variant="secondary" onClick={handleStopGeneration}>
Stop
</Button>
) : showFollowupSubmit ? (
// When follow-up instruction has content, show Submit button with Enter shortcut
<Button onClick={handleFollowupSubmit} className="tw-flex tw-items-center tw-gap-1">
<Button
size="sm"
onClick={handleFollowupSubmit}
className="tw-flex tw-items-center tw-gap-1"
>
<span>Submit</span>
<CornerDownLeft className="tw-size-3" />
</Button>
@ -311,11 +317,12 @@ function CustomCommandChatModalContent({
// Otherwise, show Insert and Replace buttons with shortcut indicators
<>
<Button
size="sm"
onClick={() => onInsert(processedMessage ?? "")}
className="tw-flex tw-items-center tw-gap-1"
>
<span>Insert</span>
<div className="tw-flex tw-items-center tw-text-xs tw-text-normal">
<div className="tw-flex tw-items-center tw-text-xs">
{Platform.isMacOS ? (
<>
<Command className="tw-size-3" />
@ -332,11 +339,12 @@ function CustomCommandChatModalContent({
</div>
</Button>
<Button
size="sm"
onClick={() => onReplace(processedMessage ?? "")}
className="tw-flex tw-items-center tw-gap-1"
>
<span>Replace</span>
<div className="tw-flex tw-items-center tw-text-xs tw-text-normal">
<div className="tw-flex tw-items-center tw-text-xs">
{Platform.isMacOS ? (
<>
<Command className="tw-size-3" />
@ -366,6 +374,7 @@ export class CustomCommandChatModal extends Modal {
private configs: {
selectedText: string;
command: CustomCommand;
systemPrompt?: string;
}
) {
super(app);
@ -374,7 +383,7 @@ export class CustomCommandChatModal extends Modal {
onOpen() {
const { contentEl } = this;
this.root = createRoot(contentEl);
const { selectedText, command } = this.configs;
const { selectedText, command, systemPrompt } = this.configs;
const handleInsert = (message: string) => {
insertIntoEditor(message);
@ -392,6 +401,7 @@ export class CustomCommandChatModal extends Modal {
command={command}
onInsert={handleInsert}
onReplace={handleReplace}
systemPrompt={systemPrompt}
/>
);
}

View file

@ -2,6 +2,7 @@ import { CustomCommand } from "@/commands/type";
export const LEGACY_SELECTED_TEXT_PLACEHOLDER = "{copilot-selection}";
export const COMMAND_NAME_MAX_LENGTH = 50;
export const QUICK_COMMAND_CODE_BLOCK = "copilotquickcommand";
export const EMPTY_COMMAND: CustomCommand = {
title: "",
content: "",

View file

@ -3,31 +3,51 @@ import { getCachedCustomCommands } from "@/commands/state";
import { COMMAND_IDS } from "@/constants";
import { Menu } from "obsidian";
import { CustomCommand } from "./type";
import { isLivePreviewModeOn } from "@/utils";
export function registerContextMenu(menu: Menu) {
// Create the main "Copilot" submenu
menu.addItem((item) => {
item.setTitle("Copilot: Add selection to chat context").onClick(() => {
(app as any).commands.executeCommandById(
`copilot:${COMMAND_IDS.ADD_SELECTION_TO_CHAT_CONTEXT}`
);
item.setTitle("Copilot");
(item as any).setSubmenu();
const submenu = (item as any).submenu;
if (!submenu) return;
// Add the main selection command
submenu.addItem((subItem: any) => {
subItem.setTitle("Add selection to chat context").onClick(() => {
(app as any).commands.executeCommandById(
`copilot:${COMMAND_IDS.ADD_SELECTION_TO_CHAT_CONTEXT}`
);
});
});
});
// Add separator if there are custom commands too
const commands = getCachedCustomCommands();
const visibleCustomCommands = commands.filter(
(command: CustomCommand) => command.showInContextMenu
);
if (visibleCustomCommands.length > 0) {
menu.addSeparator();
}
if (isLivePreviewModeOn()) {
submenu.addItem((subItem: any) => {
subItem.setTitle("Trigger quick command").onClick(() => {
(app as any).commands.executeCommandById(`copilot:${COMMAND_IDS.TRIGGER_QUICK_COMMAND}`);
});
});
}
sortCommandsByOrder(
commands.filter((command: CustomCommand) => command.showInContextMenu)
).forEach((command: CustomCommand) => {
menu.addItem((item) => {
item.setTitle(`Copilot: ${command.title}`).onClick(() => {
(app as any).commands.executeCommandById(`copilot:${getCommandId(command.title)}`);
// Get custom commands
const commands = getCachedCustomCommands();
const visibleCustomCommands = commands.filter(
(command: CustomCommand) => command.showInContextMenu
);
// Add separator if there are custom commands
if (visibleCustomCommands.length > 0) {
submenu.addSeparator();
}
// Add custom commands to submenu
sortCommandsByOrder(visibleCustomCommands).forEach((command: CustomCommand) => {
submenu.addItem((subItem: any) => {
subItem.setTitle(command.title).onClick(() => {
(app as any).commands.executeCommandById(`copilot:${getCommandId(command.title)}`);
});
});
});
});

View file

@ -18,7 +18,7 @@ import {
updateCachedCommand,
} from "@/commands/state";
import { CustomCommandManager } from "@/commands/customCommandManager";
import { logError, logInfo } from "@/logger";
import { logError } from "@/logger";
/** This manager is used to register custom commands as obsidian commands */
export class CustomCommandRegister {
@ -69,7 +69,6 @@ export class CustomCommandRegister {
return;
}
const customCommand = await parseCustomCommandFile(file);
logInfo("command file modified", file.path, customCommand);
this.registerCommand(customCommand);
updateCachedCommand(customCommand, customCommand.title);
},
@ -88,7 +87,6 @@ export class CustomCommandRegister {
return;
}
try {
logInfo("new command file created", file.path);
let customCommand = await parseCustomCommandFile(file);
if (!hasOrderFrontmatter(file)) {
// Compute the correct order for the new command
@ -125,7 +123,6 @@ export class CustomCommandRegister {
}
// Register the new command if it's still a custom command file
if (isCustomCommandFile(file)) {
logInfo("command file renamed", file.path);
const parsedCommand = await parseCustomCommandFile(file);
this.registerCommand(parsedCommand);
updateCachedCommand(parsedCommand, parsedCommand.title);

View file

@ -49,8 +49,15 @@ describe("processedPrompt()", () => {
// Set default implementations for critical mocks
(extractNoteFiles as jest.Mock).mockReturnValue([]);
// Create mock objects
mockVault = {} as Vault;
// Create mock objects with adapter.stat
mockVault = {
adapter: {
stat: jest.fn().mockResolvedValue({
ctime: Date.now(),
mtime: Date.now(),
}),
},
} as unknown as Vault;
mockActiveNote = {
path: "path/to/active/note.md",
basename: "Active Note",
@ -81,7 +88,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"This is a {variable} and {selectedText}.\n\nselectedText:\n\nhere is some selected text 12345\n\nvariable:\n\n## Variable Note\n\nhere is the note content for note0"
'This is a {variable} and {selected_text}.\n\n<selected_text>\nhere is some selected text 12345\n</selected_text>\n\n<variable name="variable">\n<variable_note>\n## Variable Note\n\nhere is the note content for note0\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockActiveNote);
});
@ -114,7 +121,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"This is a {variable1} and {variable2}.\n\nvariable1:\n\n## Variable1 Note\n\nhere is the note content for note0\n\nvariable2:\n\n## Variable2 Note\n\nnote content for note1"
'This is a {variable1} and {variable2}.\n\n<variable name="variable1">\n<variable_note>\n## Variable1 Note\n\nhere is the note content for note0\n</variable_note>\n</variable>\n\n<variable name="variable2">\n<variable_note>\n## Variable2 Note\n\nnote content for note1\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockNote1);
expect(result.includedFiles).toContain(mockNote2);
@ -135,7 +142,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Rewrite the following text {selectedText}\n\nselectedText:\n\nhere is some selected text 12345"
"Rewrite the following text {selected_text}\n\n<selected_text>\nhere is some selected text 12345\n</selected_text>"
);
expect(result.includedFiles).toEqual([]);
});
@ -159,7 +166,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"This is the active note: {activenote}\n\nactivenote:\n\n## Active Note\n\nContent of the active note"
'This is the active note: {activenote}\n\n<variable name="activenote">\n<variable_note>\n## Active Note\n\nContent of the active note\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockActiveNote);
expect(getFileContent).toHaveBeenCalledWith(mockActiveNote, mockVault);
@ -225,7 +232,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Notes related to {#tag} are:\n\n#tag:\n\n## Tagged Note\n\nNote content for #tag"
'Notes related to {#tag} are:\n\n<variable name="#tag">\n<variable_note>\n## Tagged Note\n\nNote content for #tag\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockNoteForTag);
});
@ -264,7 +271,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Notes related to {#tag1,#tag2,#tag3} are:\n\n#tag1,#tag2,#tag3:\n\n## Tagged Note 1\n\nNote content for #tag1\n\n## Tagged Note 2\n\nNote content for #tag2"
'Notes related to {#tag1,#tag2,#tag3} are:\n\n<variable name="#tag1,#tag2,#tag3">\n<variable_note>\n## Tagged Note 1\n\nNote content for #tag1\n</variable_note>\n\n<variable_note>\n## Tagged Note 2\n\nNote content for #tag2\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockNoteForTag1);
expect(result.includedFiles).toContain(mockNoteForTag2);
@ -282,9 +289,11 @@ describe("processedPrompt()", () => {
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toContain("Content of [[Test Note]] is important");
expect(result.processedPrompt).toContain(
"Title: [[Test Note]]\nPath: Test Note.md\n\nTest note content"
);
expect(result.processedPrompt).toContain("<note_context>");
expect(result.processedPrompt).toContain("<title>Test Note</title>");
expect(result.processedPrompt).toContain("<path>Test Note.md</path>");
expect(result.processedPrompt).toContain("Test note content");
expect(result.processedPrompt).toContain("</note_context>");
expect(result.includedFiles).toContain(mockTestNote);
});
@ -311,7 +320,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Content of {[[Test Note]]} is important. Look at [[Test Note]].\n\n[[Test Note]]:\n\n## Test Note\n\nTest note content"
'Content of {[[Test Note]]} is important. Look at [[Test Note]].\n\n<variable name="[[Test Note]]">\n<variable_note>\n## Test Note\n\nTest note content\n</variable_note>\n</variable>'
);
// Note: extractNoteFiles will still find [[Test Note]], but processPrompt should skip adding it again because it's already in includedFiles from the variable processing
expect(result.includedFiles).toEqual([mockNoteFile]);
@ -356,7 +365,11 @@ describe("processedPrompt()", () => {
"{[[Note1]]} content and [[Note2]] are both important"
);
expect(result.processedPrompt).toContain("## Note1\n\nNote1 content");
expect(result.processedPrompt).toContain("Title: [[Note2]]\nPath: Note2.md\n\nNote2 content");
expect(result.processedPrompt).toContain("<note_context>");
expect(result.processedPrompt).toContain("<title>Note2</title>");
expect(result.processedPrompt).toContain("<path>Note2.md</path>");
expect(result.processedPrompt).toContain("Note2 content");
expect(result.processedPrompt).toContain("</note_context>");
// Note2 is added via [[Note2]] processing
expect(result.includedFiles).toEqual(expect.arrayContaining([mockNote1, mockNote2]));
expect(result.includedFiles.length).toBe(2);
@ -385,9 +398,17 @@ describe("processedPrompt()", () => {
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toContain("[[Note1]] is related to [[Note2]] and [[Note3]].");
expect(result.processedPrompt).toContain("Title: [[Note1]]\nPath: Note1.md\n\nNote1 content");
expect(result.processedPrompt).toContain("Title: [[Note2]]\nPath: Note2.md\n\nNote2 content");
expect(result.processedPrompt).toContain("Title: [[Note3]]\nPath: Note3.md\n\nNote3 content");
// All notes should be in note_context format
expect(result.processedPrompt).toContain("<note_context>");
expect(result.processedPrompt).toContain("<title>Note1</title>");
expect(result.processedPrompt).toContain("<path>Note1.md</path>");
expect(result.processedPrompt).toContain("Note1 content");
expect(result.processedPrompt).toContain("<title>Note2</title>");
expect(result.processedPrompt).toContain("<path>Note2.md</path>");
expect(result.processedPrompt).toContain("Note2 content");
expect(result.processedPrompt).toContain("<title>Note3</title>");
expect(result.processedPrompt).toContain("<path>Note3.md</path>");
expect(result.processedPrompt).toContain("Note3 content");
expect(result.includedFiles).toEqual(expect.arrayContaining([mockNote1, mockNote2, mockNote3]));
expect(result.includedFiles.length).toBe(3);
});
@ -428,7 +449,7 @@ describe("processedPrompt()", () => {
// Check that getFileContent was called with the active note at least once
expect(getFileContent).toHaveBeenCalledWith(mockActiveNote, mockVault);
expect(result.processedPrompt).toBe(
"This is the active note: {activeNote}. And again: {activeNote}\n\nactiveNote:\n\n## Active Note\n\nContent of the active note"
'This is the active note: {activeNote}. And again: {activeNote}\n\n<variable name="activeNote">\n<variable_note>\n## Active Note\n\nContent of the active note\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockActiveNote);
});
@ -450,7 +471,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Summarize this: {selectedText}\n\nselectedText (entire active note):\n\nContent of the active note"
'Summarize this: {selected_text}\n\n<selected_text type="active_note">\nContent of the active note\n</selected_text>'
);
// Active note should be included because of {}
expect(result.includedFiles).toContain(mockActiveNote);
@ -476,7 +497,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Summarize this: {selectedText}. Additional info: {activeNote}\n\nselectedText (entire active note):\n\nContent of the active note"
'Summarize this: {selected_text}. Additional info: {activeNote}\n\n<selected_text type="active_note">\nContent of the active note\n</selected_text>'
);
// Ensure getFileContent was called for the {} replacement
expect(getFileContent).toHaveBeenCalledWith(mockActiveNote, mockVault);
@ -501,7 +522,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"Analyze this: {selectedText}\n\nselectedText:\n\nThis is the selected text"
"Analyze this: {selected_text}\n\n<selected_text>\nThis is the selected text\n</selected_text>"
);
// Active note should not be included when selected text is present for {}
expect(result.includedFiles).toEqual([]);
@ -539,7 +560,7 @@ describe("processedPrompt()", () => {
const result = await processPrompt(doc.content, selectedText, mockVault, mockActiveNote);
expect(result.processedPrompt).toBe(
"This is a test prompt with {invalidVariable} name and {activeNote}\n\nactiveNote:\n\n## Active Note\n\nActive Note Content"
'This is a test prompt with {invalidVariable} name and {activeNote}\n\n<variable name="activeNote">\n<variable_note>\n## Active Note\n\nActive Note Content\n</variable_note>\n</variable>'
);
expect(result.includedFiles).toContain(mockActiveNote);
// Expect the warning for the invalid variable

View file

@ -6,9 +6,10 @@ import {
COPILOT_COMMAND_SLASH_ENABLED,
EMPTY_COMMAND,
LEGACY_SELECTED_TEXT_PLACEHOLDER,
QUICK_COMMAND_CODE_BLOCK,
} from "@/commands/constants";
import { CustomCommand } from "@/commands/type";
import { normalizePath, Notice, TAbstractFile, TFile, Vault } from "obsidian";
import { normalizePath, Notice, TAbstractFile, TFile, Vault, Editor } from "obsidian";
import { getSettings } from "@/settings/model";
import {
updateCachedCommands,
@ -25,7 +26,12 @@ import {
getNotesFromTags,
processVariableNameForNotePath,
} from "@/utils";
import { NOTE_CONTEXT_PROMPT_TAG } from "@/constants";
import {
NOTE_CONTEXT_PROMPT_TAG,
SELECTED_TEXT_TAG,
VARIABLE_TAG,
VARIABLE_NOTE_TAG,
} from "@/constants";
export function validateCommandName(
name: string,
@ -195,8 +201,8 @@ export async function processCommandPrompt(
const processedPrompt = result.processedPrompt;
if (processedPrompt.includes("{selectedText}") || skipAppendingSelectedText) {
// Containing {selectedText} means the prompt was using the custom prompt
if (processedPrompt.includes(`{${SELECTED_TEXT_TAG}}`) || skipAppendingSelectedText) {
// Containing {selected_text} means the prompt was using the custom prompt
// processor way of handling the selected text. No need to go through the
// legacy placeholder.
return processedPrompt;
@ -211,7 +217,16 @@ export async function processCommandPrompt(
// `{copilot-selection}` is found, append the selected text to the prompt.
const index = processedPrompt.indexOf(LEGACY_SELECTED_TEXT_PLACEHOLDER);
if (index === -1 && selectedText.trim()) {
return processedPrompt + "\n\n<selectedText>" + selectedText + "</selectedText>";
return (
processedPrompt +
"\n\n<" +
SELECTED_TEXT_TAG +
">" +
selectedText +
"</" +
SELECTED_TEXT_TAG +
">"
);
}
return (
processedPrompt.slice(0, index) +
@ -257,7 +272,7 @@ async function extractVariablesFromPrompt(
if (activeNote) {
const content = await getFileContent(activeNote, vault);
if (content) {
variableResult.content = `## ${getFileName(activeNote)}\n\n${content}`;
variableResult.content = `<${VARIABLE_NOTE_TAG}>\n## ${getFileName(activeNote)}\n\n${content}\n</${VARIABLE_NOTE_TAG}>`;
variableResult.files.push(activeNote);
}
} else {
@ -274,7 +289,9 @@ async function extractVariablesFromPrompt(
for (const file of noteFiles) {
const content = await getFileContent(file, vault);
if (content) {
notesContent.push(`## ${getFileName(file)}\n\n${content}`);
notesContent.push(
`<${VARIABLE_NOTE_TAG}>\n## ${getFileName(file)}\n\n${content}\n</${VARIABLE_NOTE_TAG}>`
);
variableResult.files.push(file);
}
}
@ -286,7 +303,9 @@ async function extractVariablesFromPrompt(
for (const file of noteFiles) {
const content = await getFileContent(file, vault);
if (content) {
notesContent.push(`## ${getFileName(file)}\n\n${content}`);
notesContent.push(
`<${VARIABLE_NOTE_TAG}>\n## ${getFileName(file)}\n\n${content}\n</${VARIABLE_NOTE_TAG}>`
);
variableResult.files.push(file);
}
}
@ -353,16 +372,16 @@ export async function processPrompt(
let activeNoteContent: string | null = null;
if (processedPrompt.includes("{}")) {
processedPrompt = processedPrompt.replace(/\{\}/g, "{selectedText}");
processedPrompt = processedPrompt.replace(/\{\}/g, `{${SELECTED_TEXT_TAG}}`);
if (selectedText) {
additionalInfo += `selectedText:\n\n${selectedText}`;
additionalInfo += `<${SELECTED_TEXT_TAG}>\n${selectedText}\n</${SELECTED_TEXT_TAG}>`;
// Note: selectedText doesn't directly correspond to a file inclusion here
} else if (activeNote) {
activeNoteContent = await getFileContent(activeNote, vault);
additionalInfo += `selectedText (entire active note):\n\n${activeNoteContent}`;
additionalInfo += `<${SELECTED_TEXT_TAG} type="active_note">\n${activeNoteContent || ""}\n</${SELECTED_TEXT_TAG}>`;
includedFiles.add(activeNote); // Ensure active note is tracked if used for {}
} else {
additionalInfo += `selectedText:\n\n(No selected text or active note available)`;
additionalInfo += `<${SELECTED_TEXT_TAG}>\n(No selected text or active note available)\n</${SELECTED_TEXT_TAG}>`;
}
}
@ -374,9 +393,9 @@ export async function processPrompt(
continue;
}
if (additionalInfo) {
additionalInfo += `\n\n${varName}:\n\n${content}`;
additionalInfo += `\n\n<${VARIABLE_TAG} name="${varName}">\n${content}\n</${VARIABLE_TAG}>`;
} else {
additionalInfo += `${varName}:\n\n${content}`;
additionalInfo += `<${VARIABLE_TAG} name="${varName}">\n${content}\n</${VARIABLE_TAG}>`;
}
}
@ -388,7 +407,12 @@ export async function processPrompt(
if (!includedFiles.has(noteFile)) {
const noteContent = await getFileContent(noteFile, vault);
if (noteContent) {
const noteContext = `<${NOTE_CONTEXT_PROMPT_TAG}> \n Title: [[${noteFile.basename}]]\nPath: ${noteFile.path}\n\n${noteContent}\n</${NOTE_CONTEXT_PROMPT_TAG}>`;
// Get file metadata
const stats = await vault.adapter.stat(noteFile.path);
const ctime = stats ? new Date(stats.ctime).toISOString() : "Unknown";
const mtime = stats ? new Date(stats.mtime).toISOString() : "Unknown";
const noteContext = `<${NOTE_CONTEXT_PROMPT_TAG}>\n<title>${noteFile.basename}</title>\n<path>${noteFile.path}</path>\n<ctime>${ctime}</ctime>\n<mtime>${mtime}</mtime>\n<content>\n${noteContent}\n</content>\n</${NOTE_CONTEXT_PROMPT_TAG}>`;
if (additionalInfo) {
additionalInfo += `\n\n`;
}
@ -468,3 +492,68 @@ export async function ensureCommandFrontmatter(file: TFile, command: CustomComma
removePendingFileWrite(file.path);
}
}
/**
* Removes all quick command code blocks from the editor while preserving cursor position and selection
* @param editor - The Obsidian editor instance
* @returns true if any blocks were removed, false otherwise
*/
export function removeQuickCommandBlocks(editor: Editor): boolean {
// Store original selection positions
const originalFrom = editor.getCursor("from");
const originalTo = editor.getCursor("to");
const content = editor.getValue();
const lines = content.split("\n");
let hasExisting = false;
const newLines = [];
let removedLinesBeforeFrom = 0;
let removedLinesBeforeTo = 0;
let i = 0;
while (i < lines.length) {
if (lines[i].trim() === `\`\`\`${QUICK_COMMAND_CODE_BLOCK}`) {
hasExisting = true;
const blockStartLine = i;
// Skip the opening line
i++;
// Skip until we find the closing ```
while (i < lines.length && lines[i].trim() !== "```") {
i++;
}
// Skip the closing line
i++;
const removedLineCount = i - blockStartLine;
// Calculate how many lines were removed before the selection positions
if (blockStartLine <= originalFrom.line) {
removedLinesBeforeFrom += removedLineCount;
}
if (blockStartLine <= originalTo.line) {
removedLinesBeforeTo += removedLineCount;
}
} else {
newLines.push(lines[i]);
i++;
}
}
// Update editor content and restore selection if we removed existing blocks
if (hasExisting) {
editor.setValue(newLines.join("\n"));
// Calculate new selection positions accounting for removed lines
const newFromLine = Math.max(0, originalFrom.line - removedLinesBeforeFrom);
const newToLine = Math.max(0, originalTo.line - removedLinesBeforeTo);
// Restore the selection
editor.setSelection(
{ line: newFromLine, ch: originalFrom.ch },
{ line: newToLine, ch: originalTo.ch }
);
}
return hasExisting;
}

View file

@ -0,0 +1,148 @@
import { processPrompt } from "@/commands/customCommandUtils";
import { TFile, Vault } from "obsidian";
import { getFileContent, getFileName, getNotesFromPath } from "@/utils";
// Mock the dependencies
jest.mock("@/utils", () => ({
extractNoteFiles: jest.fn().mockReturnValue([]),
getFileContent: jest.fn(),
getFileName: jest.fn(),
getNotesFromPath: jest.fn(),
getNotesFromTags: jest.fn(),
processVariableNameForNotePath: jest.fn(),
}));
jest.mock("@/settings/model", () => ({
getSettings: jest.fn(() => ({
debug: false,
enableCustomPromptTemplating: true,
})),
}));
describe("XML Escaping in processPrompt", () => {
let mockVault: Vault;
let mockActiveNote: TFile;
beforeEach(() => {
jest.clearAllMocks();
mockVault = {
adapter: {
stat: jest.fn().mockResolvedValue({
ctime: Date.now(),
mtime: Date.now(),
}),
},
} as unknown as Vault;
mockActiveNote = {
path: "path/to/active/note.md",
basename: "Active Note",
} as TFile;
});
it("should NOT escape XML special characters in selected text", async () => {
const customPrompt = "Process this: {}";
const selectedText = "<tag>content & \"quotes\" 'apostrophes'</tag>";
const result = await processPrompt(customPrompt, selectedText, mockVault, mockActiveNote);
// Should contain the original unescaped text
expect(result.processedPrompt).toContain(selectedText);
// Should NOT contain escaped characters
expect(result.processedPrompt).not.toContain("&lt;");
expect(result.processedPrompt).not.toContain("&amp;");
expect(result.processedPrompt).not.toContain("&quot;");
expect(result.processedPrompt).not.toContain("&apos;");
});
it("should NOT escape XML in variable names", async () => {
const customPrompt = 'Use {my"variable<>}';
const mockNote = {
basename: 'Note with <special> & "chars"',
path: "special.md",
} as TFile;
(getNotesFromPath as jest.Mock).mockResolvedValue([mockNote]);
(getFileName as jest.Mock).mockReturnValue(mockNote.basename);
(getFileContent as jest.Mock).mockResolvedValue('Content with <xml> & special "chars"');
const result = await processPrompt(customPrompt, "", mockVault, mockActiveNote);
// Check variable name is NOT escaped in attribute
expect(result.processedPrompt).toContain('name="my"variable<>"');
// Check note title is NOT escaped
expect(result.processedPrompt).toContain('Note with <special> & "chars"');
// Check content is NOT escaped
expect(result.processedPrompt).toContain('Content with <xml> & special "chars"');
});
it("should NOT escape XML in active note content", async () => {
const customPrompt = "Process {activeNote}";
mockActiveNote.basename = "Note <with> \"XML\" & 'special' chars";
(getFileContent as jest.Mock).mockResolvedValue(
'Content: <script>alert("xss")</script> & more'
);
(getFileName as jest.Mock).mockReturnValue(mockActiveNote.basename);
const result = await processPrompt(customPrompt, "", mockVault, mockActiveNote);
// Check basename is NOT escaped
expect(result.processedPrompt).toContain("Note <with> \"XML\" & 'special' chars");
// Check content is NOT escaped
expect(result.processedPrompt).toContain('Content: <script>alert("xss")</script> & more');
});
it("should NOT escape XML in note paths and metadata", async () => {
const customPrompt = "[[Special Note]]";
const mockNote = {
basename: 'Special & "Note"',
path: 'folder<with>/special&chars/"note".md',
} as TFile;
(getNotesFromPath as jest.Mock).mockResolvedValue([]);
jest.requireMock("@/utils").extractNoteFiles.mockReturnValue([mockNote]);
(getFileContent as jest.Mock).mockResolvedValue("Content with & and < and >");
const result = await processPrompt(customPrompt, "", mockVault, mockActiveNote);
// Check title is NOT escaped
expect(result.processedPrompt).toContain('<title>Special & "Note"</title>');
// Check path is NOT escaped
expect(result.processedPrompt).toContain('folder<with>/special&chars/"note".md');
// Check content is NOT escaped
expect(result.processedPrompt).toContain("Content with & and < and >");
});
it("should NOT escape XML in tag variables", async () => {
const customPrompt = "Notes for {#tag&special}";
const mockNote = {
basename: 'Tagged & "Note"',
path: "tagged.md",
} as TFile;
(getNotesFromPath as jest.Mock).mockResolvedValue([]);
jest.requireMock("@/utils").getNotesFromTags.mockResolvedValue([mockNote]);
(getFileName as jest.Mock).mockReturnValue(mockNote.basename);
(getFileContent as jest.Mock).mockResolvedValue("Content: <tag> & </tag>");
const result = await processPrompt(customPrompt, "", mockVault, mockActiveNote);
// Check tag variable name is NOT escaped
expect(result.processedPrompt).toContain('name="#tag&special"');
// Check content is NOT escaped
expect(result.processedPrompt).toContain('Tagged & "Note"');
expect(result.processedPrompt).toContain("Content: <tag> & </tag>");
});
});

View file

@ -2,7 +2,7 @@ import { addSelectedTextContext, getChainType } from "@/aiParams";
import { FileCache } from "@/cache/fileCache";
import { ProjectContextCache } from "@/cache/projectContextCache";
import { ChainType } from "@/chainFactory";
import { AdhocPromptModal } from "@/components/modals/AdhocPromptModal";
import { DebugSearchModal } from "@/components/modals/DebugSearchModal";
import { OramaSearchModal } from "@/components/modals/OramaSearchModal";
import { RemoveFromIndexModal } from "@/components/modals/RemoveFromIndexModal";
@ -10,13 +10,17 @@ import CopilotPlugin from "@/main";
import { getAllQAMarkdownContent } from "@/search/searchUtils";
import { CopilotSettings, getSettings, updateSetting } from "@/settings/model";
import { SelectedTextContext } from "@/types/message";
import { Editor, Notice, TFile } from "obsidian";
import { Editor, Notice, TFile, MarkdownView } from "obsidian";
import { v4 as uuidv4 } from "uuid";
import { COMMAND_IDS, COMMAND_NAMES, CommandId } from "../constants";
import { CustomCommandSettingsModal } from "@/commands/CustomCommandSettingsModal";
import { EMPTY_COMMAND } from "@/commands/constants";
import { getCachedCustomCommands } from "@/commands/state";
import { CustomCommandManager } from "@/commands/customCommandManager";
import { QUICK_COMMAND_CODE_BLOCK } from "@/commands/constants";
import { removeQuickCommandBlocks } from "@/commands/customCommandUtils";
import { isLivePreviewModeOn } from "@/utils";
import { ApplyCustomCommandModal } from "@/components/modals/ApplyCustomCommandModal";
/**
* Add a command to the plugin.
@ -98,17 +102,45 @@ export function registerCommands(
plugin.newChat();
});
addCommand(plugin, COMMAND_IDS.APPLY_ADHOC_PROMPT, async () => {
const modal = new AdhocPromptModal(plugin.app, async (adhocPrompt: string) => {
try {
plugin.processCustomPrompt(COMMAND_IDS.APPLY_ADHOC_PROMPT, adhocPrompt);
} catch (err) {
console.error(err);
new Notice("An error occurred.");
}
});
addCheckCommand(plugin, COMMAND_IDS.TRIGGER_QUICK_COMMAND, (checking: boolean) => {
const activeView = plugin.app.workspace.getActiveViewOfType(MarkdownView);
modal.open();
if (checking) {
// Return true only if we're in live preview mode
return !!(isLivePreviewModeOn() && activeView && activeView.editor);
}
// Need to check this again because it can still be triggered via shortcut.
if (!isLivePreviewModeOn()) {
new Notice("Quick commands are only available in live preview mode.");
return false;
}
// When not checking, execute the command
if (!activeView || !activeView.editor) {
new Notice("No active editor found.");
return false;
}
const editor = activeView.editor;
const selectedText = editor.getSelection();
if (!selectedText.trim()) {
new Notice("Please select some text first. Selected text is required for quick commands.");
return false;
}
removeQuickCommandBlocks(editor);
// Get the current cursor/selection position (after potential content update)
const cursor = editor.getCursor("from");
const line = cursor.line;
// Insert the quick command code block above the selected text
const codeBlock = `\`\`\`${QUICK_COMMAND_CODE_BLOCK}\n\`\`\`\n`;
editor.replaceRange(codeBlock, { line, ch: 0 });
return true;
});
addCommand(plugin, COMMAND_IDS.CLEAR_LOCAL_COPILOT_INDEX, async () => {
@ -359,4 +391,10 @@ export function registerCommands(
);
modal.open();
});
// Add command to apply a custom command
addCommand(plugin, COMMAND_IDS.APPLY_CUSTOM_COMMAND, () => {
const modal = new ApplyCustomCommandModal(plugin.app);
modal.open();
});
}

View file

@ -9,13 +9,13 @@ import {
useSelectedTextContexts,
} from "@/aiParams";
import { ChainType } from "@/chainFactory";
import { processPrompt } from "@/commands/customCommandUtils";
import { ChatControls, reloadCurrentProject } from "@/components/chat-components/ChatControls";
import ChatInput from "@/components/chat-components/ChatInput";
import ChatMessages from "@/components/chat-components/ChatMessages";
import { NewVersionBanner } from "@/components/chat-components/NewVersionBanner";
import { ProjectList } from "@/components/chat-components/ProjectList";
import { ABORT_REASON, COMMAND_IDS, EVENT_NAMES, LOADING_MESSAGES, USER_SENDER } from "@/constants";
import { ABORT_REASON, EVENT_NAMES, LOADING_MESSAGES, USER_SENDER } from "@/constants";
import { AppContext, EventTargetContext } from "@/context";
import { useChatManager } from "@/hooks/useChatManager";
import { getAIResponse } from "@/langchainStream";
@ -23,12 +23,7 @@ import ChainManager from "@/LLMProviders/chainManager";
import CopilotPlugin from "@/main";
import { Mention } from "@/mentions/Mention";
import { useIsPlusUser } from "@/plusUtils";
import {
getComposerOutputPrompt,
getSettings,
updateSetting,
useSettingsValue,
} from "@/settings/model";
import { updateSetting, useSettingsValue } from "@/settings/model";
import { ChatUIState } from "@/state/ChatUIState";
import { FileParserManager } from "@/tools/FileParserManager";
import { err2String } from "@/utils";
@ -141,11 +136,6 @@ const Chat: React.FC<ChatProps> = ({
// Handle composer prompt
let displayText = inputMessage;
const composerPrompt = await getComposerOutputPrompt();
if (inputMessage.includes("@composer") && composerPrompt !== "") {
displayText =
inputMessage + "\n\n<output_format>\n" + composerPrompt + "\n</output_format>";
}
// Add tool calls if present
if (toolCalls) {
@ -170,7 +160,8 @@ const Chat: React.FC<ChatProps> = ({
displayText,
context,
currentChain,
includeActiveNote
includeActiveNote,
content.length > 0 ? content : undefined
);
// Add to user message history
@ -253,6 +244,8 @@ const Chat: React.FC<ChatProps> = ({
return;
}
// Clear current AI message and set loading state
setCurrentAiMessage("");
setLoading(true);
try {
const success = await chatUIState.regenerateMessage(
@ -355,76 +348,6 @@ const Chat: React.FC<ChatProps> = ({
]
);
const createEffect = (
eventType: string,
promptFn: (selectedText: string, eventSubtype?: string) => string | Promise<string>
) => {
return () => {
const debug = getSettings().debug;
const handleSelection = async (event: CustomEvent) => {
const messageWithPrompt = await promptFn(
event.detail.selectedText,
event.detail.eventSubtype
);
// Send the prompt message through ChatManager (simplified approach)
try {
const messageId = await chatUIState.sendMessage(
messageWithPrompt,
{ notes: [], urls: [], selectedTextContexts: [] },
currentChain,
includeActiveNote
);
// Get the LLM message for AI processing
const llmMessage = chatUIState.getLLMMessage(messageId);
if (llmMessage) {
setLoading(true);
await getAIResponse(
llmMessage,
chainManager,
addMessage,
setCurrentAiMessage,
setAbortController,
{
debug,
ignoreSystemMessage: true,
}
);
setLoading(false);
}
} catch (error) {
console.error("Error processing adhoc prompt:", error);
setLoading(false);
}
};
eventTarget?.addEventListener(eventType, handleSelection);
// Cleanup function to remove the event listener when the component unmounts
return () => {
eventTarget?.removeEventListener(eventType, handleSelection);
};
};
};
// eslint-disable-next-line react-hooks/exhaustive-deps
useEffect(
createEffect(COMMAND_IDS.APPLY_ADHOC_PROMPT, async (selectedText, customPrompt) => {
if (!customPrompt) {
return selectedText;
}
const result = await processPrompt(
customPrompt,
selectedText,
app.vault,
app.workspace.getActiveFile()
);
return result.processedPrompt; // Extract just the processed prompt string
}),
[]
);
// Expose handleSaveAsNote to parent
useEffect(() => {
if (onSaveChat) {

View file

@ -0,0 +1,199 @@
import React, { useState, useEffect, useRef } from "react";
import { Notice, MarkdownView } from "obsidian";
import { createRoot } from "react-dom/client";
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { Checkbox } from "@/components/ui/checkbox";
import { ModelSelector } from "@/components/ui/ModelSelector";
import { CustomCommandChatModal } from "@/commands/CustomCommandChatModal";
import { useModelKey } from "@/aiParams";
import { CustomCommand } from "@/commands/type";
import { removeQuickCommandBlocks } from "@/commands/customCommandUtils";
import CopilotPlugin from "@/main";
interface QuickCommandProps {
plugin: CopilotPlugin;
onRemove: () => void;
}
export function QuickCommand({ plugin, onRemove }: QuickCommandProps) {
const [prompt, setPrompt] = useState("");
const [includeActiveNote, setIncludeActiveNote] = useState(true);
const [selectedText, setSelectedText] = useState("");
const [globalModelKey] = useModelKey();
const [selectedModelKey, setSelectedModelKey] = useState(globalModelKey);
const textareaRef = useRef<HTMLTextAreaElement>(null);
// Get the currently selected text from the editor
useEffect(() => {
const activeView = plugin.app.workspace.getActiveViewOfType(MarkdownView);
if (activeView && activeView.editor) {
const currentSelection = activeView.editor.getSelection();
setSelectedText(currentSelection);
}
}, [plugin.app]);
// Auto-focus textarea on mount
useEffect(() => {
if (textareaRef.current) {
textareaRef.current.focus();
}
}, []);
const handleSubmit = async () => {
if (!prompt.trim()) {
new Notice("Please enter a prompt");
return;
}
const systemPrompt = `
You are an AI assistant designed to execute user instructions with precision. Your responses should be:
- Direct and focused: Address only what is explicitly requested
- Concise: Avoid unnecessary elaboration unless the user asks for details
- Context-aware: When text is selected or highlighted, treat it as the primary target for any requested action
- Action-oriented: Prioritize completing the task over explaining the process
Key principles:
- Follow instructions literally and completely
- Assume selected/highlighted text is the focus unless told otherwise
- Use all provided context: Consider any additional information, examples, or constraints the user provides to better complete the task
- Add explanations only when explicitly requested or when clarification is essential
- Maintain the user's preferred format and style
Response format: Match the format implied by the user's request (e.g., if they ask for a list, provide a list; if they ask for a rewrite, provide only the rewritten text).
`;
let userContent = prompt;
if (includeActiveNote) {
// Check if placeholders already exist to avoid duplication
const hasSelectedTextPlaceholder = userContent.includes("{}");
const hasActiveNotePlaceholder = /\{activenote\}/i.test(userContent);
// Only append placeholders that don't already exist
const placeholdersToAdd = [];
if (!hasSelectedTextPlaceholder) {
placeholdersToAdd.push("{}");
}
if (!hasActiveNotePlaceholder) {
placeholdersToAdd.push("{activeNote}");
}
if (placeholdersToAdd.length > 0) {
userContent += `\n\n${placeholdersToAdd.join("\n\n")}`;
}
}
const quickCommand: CustomCommand = {
title: "Quick Command",
content: userContent,
showInContextMenu: false,
showInSlashMenu: false,
order: 0,
modelKey: selectedModelKey,
lastUsedMs: Date.now(),
};
const modal = new CustomCommandChatModal(plugin.app, {
selectedText,
command: quickCommand,
systemPrompt,
});
modal.open();
onRemove();
};
const handleCancel = () => {
onRemove();
};
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === "Escape") {
e.preventDefault();
handleCancel();
} else if (e.key === "Enter" && !e.shiftKey) {
e.preventDefault();
handleSubmit();
}
};
return (
<div
className="tw-rounded-lg tw-border tw-border-solid tw-border-border tw-bg-primary tw-p-4"
onKeyDown={handleKeyDown}
>
<div className="tw-space-y-4">
<Textarea
ref={textareaRef}
value={prompt}
onChange={(e) => setPrompt(e.target.value)}
placeholder="Ask me anything..."
className="tw-min-h-24 tw-resize-none"
rows={3}
/>
<div className="tw-flex tw-items-center tw-justify-between tw-gap-4">
<div className="tw-flex tw-items-center tw-gap-4">
<ModelSelector
size="sm"
variant="ghost"
value={selectedModelKey}
onChange={setSelectedModelKey}
/>
<div className="tw-flex tw-items-center tw-gap-2">
<Checkbox
id="includeActiveNote"
checked={includeActiveNote}
onCheckedChange={(checked) => setIncludeActiveNote(!!checked)}
/>
<label
htmlFor="includeActiveNote"
className="tw-cursor-pointer tw-text-sm tw-text-muted"
>
Include note context
</label>
</div>
</div>
<div className="tw-flex tw-items-center tw-gap-2">
<Button variant="secondary" onClick={handleCancel} size="sm">
Cancel
</Button>
<Button onClick={handleSubmit} size="sm">
Submit
</Button>
</div>
</div>
</div>
</div>
);
}
export interface QuickCommandContainerProps {
plugin: CopilotPlugin;
element: HTMLElement;
}
export function createQuickCommandContainer({ plugin, element }: QuickCommandContainerProps) {
const container = document.createElement("div");
element.appendChild(container);
const root = createRoot(container);
const handleRemove = () => {
const activeView = plugin.app.workspace.getActiveViewOfType(MarkdownView);
if (activeView && activeView.editor) {
removeQuickCommandBlocks(activeView.editor);
}
root.unmount();
container.remove();
};
root.render(<QuickCommand plugin={plugin} onRemove={handleRemove} />);
return { root, container };
}

View file

@ -7,42 +7,36 @@ import {
useProjectLoading,
} from "@/aiParams";
import { ChainType } from "@/chainFactory";
import { CustomCommandManager } from "@/commands/customCommandManager";
import { sortSlashCommands } from "@/commands/customCommandUtils";
import { getCachedCustomCommands } from "@/commands/state";
import { AddContextNoteModal } from "@/components/modals/AddContextNoteModal";
import { AddImageModal } from "@/components/modals/AddImageModal";
import { ListPromptModal } from "@/components/modals/ListPromptModal";
import { Button } from "@/components/ui/button";
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { ModelDisplay } from "@/components/ui/model-display";
import { ModelSelector } from "@/components/ui/ModelSelector";
import { ContextProcessor } from "@/contextProcessor";
import { CustomCommandManager } from "@/commands/customCommandManager";
import { cn } from "@/lib/utils";
import { COPILOT_TOOL_NAMES } from "@/LLMProviders/intentAnalyzer";
import { Mention } from "@/mentions/Mention";
import { getModelKeyFromModel, useSettingsValue } from "@/settings/model";
import { updateSetting, useSettingsValue } from "@/settings/model";
import { SelectedTextContext } from "@/types/message";
import { getToolDescription } from "@/tools/toolManager";
import { extractNoteFiles, isAllowedFileForContext, isNoteTitleUnique } from "@/utils";
import {
checkModelApiKey,
err2String,
extractNoteFiles,
isAllowedFileForContext,
isNoteTitleUnique,
} from "@/utils";
import {
ArrowBigUp,
ChevronDown,
Command,
CornerDownLeft,
Database,
Globe,
Image,
Loader2,
StopCircle,
X,
Pen,
Sparkles,
Brain,
} from "lucide-react";
import { App, Notice, Platform, TFile } from "obsidian";
import { App, Platform, TFile } from "obsidian";
import React, {
forwardRef,
useCallback,
@ -54,8 +48,6 @@ import React, {
} from "react";
import { useDropzone } from "react-dropzone";
import ContextControl from "./ContextControl";
import { getCachedCustomCommands } from "@/commands/state";
import { sortSlashCommands } from "@/commands/customCommandUtils";
interface ChatInputProps {
inputMessage: string;
@ -104,22 +96,28 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
},
ref
) => {
const [isModelDropdownOpen, setIsModelDropdownOpen] = useState(false);
const [contextUrls, setContextUrls] = useState<string[]>([]);
const textAreaRef = useRef<HTMLTextAreaElement>(null);
const containerRef = useRef<HTMLDivElement>(null);
const [currentModelKey, setCurrentModelKey] = useModelKey();
const [modelError, setModelError] = useState<string | null>(null);
const [currentChain] = useChainType();
const [isProjectLoading] = useProjectLoading();
const settings = useSettingsValue();
const [currentActiveNote, setCurrentActiveNote] = useState<TFile | null>(() => {
const activeFile = app.workspace.getActiveFile();
return isAllowedFileForContext(activeFile) ? activeFile : null;
});
const [selectedProject, setSelectedProject] = useState<ProjectConfig | null>(null);
const settings = useSettingsValue();
const isCopilotPlus =
currentChain === ChainType.COPILOT_PLUS_CHAIN || currentChain === ChainType.PROJECT_CHAIN;
// Toggle states for vault, web search, composer, and autonomous agent
const [vaultToggle, setVaultToggle] = useState(false);
const [webToggle, setWebToggle] = useState(false);
const [composerToggle, setComposerToggle] = useState(false);
const [autonomousAgentToggle, setAutonomousAgentToggle] = useState(
settings.enableAutonomousAgent
);
const [loadingMessageIndex, setLoadingMessageIndex] = useState(0);
const loadingMessages = [
"Loading the project context...",
@ -133,6 +131,17 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
},
}));
// Sync autonomous agent toggle with settings and chain type
useEffect(() => {
if (currentChain === ChainType.PROJECT_CHAIN) {
// Force off in Projects mode
setAutonomousAgentToggle(false);
} else {
// In other modes, use the actual settings value
setAutonomousAgentToggle(settings.enableAutonomousAgent);
}
}, [settings.enableAutonomousAgent, currentChain]);
useEffect(() => {
if (currentChain === ChainType.PROJECT_CHAIN) {
setSelectedProject(getCurrentProject());
@ -170,14 +179,24 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
return currentModelKey;
};
const onSendMessage = (includeVault: boolean) => {
const onSendMessage = () => {
if (!isCopilotPlus) {
handleSendMessage();
return;
}
// Build tool calls based on toggle states
const toolCalls: string[] = [];
// Only add tool calls when autonomous agent is off
// When autonomous agent is on, it handles all tools internally
if (!autonomousAgentToggle) {
if (vaultToggle) toolCalls.push("@vault");
if (webToggle) toolCalls.push("@websearch");
if (composerToggle) toolCalls.push("@composer");
}
handleSendMessage({
toolCalls: includeVault ? ["@vault"] : [],
toolCalls,
contextNotes,
urls: contextUrls,
});
@ -336,14 +355,6 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.nativeEvent.isComposing) return;
// Check for Cmd+Shift+Enter (Mac) or Ctrl+Shift+Enter (Windows)
if (e.key === "Enter" && e.shiftKey && (Platform.isMacOS ? e.metaKey : e.ctrlKey)) {
e.preventDefault();
e.stopPropagation();
onSendMessage(true);
return;
}
if (e.key === "Enter") {
/**
* send msg:
@ -359,7 +370,7 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
}
e.preventDefault();
onSendMessage(false);
onSendMessage();
}
};
@ -563,71 +574,19 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
<span>Generating...</span>
</div>
) : (
<DropdownMenu open={isModelDropdownOpen} onOpenChange={setIsModelDropdownOpen}>
<DropdownMenuTrigger asChild>
<Button variant="ghost2" size="fit" disabled={disableModelSwitch}>
{modelError ? (
<span className="tw-text-error">Model Load Failed</span>
) : settings.activeModels.find(
(model) =>
model.enabled && getModelKeyFromModel(model) === getDisplayModelKey()
) ? (
<ModelDisplay
model={
settings.activeModels.find(
(model) =>
model.enabled && getModelKeyFromModel(model) === getDisplayModelKey()
)!
}
iconSize={8}
/>
) : (
"Select Model"
)}
{!disableModelSwitch && <ChevronDown className="tw-mt-0.5 tw-size-5" />}
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start">
{!disableModelSwitch &&
settings.activeModels
.filter((model) => model.enabled)
.map((model) => {
const { hasApiKey, errorNotice } = checkModelApiKey(model, settings);
return (
<DropdownMenuItem
key={getModelKeyFromModel(model)}
onSelect={async (event) => {
if (!hasApiKey && errorNotice) {
event.preventDefault();
new Notice(errorNotice);
return;
}
try {
setModelError(null);
setCurrentModelKey(getModelKeyFromModel(model));
} catch (error) {
const msg = `Model switch failed: ` + err2String(error);
setModelError(msg);
new Notice(msg);
// Restore to the last valid model
const lastValidModel = settings.activeModels.find(
(m) => m.enabled && getModelKeyFromModel(m) === currentModelKey
);
if (lastValidModel) {
setCurrentModelKey(getModelKeyFromModel(lastValidModel));
}
}
}}
className={!hasApiKey ? "tw-cursor-not-allowed tw-opacity-50" : ""}
>
<ModelDisplay model={model} iconSize={12} />
</DropdownMenuItem>
);
})}
</DropdownMenuContent>
</DropdownMenu>
<ModelSelector
variant="ghost2"
size="fit"
disabled={disableModelSwitch}
value={getDisplayModelKey()}
onChange={(modelKey) => {
// In project mode, we don't update the global model key
// as the project model takes precedence
if (currentChain !== ChainType.PROJECT_CHAIN) {
setCurrentModelKey(modelKey);
}
}}
/>
)}
<div className="tw-flex tw-items-center tw-gap-1">
@ -643,13 +602,80 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
</Button>
) : (
<>
{isCopilotPlus && (
{/* Autonomous Agent button - only show in Copilot Plus mode and NOT in Projects mode */}
{isCopilotPlus && currentChain !== ChainType.PROJECT_CHAIN && (
<Button
variant="ghost2"
size="fit"
onClick={() => {
const newValue = !autonomousAgentToggle;
setAutonomousAgentToggle(newValue);
updateSetting("enableAutonomousAgent", newValue);
}}
className={cn(
"tw-mr-2 tw-text-muted hover:tw-text-accent",
autonomousAgentToggle && "tw-text-accent tw-bg-accent/10"
)}
title="Toggle autonomous agent mode"
>
<Brain className="tw-size-4" />
</Button>
)}
{/* Toggle buttons for vault, web search, and composer - show when Autonomous Agent is off */}
{!autonomousAgentToggle && isCopilotPlus && (
<>
<Button
variant="ghost2"
size="fit"
onClick={() => setVaultToggle(!vaultToggle)}
className={cn(
"tw-mr-2 tw-text-muted hover:tw-text-accent",
vaultToggle && "tw-text-accent tw-bg-accent/10"
)}
title="Toggle vault search"
>
<Database className="tw-size-4" />
</Button>
<Button
variant="ghost2"
size="fit"
onClick={() => setWebToggle(!webToggle)}
className={cn(
"tw-mr-2 tw-text-muted hover:tw-text-accent",
webToggle && "tw-text-accent tw-bg-accent/10"
)}
title="Toggle web search"
>
<Globe className="tw-size-4" />
</Button>
<Button
variant="ghost2"
size="fit"
onClick={() => setComposerToggle(!composerToggle)}
className={cn(
"tw-mr-2 tw-text-muted hover:tw-text-accent",
composerToggle && "tw-text-accent tw-bg-accent/10"
)}
title="Toggle composer (note editing)"
>
<span className="tw-flex tw-items-center tw-gap-0.5">
<Sparkles className="tw-size-2" />
<Pen className="tw-size-3" />
</span>
</Button>
</>
)}
{isCopilotPlus && (
<Button
variant="ghost2"
size="fit"
className="tw-text-muted hover:tw-text-accent"
onClick={() => {
new AddImageModal(app, onAddImage).open();
}}
title="Add image(s)"
>
<Image className="tw-size-4" />
</Button>
@ -658,37 +684,11 @@ const ChatInput = forwardRef<{ focus: () => void }, ChatInputProps>(
variant="ghost2"
size="fit"
className="tw-text-muted"
onClick={() => onSendMessage(false)}
onClick={() => onSendMessage()}
>
<CornerDownLeft className="!tw-size-3" />
<span>chat</span>
</Button>
{currentChain === "copilot_plus" && (
<Button
variant="ghost2"
size="fit"
className="tw-hidden tw-text-muted @xs/chat-input:tw-inline-flex"
onClick={() => onSendMessage(true)}
>
<div className="tw-flex tw-items-center tw-gap-1">
{Platform.isMacOS ? (
<div className="tw-flex tw-items-center">
<Command className="!tw-size-3" />
<ArrowBigUp className="!tw-size-3" />
<CornerDownLeft className="!tw-size-3" />
</div>
) : (
<div className="tw-flex tw-items-center">
<span>Ctrl</span>
<ArrowBigUp className="tw-size-4" />
<CornerDownLeft className="!tw-size-3" />
</div>
)}
<span>vault</span>
</div>
</Button>
)}
</>
)}
</div>

View file

@ -1,6 +1,8 @@
import ChatSingleMessage from "@/components/chat-components/ChatSingleMessage";
import { RelevantNotes } from "@/components/chat-components/RelevantNotes";
import { SuggestedPrompts } from "@/components/chat-components/SuggestedPrompts";
import { USER_SENDER } from "@/constants";
import { useChatScrolling } from "@/hooks/useChatScrolling";
import { useSettingsValue } from "@/settings/model";
import { ChatMessage } from "@/types/message";
import { App } from "obsidian";
@ -37,18 +39,11 @@ const ChatMessages = memo(
const [loadingDots, setLoadingDots] = useState("");
const settings = useSettingsValue();
const scrollToBottom = () => {
const chatMessagesContainer = document.querySelector("[data-testid='chat-messages']");
if (chatMessagesContainer) {
chatMessagesContainer.scrollTop = chatMessagesContainer.scrollHeight;
}
};
useEffect(() => {
if (!loading) {
scrollToBottom();
}
}, [loading]);
// Chat scrolling behavior
const { containerMinHeight, scrollContainerCallbackRef, getMessageKey } = useChatScrolling({
chatHistory,
});
useEffect(() => {
let intervalId: NodeJS.Timeout;
@ -94,38 +89,60 @@ const ChatMessages = memo(
/>
)}
<div
ref={scrollContainerCallbackRef}
data-testid="chat-messages"
className="tw-mt-auto tw-box-border tw-flex tw-w-full tw-flex-1 tw-select-text tw-flex-col tw-items-start tw-justify-start tw-overflow-y-auto tw-scroll-smooth tw-break-words tw-text-[calc(var(--font-text-size)_-_2px)]"
className="tw-relative tw-flex tw-w-full tw-flex-1 tw-select-text tw-flex-col tw-items-start tw-justify-start tw-overflow-y-auto tw-scroll-smooth tw-break-words tw-text-[calc(var(--font-text-size)_-_2px)]"
>
{chatHistory.map(
(message, index) =>
{chatHistory.map((message, index) => {
const visibleMessages = chatHistory.filter((m) => m.isVisible);
const isLastMessage = index === visibleMessages.length - 1;
// Only apply min-height to AI messages that are last
const shouldApplyMinHeight = isLastMessage && message.sender !== USER_SENDER;
return (
message.isVisible && (
<ChatSingleMessage
key={index}
message={message}
app={app}
isStreaming={false}
onRegenerate={() => onRegenerate(index)}
onEdit={(newMessage) => onEdit(index, newMessage)}
onDelete={() => onDelete(index)}
chatHistory={chatHistory}
/>
<div
key={getMessageKey(message, index)}
data-message-key={getMessageKey(message, index)}
className="tw-w-full"
style={{
minHeight: shouldApplyMinHeight ? `${containerMinHeight}px` : "auto",
}}
>
<ChatSingleMessage
message={message}
app={app}
isStreaming={false}
onRegenerate={() => onRegenerate(index)}
onEdit={(newMessage) => onEdit(index, newMessage)}
onDelete={() => onDelete(index)}
chatHistory={chatHistory}
/>
</div>
)
)}
);
})}
{(currentAiMessage || loading) && (
<ChatSingleMessage
key={`ai_message_${currentAiMessage}`}
message={{
sender: "AI",
message: currentAiMessage || getLoadingMessage(),
isVisible: true,
timestamp: null,
<div
className="tw-w-full"
style={{
minHeight: `${containerMinHeight}px`,
}}
app={app}
isStreaming={true}
onDelete={() => {}}
chatHistory={chatHistory}
/>
>
<ChatSingleMessage
key="ai_message_streaming"
message={{
sender: "AI",
message: currentAiMessage || getLoadingMessage(),
isVisible: true,
timestamp: null,
}}
app={app}
isStreaming={true}
onDelete={() => {}}
chatHistory={chatHistory}
/>
</div>
)}
</div>
</div>

View file

@ -1,17 +1,23 @@
import { ChatButtons } from "@/components/chat-components/ChatButtons";
import { ToolCallBanner } from "@/components/chat-components/ToolCallBanner";
import { SourcesModal } from "@/components/modals/SourcesModal";
import { Badge } from "@/components/ui/badge";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import { USER_SENDER } from "@/constants";
import { cn } from "@/lib/utils";
import { parseToolCallMarkers } from "@/LLMProviders/chainRunner/utils/toolCallParser";
import { ChatMessage } from "@/types/message";
import { insertIntoEditor } from "@/utils";
import { cleanMessageForCopy, insertIntoEditor } from "@/utils";
import { Bot, User } from "lucide-react";
import { App, Component, MarkdownRenderer, MarkdownView, TFile } from "obsidian";
import { diffTrimmedLines, Change } from "diff";
import React, { useCallback, useEffect, useRef, useState } from "react";
import { createRoot, Root } from "react-dom/client";
import { ComposerCodeBlock } from "./ComposerCodeBlock";
import ReactDOM, { Root } from "react-dom/client";
declare global {
interface Window {
__copilotToolCallRoots?: Map<string, Map<string, Root>>;
}
}
function MessageContext({ context }: { context: ChatMessage["context"] }) {
if (!context || (!context.notes?.length && !context.urls?.length)) {
@ -69,13 +75,38 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
const contentRef = useRef<HTMLDivElement>(null);
const componentRef = useRef<Component | null>(null);
const textareaRef = useRef<HTMLTextAreaElement>(null);
// Use a stable ID for the message to preserve tool call roots across re-renders
const messageId = useRef(
message.timestamp?.epoch
? String(message.timestamp.epoch)
: `temp-${Date.now()}-${Math.random()}`
);
// Store roots in a global map to preserve them across component instances
const getGlobalRootsMap = () => {
if (!window.__copilotToolCallRoots) {
window.__copilotToolCallRoots = new Map<string, Map<string, Root>>();
}
return window.__copilotToolCallRoots;
};
const getRootsForMessage = () => {
const globalMap = getGlobalRootsMap();
if (!globalMap.has(messageId.current)) {
globalMap.set(messageId.current, new Map<string, Root>());
}
return globalMap.get(messageId.current)!;
};
const rootsRef = useRef<Map<string, Root>>(getRootsForMessage());
const copyToClipboard = () => {
if (!navigator.clipboard || !navigator.clipboard.writeText) {
return;
}
navigator.clipboard.writeText(message.message).then(() => {
const cleanedContent = cleanMessageForCopy(message.message);
navigator.clipboard.writeText(cleanedContent).then(() => {
setIsCopied(true);
setTimeout(() => {
@ -89,70 +120,93 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
const activeFile = app.workspace.getActiveFile();
const sourcePath = activeFile ? activeFile.path : "";
const processThinkSection = (content: string): string => {
const processCollapsibleSection = (
content: string,
tagName: string,
summaryText: string,
streamingSummaryText: string
): string => {
// Common styles as template strings
const detailsStyle = `margin: 0.5rem 0 1.5rem; padding: 0.75rem; border: 1px solid var(--background-modifier-border); border-radius: 4px; background-color: var(--background-secondary)`;
const summaryStyle = `cursor: pointer; color: var(--text-muted); font-size: 0.8em; margin-bottom: 0.5rem; user-select: none`;
const contentStyle = `margin-top: 0.75rem; padding: 0.75rem; border-radius: 4px; background-color: var(--background-primary)`;
// During streaming, if we find any think tag that's either unclosed or being processed
if (isStreaming && content.includes("<think>")) {
// Replace any complete think sections first
content = content.replace(/<think>([\s\S]*?)<\/think>/g, (match, thinkContent) => {
const openTag = `<${tagName}>`;
// During streaming, if we find any tag that's either unclosed or being processed
if (isStreaming && content.includes(openTag)) {
// Replace any complete sections first
const completeRegex = new RegExp(`<${tagName}>([\\s\\S]*?)<\\/${tagName}>`, "g");
content = content.replace(completeRegex, (match, sectionContent) => {
return `<details style="${detailsStyle}">
<summary style="${summaryStyle}">Thought for a second</summary>
<div class="tw-text-muted" style="${contentStyle}">${thinkContent.trim()}</div>
<summary style="${summaryStyle}">${summaryText}</summary>
<div class="tw-text-muted" style="${contentStyle}">${sectionContent.trim()}</div>
</details>\n\n`;
});
// Then handle any unclosed think tag, but preserve the streamed content
// Then handle any unclosed tag, but preserve the streamed content
const unClosedRegex = new RegExp(`<${tagName}>([\\s\\S]*)$`);
content = content.replace(
/<think>([\s\S]*)$/,
unClosedRegex,
(match, partialContent) => `<div style="${detailsStyle}">
<div style="${summaryStyle}">Thinking...</div>
<div style="${summaryStyle}">${streamingSummaryText}</div>
<div class="tw-text-muted" style="${contentStyle}">${partialContent.trim()}</div>
</div>`
);
return content;
}
// Not streaming, process all think sections normally
const thinkRegex = /<think>([\s\S]*?)<\/think>/g;
return content.replace(thinkRegex, (match, thinkContent) => {
// Not streaming, process all sections normally
const regex = new RegExp(`<${tagName}>([\\s\\S]*?)<\\/${tagName}>`, "g");
return content.replace(regex, (match, sectionContent) => {
return `<details style="${detailsStyle}">
<summary style="${summaryStyle}">Thought for a second</summary>
<div class="tw-text-muted" style="${contentStyle}">${thinkContent.trim()}</div>
<summary style="${summaryStyle}">${summaryText}</summary>
<div class="tw-text-muted" style="${contentStyle}">${sectionContent.trim()}</div>
</details>\n\n`;
});
};
// Showing loading placeholders for composer output during streaming
const processComposerCodeBlocks = (text: string): string => {
// Helper function to create the loading placeholder
const createPlaceholder = (path: string) => {
return `⏳ Generating changes for ${path}...`;
const processThinkSection = (content: string): string => {
return processCollapsibleSection(content, "think", "Thought for a while", "Thinking...");
};
const processWriteToFileSection = (content: string): string => {
// First, unwrap any XML codeblocks that contain writeToFile tags
const unwrapXmlCodeblocks = (text: string): string => {
// Pattern to match XML codeblocks that contain writeToFile tags
const xmlCodeblockRegex =
/```xml\s*([\s\S]*?<writeToFile>[\s\S]*?<\/writeToFile>[\s\S]*?)\s*```/g;
return text.replace(xmlCodeblockRegex, (match, xmlContent) => {
// Extract just the content inside the codeblock and return it without the codeblock wrapper
return xmlContent.trim();
});
};
if (isStreaming) {
// Look for any content containing "type": "composer"
const composerRegex = /(\{[\s\S]*?"type"\s*:\s*"composer"[\s\S]*?)(?=\}|$)/g;
// During streaming, also handle unclosed writeToFile tags in XML codeblocks
const unwrapStreamingXmlCodeblocks = (text: string): string => {
if (!isStreaming) return text;
let match;
while ((match = composerRegex.exec(text)) !== null) {
const jsonStr = match[1];
const start = match.index;
// Pattern to match XML codeblocks that contain unclosed writeToFile tags
const streamingXmlCodeblockRegex = /```xml\s*([\s\S]*?<writeToFile>[\s\S]*?)$/g;
// Try to extract the path if available
const pathMatch = jsonStr.match(/"path"\s*:\s*"([^"]+)"/);
const path = pathMatch ? pathMatch[1].trim() : "...";
return text.replace(streamingXmlCodeblockRegex, (match, xmlContent) => {
// Extract the content and return it without the codeblock wrapper
return xmlContent.trim();
});
};
// Replace from the start of the JSON to the end of the text
text = text.substring(0, start) + createPlaceholder(path);
break; // Only process the first match
}
}
// Unwrap XML codeblocks first
let processedContent = unwrapXmlCodeblocks(content);
processedContent = unwrapStreamingXmlCodeblocks(processedContent);
return text;
// Then process the writeToFile sections normally
return processCollapsibleSection(
processedContent,
"writeToFile",
"Generated new content",
"Generating changes..."
);
};
const replaceLinks = (text: string, regex: RegExp, template: (file: TFile) => string) => {
@ -182,12 +236,9 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
.replace(/\\\(\s*/g, "$")
.replace(/\s*\\\)/g, "$");
// Process code blocks first for streaming case
const codeBlocksProcessed = processComposerCodeBlocks(latexProcessed);
// Process only Obsidian internal images (starting with ![[)
const noteImageProcessed = replaceLinks(
codeBlocksProcessed,
latexProcessed,
/!\[\[(.*?)]]/g,
(file) => `![](${app.vault.getResourcePath(file)})`
);
@ -195,8 +246,11 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
// Process think sections
const thinkSectionProcessed = processThinkSection(noteImageProcessed);
// Process writeToFile sections
const writeToFileSectionProcessed = processWriteToFileSection(thinkSectionProcessed);
// Transform markdown sources section into HTML structure
const sourcesSectionProcessed = processSourcesSection(thinkSectionProcessed);
const sourcesSectionProcessed = processSourcesSection(writeToFileSectionProcessed);
// Transform [[link]] to clickable format but exclude ![[]] image links
const noteLinksProcessed = replaceLinks(
@ -236,138 +290,198 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
};
useEffect(() => {
const roots: Root[] = [];
let isUnmounting = false;
if (contentRef.current && message.sender !== USER_SENDER) {
contentRef.current.innerHTML = "";
// Create a new Component instance if it doesn't exist
if (!componentRef.current) {
componentRef.current = new Component();
}
const processedMessage = preprocess(message.message);
const parsedMessage = parseToolCallMarkers(processedMessage);
if (!isUnmounting) {
// Use Obsidian's MarkdownRenderer to render the message
MarkdownRenderer.renderMarkdown(
processedMessage,
contentRef.current,
"", // Empty string for sourcePath as we don't have a specific source file
componentRef.current
// Track existing tool call IDs
const existingToolCallIds = new Set<string>();
const existingElements = contentRef.current.querySelectorAll('[id^="tool-call-"]');
existingElements.forEach((el) => {
const id = el.id.replace("tool-call-", "");
existingToolCallIds.add(id);
});
// Clear only text content divs, preserve tool call containers
const textDivs = contentRef.current.querySelectorAll(".message-segment");
textDivs.forEach((div) => div.remove());
// Process segments and only update what's needed
let currentIndex = 0;
parsedMessage.segments.forEach((segment, index) => {
if (segment.type === "text" && segment.content.trim()) {
// Find where to insert this text segment
const insertBefore = contentRef.current!.children[currentIndex];
const textDiv = document.createElement("div");
textDiv.className = "message-segment";
if (insertBefore) {
contentRef.current!.insertBefore(textDiv, insertBefore);
} else {
contentRef.current!.appendChild(textDiv);
}
MarkdownRenderer.renderMarkdown(segment.content, textDiv, "", componentRef.current!);
currentIndex++;
} else if (segment.type === "toolCall" && segment.toolCall) {
const toolCallId = segment.toolCall.id;
const existingDiv = document.getElementById(`tool-call-${toolCallId}`);
if (existingDiv) {
// Update existing tool call
const root = rootsRef.current.get(toolCallId);
if (root) {
root.render(
<ToolCallBanner
toolName={segment.toolCall.toolName}
displayName={segment.toolCall.displayName}
emoji={segment.toolCall.emoji}
isExecuting={segment.toolCall.isExecuting}
result={segment.toolCall.result || null}
confirmationMessage={segment.toolCall.confirmationMessage}
/>
);
}
currentIndex++;
} else {
// Create new tool call
const insertBefore = contentRef.current!.children[currentIndex];
const toolDiv = document.createElement("div");
toolDiv.className = "tool-call-container";
toolDiv.id = `tool-call-${toolCallId}`;
if (insertBefore) {
contentRef.current!.insertBefore(toolDiv, insertBefore);
} else {
contentRef.current!.appendChild(toolDiv);
}
const root = ReactDOM.createRoot(toolDiv);
rootsRef.current.set(toolCallId, root);
root.render(
<ToolCallBanner
toolName={segment.toolCall.toolName}
displayName={segment.toolCall.displayName}
emoji={segment.toolCall.emoji}
isExecuting={segment.toolCall.isExecuting}
result={segment.toolCall.result || null}
confirmationMessage={segment.toolCall.confirmationMessage}
/>
);
currentIndex++;
}
}
});
// Clean up any tool calls that no longer exist
const currentToolCallIds = new Set(
parsedMessage.segments
.filter((s) => s.type === "toolCall" && s.toolCall)
.map((s) => s.toolCall!.id)
);
// Only process code blocks with file paths after streaming is complete
if (!isStreaming) {
// Process code blocks after rendering
const codeBlocks = contentRef.current.querySelectorAll("pre");
if (codeBlocks.length > 0) {
codeBlocks.forEach((pre) => {
if (isUnmounting) return;
const codeElement = pre.querySelector("code");
if (!codeElement) return;
const originalCode = codeElement.textContent || "";
// Check for JSON composer format
try {
// Look for complete JSON objects
if (originalCode.trim().startsWith("{") && originalCode.trim().endsWith("}")) {
const composerData = JSON.parse(originalCode);
if (
composerData.type === "composer" &&
composerData.path &&
// `content` and `canvas_json` should never exist together
(typeof composerData.content === "string" ||
typeof composerData.canvas_json === "object")
) {
let newContent;
if (typeof composerData.content === "string") {
newContent = composerData.content;
} else {
newContent = JSON.stringify(composerData.canvas_json);
}
let path = composerData.path.trim();
// If path starts with a /, remove it
if (path.startsWith("/")) {
path = path.slice(1);
}
// Create a container for the React component
const container = document.createElement("div");
pre.parentNode?.replaceChild(container, pre);
// Create a root and render the CodeBlock component
const root = createRoot(container);
roots.push(root);
const file = app.vault.getAbstractFileByPath(path);
let note_changes: Change[] = [];
// Use async IIFE here
(async () => {
if (file instanceof TFile) {
// Update existing file
const originalContent = await app.vault.read(file);
note_changes = diffTrimmedLines(originalContent, newContent, {
newlineIsToken: true,
});
} else {
// Create new file
// get the file name from `path` without the extension
const fileName = path.split("/").pop()?.split(".")[0];
// Check first line of content for `# ${fileName}\n` and remove it
const lines = newContent.split("\n");
if (lines[0] === `# ${fileName}\n` || lines[0] === `## ${fileName}`) {
lines.shift();
}
newContent = lines.join("\n");
}
if (!isUnmounting) {
root.render(
<ComposerCodeBlock
note_path={path}
note_content={newContent}
note_changes={note_changes}
/>
);
}
})();
existingToolCallIds.forEach((id) => {
if (!currentToolCallIds.has(id)) {
const element = document.getElementById(`tool-call-${id}`);
if (element) {
const root = rootsRef.current.get(id);
if (root) {
// Defer unmounting to avoid React rendering conflicts
setTimeout(() => {
try {
root.unmount();
} catch (error) {
console.debug("Error unmounting tool call root:", error);
}
}
} catch (e) {
console.error("Failed to parse composer JSON:", e);
rootsRef.current.delete(id);
}, 0);
}
});
element.remove();
}
}
}
});
}
}
// Cleanup function
return () => {
isUnmounting = true;
// Schedule cleanup to run after current render cycle
setTimeout(() => {
if (componentRef.current) {
componentRef.current.unload();
componentRef.current = null;
}
roots.forEach((root) => {
try {
root.unmount();
} catch {
// Ignore unmount errors during cleanup
}
});
}, 0);
};
}, [message, app, componentRef, isStreaming, preprocess]);
// Cleanup effect that only runs on component unmount
useEffect(() => {
const currentComponentRef = componentRef;
const currentMessageId = messageId.current;
// Clean up old message roots to prevent memory leaks (older than 1 hour)
const cleanupOldRoots = () => {
const globalMap = getGlobalRootsMap();
const oneHourAgo = Date.now() - 60 * 60 * 1000;
globalMap.forEach((roots, msgId) => {
// Extract timestamp from message ID if it's in epoch format
const timestamp = parseInt(msgId);
if (!isNaN(timestamp) && timestamp < oneHourAgo) {
// Defer cleanup to avoid React rendering conflicts
setTimeout(() => {
roots.forEach((root) => {
try {
root.unmount();
} catch {
// Ignore errors
}
});
globalMap.delete(msgId);
}, 0);
}
});
};
// Run cleanup on mount
cleanupOldRoots();
return () => {
// Defer cleanup to avoid React rendering conflicts
setTimeout(() => {
// Clean up component
if (currentComponentRef.current) {
currentComponentRef.current.unload();
currentComponentRef.current = null;
}
// Only clean up roots if this is a temporary message (streaming message)
// Permanent messages keep their roots to preserve tool call banners
if (currentMessageId.startsWith("temp-")) {
const globalMap = getGlobalRootsMap();
const messageRoots = globalMap.get(currentMessageId);
if (messageRoots) {
messageRoots.forEach((root) => {
try {
root.unmount();
} catch (error) {
// Ignore unmount errors during cleanup
console.debug("Error unmounting React root during cleanup:", error);
}
});
globalMap.delete(currentMessageId);
}
}
}, 0);
};
}, []); // Empty dependency array ensures this only runs on unmount
useEffect(() => {
if (isEditing && textareaRef.current) {
adjustTextareaHeight(textareaRef.current);
@ -430,7 +544,8 @@ const ChatSingleMessage: React.FC<ChatSingleMessageProps> = ({
const editor = leaf.view.editor;
const hasSelection = editor.getSelection().length > 0;
insertIntoEditor(message.message, hasSelection);
const cleanedContent = cleanMessageForCopy(message.message);
insertIntoEditor(cleanedContent, hasSelection);
};
const renderMessageContent = () => {

View file

@ -1,133 +0,0 @@
import React from "react";
import { Button } from "@/components/ui/button";
import { Check, Clipboard } from "lucide-react";
import { APPLY_VIEW_TYPE } from "@/components/composer/ApplyView";
import { logError } from "@/logger";
import { Notice } from "obsidian";
import { TFile } from "obsidian";
import { Change } from "diff";
import { getRelevantChangesMarkdown, getChangeBlocks } from "@/composerUtils";
interface ComposerCodeBlockProps {
note_path: string;
note_content: string;
note_changes: Change[];
}
export const ComposerCodeBlock: React.FC<ComposerCodeBlockProps> = ({
note_path,
note_content,
note_changes,
}) => {
const handlePreview = async () => {
if (!note_path) return;
try {
let file = app.vault.getAbstractFileByPath(note_path);
let isNewFile = false;
// If file doesn't exist, create it
if (!file) {
try {
// Create the folder if it doesn't exist
if (note_path.includes("/")) {
const folderPath = note_path.split("/").slice(0, -1).join("/");
const folder = app.vault.getAbstractFileByPath(folderPath);
if (!folder) {
await app.vault.createFolder(folderPath);
}
}
file = await app.vault.create(note_path, note_content);
if (file) {
new Notice(`Created new file: ${note_path}`);
isNewFile = true;
} else {
new Notice(`Failed to create file: ${note_path}`);
return;
}
isNewFile = true;
} catch (createError) {
logError("Error creating file:", createError);
new Notice(`Failed to create file: ${createError.message}`);
return;
}
}
if (!(file instanceof TFile)) {
new Notice(`Path is not a file: ${note_path}`);
return;
}
// Check if the current active note is the same as the target note
const activeFile = app.workspace.getActiveFile();
if (!activeFile || activeFile.path !== note_path) {
// If not, open the target file in the current leaf
await app.workspace.getLeaf().openFile(file);
new Notice(`Switched to ${file.name}`);
}
// If the file is newly created, don't show the apply view
if (isNewFile) {
return;
}
// Open the Apply View in a new leaf with the processed content
const leaf = app.workspace.getLeaf(true);
await leaf.setViewState({
type: APPLY_VIEW_TYPE,
active: true,
state: {
changes: note_changes,
path: note_path,
},
});
} catch (error) {
logError("Error calling composer apply:", error);
new Notice(`Error processing code: ${error.message}`);
}
};
const handleCopy = () => {
try {
const contentToCopy =
note_changes.length > 0
? getRelevantChangesMarkdown(getChangeBlocks(note_changes))
: note_content;
navigator.clipboard.writeText(contentToCopy);
new Notice("Content copied to clipboard");
} catch (error) {
logError("Error copying to clipboard:", error);
new Notice(`Failed to copy: ${error.message}`);
}
};
return (
<div className="tw-my-2 tw-flex tw-flex-col tw-overflow-hidden tw-rounded-md tw-border tw-border-solid tw-border-border">
{note_path && (
<div className="tw-flex tw-items-center tw-justify-between tw-gap-2 tw-overflow-hidden tw-border-[0px] tw-border-b tw-border-solid tw-border-border tw-p-2">
<div className="tw-flex-1 tw-truncate tw-p-1 tw-text-xs tw-text-muted">{note_path}</div>
<div className="tw-flex tw-gap-2">
<Button className="tw-text-muted" variant="ghost2" size="fit" onClick={handleCopy}>
<Clipboard className="tw-size-4" />
Copy
</Button>
<Button className="tw-text-muted" variant="ghost2" size="fit" onClick={handlePreview}>
<Check className="tw-size-4" />
{!app.vault.getAbstractFileByPath(note_path) ? "Apply" : "Preview"}
</Button>
</div>
</div>
)}
<pre className="tw-m-0 tw-border-none">
<code>
{note_changes.length > 0
? getRelevantChangesMarkdown(getChangeBlocks(note_changes))
: note_content}
</code>
</pre>
</div>
);
};

View file

@ -138,6 +138,7 @@ function RelevantNote({
}
}}
className="tw-block tw-w-full tw-truncate tw-text-sm tw-font-bold tw-text-normal"
title={note.document.title}
>
{note.document.title}
</a>

View file

@ -0,0 +1,133 @@
import React, { useState } from "react";
import { ChevronRight, Check, X } from "lucide-react";
import { Collapsible, CollapsibleContent, CollapsibleTrigger } from "@/components/ui/collapsible";
import { cn } from "@/lib/utils";
import { ToolResultFormatter } from "@/tools/ToolResultFormatter";
// Animation constants
// The shimmer keyframe is defined in the global CSS (see styles.css)
const SHIMMER_ANIMATION = "shimmer 2s ease-in-out infinite";
interface ToolCallBannerProps {
toolName: string;
displayName: string;
emoji: string;
isExecuting: boolean;
result: string | null;
confirmationMessage?: string | null;
onAccept?: () => void;
onReject?: () => void;
}
export const ToolCallBanner: React.FC<ToolCallBannerProps> = ({
toolName,
displayName,
emoji,
isExecuting,
result,
confirmationMessage,
onAccept,
onReject,
}) => {
const [isOpen, setIsOpen] = useState(false);
// Don't allow expanding while executing
const canExpand = !isExecuting && result !== null;
return (
<Collapsible
open={isOpen}
onOpenChange={setIsOpen}
disabled={!canExpand}
aria-disabled={!canExpand}
className="tw-my-3 tw-w-full sm:tw-max-w-sm"
>
<div
className={cn(
"tw-rounded-md tw-border tw-border-border tw-bg-secondary/50",
isExecuting && "tw-relative tw-overflow-hidden"
)}
>
{/* Shimmer effect overlay */}
{isExecuting && (
<div className="tw-absolute tw-inset-0 tw-z-[1] tw-overflow-hidden">
<div
className="tw-absolute tw-inset-0 -tw-translate-x-full"
style={{
background:
"linear-gradient(90deg, transparent 0%, rgba(255, 255, 255, 0.3) 50%, transparent 100%)",
animation: SHIMMER_ANIMATION,
}}
/>
</div>
)}
<CollapsibleTrigger
className={cn(
"tw-flex tw-w-full tw-items-center tw-justify-between tw-px-3 tw-py-2.5 tw-text-sm sm:tw-px-4 sm:tw-py-3",
canExpand && "hover:tw-bg-secondary/70",
!canExpand && "tw-cursor-default"
)}
>
<div className="tw-flex tw-items-center tw-gap-2">
<span className="tw-text-base">{emoji}</span>
<span className="tw-font-medium">
{isExecuting ? "Calling" : "Called"} {displayName}
{isExecuting && "..."}
</span>
{isExecuting && confirmationMessage && (
<span className="tw-text-xs tw-text-muted"> {confirmationMessage}...</span>
)}
</div>
<div className="tw-flex tw-items-center tw-gap-2">
{/* Future: Accept/Reject buttons */}
{!isExecuting && onAccept && onReject && (
<>
<button
onClick={(e) => {
e.stopPropagation();
onAccept();
}}
className="hover:tw-bg-green-rgb/20 tw-rounded tw-p-1"
title="Accept"
>
<Check className="tw-size-4 tw-text-success" />
</button>
<button
onClick={(e) => {
e.stopPropagation();
onReject();
}}
className="hover:tw-bg-red-rgb/20 tw-rounded tw-p-1"
title="Reject"
>
<X className="tw-size-4 tw-text-error" />
</button>
</>
)}
{canExpand && (
<ChevronRight
className={cn(
"tw-size-4 tw-text-muted tw-transition-transform",
isOpen && "tw-rotate-90"
)}
/>
)}
</div>
</CollapsibleTrigger>
<CollapsibleContent>
<div className="tw-border-t tw-border-border tw-px-3 tw-py-2.5 sm:tw-px-4 sm:tw-py-3">
<div className="tw-text-sm tw-text-muted">
<pre className="tw-overflow-x-auto tw-whitespace-pre-wrap tw-font-mono tw-text-xs">
{result ? ToolResultFormatter.format(toolName, result) : "No result available"}
</pre>
</div>
</div>
</CollapsibleContent>
</div>
</Collapsible>
);
};

View file

@ -8,15 +8,17 @@ import { createRoot } from "react-dom/client";
import { Button } from "../ui/button";
import { useState } from "react";
import { getChangeBlocks } from "@/composerUtils";
import { ApplyViewResult } from "@/types";
export const APPLY_VIEW_TYPE = "obsidian-copilot-apply-view";
export interface ApplyViewState {
changes: Change[];
path: string;
resultCallback?: (result: ApplyViewResult) => void;
}
// Extended Change interface to track user decisions
// Extended Change interface to track user acceptance
interface ExtendedChange extends Change {
accepted: boolean | null;
}
@ -24,6 +26,7 @@ interface ExtendedChange extends Change {
export class ApplyView extends ItemView {
private root: ReturnType<typeof createRoot> | null = null;
private state: ApplyViewState | null = null;
private result: ApplyViewResult | null = null;
constructor(leaf: WorkspaceLeaf) {
super(leaf);
@ -51,6 +54,8 @@ export class ApplyView extends ItemView {
this.root.unmount();
this.root = null;
}
this.state?.resultCallback?.(this.result ? this.result : "aborted");
}
private render() {
@ -66,8 +71,16 @@ export class ApplyView extends ItemView {
this.root = createRoot(rootEl);
}
// Pass a close function that takes a result
this.root.render(
<ApplyViewRoot app={this.app} state={this.state} close={() => this.leaf.detach()} />
<ApplyViewRoot
app={this.app}
state={this.state}
close={(result) => {
this.result = result;
this.leaf.detach();
}}
/>
);
}
}
@ -75,7 +88,7 @@ export class ApplyView extends ItemView {
interface ApplyViewRootProps {
app: App;
state: ApplyViewState;
close: () => void;
close: (result: ApplyViewResult) => void;
}
// Convert renderWordDiff to a React component
@ -126,7 +139,7 @@ const ApplyViewRoot: React.FC<ApplyViewRootProps> = ({ app, state, close }) => {
return (
<div className="tw-flex tw-h-full tw-flex-col tw-items-center tw-justify-center">
<div className="tw-text-error">Error: Invalid state - missing changes</div>
<Button onClick={close} className="tw-mt-4">
<Button onClick={() => close("failed")} className="tw-mt-4">
Close
</Button>
</div>
@ -141,10 +154,12 @@ const ApplyViewRoot: React.FC<ApplyViewRootProps> = ({ app, state, close }) => {
change.accepted === null ? { ...change, accepted: true } : change
);
await applyDecidedChangesToFile(updatedDiff);
const result = await applyDecidedChangesToFile(updatedDiff);
close(result ? "accepted" : "failed"); // Pass result
} catch (error) {
logError("Error applying changes:", error);
new Notice(`Error applying changes: ${error.message}`);
close("failed"); // fallback, but you may want to handle this differently
}
};
@ -156,13 +171,31 @@ const ApplyViewRoot: React.FC<ApplyViewRootProps> = ({ app, state, close }) => {
change.accepted === null ? { ...change, accepted: false } : change
);
await applyDecidedChangesToFile(updatedDiff);
const result = await applyDecidedChangesToFile(updatedDiff);
close(result ? "rejected" : "failed"); // Pass result
} catch (error) {
logError("Error applying changes:", error);
new Notice(`Error applying changes: ${error.message}`);
close("failed");
}
};
const getFile = async (file_path: string) => {
const file = app.vault.getAbstractFileByPath(file_path);
if (file) {
return file;
}
// Create the folder if it doesn't exist
if (file_path.includes("/")) {
const folderPath = file_path.split("/").slice(0, -1).join("/");
const folder = app.vault.getAbstractFileByPath(folderPath);
if (!folder) {
await app.vault.createFolder(folderPath);
}
}
return await app.vault.create(file_path, "");
};
// Shared function to apply changes to file
const applyDecidedChangesToFile = async (updatedDiff: ExtendedChange[]) => {
// Apply changes based on their accepted status
@ -175,16 +208,16 @@ const ApplyViewRoot: React.FC<ApplyViewRootProps> = ({ app, state, close }) => {
.map((change) => change.value)
.join("");
const file = app.vault.getAbstractFileByPath(state.path);
const file = await getFile(state.path);
if (!file || !(file instanceof TFile)) {
new Notice("File not found:" + state.path);
close();
return;
logError("Error in getting file", state.path);
new Notice("Failed to create file");
return false;
}
await app.vault.modify(file, newContent);
new Notice("Changes applied successfully");
close();
return true;
};
// Function to focus on the next change block or scroll to top if it's the last block
@ -285,7 +318,7 @@ const ApplyViewRoot: React.FC<ApplyViewRootProps> = ({ app, state, close }) => {
// Check if this block contains any changes (added or removed)
const hasChanges = block.some((change) => change.added || change.removed);
// Get the decision status for this block
// Get the result status for this block
const blockStatus = hasChanges
? block.every(
(change) =>

View file

@ -1,62 +0,0 @@
import { App, Modal } from "obsidian";
export class AdhocPromptModal extends Modal {
result: string;
onSubmit: (result: string) => void;
private placeholderText = "Please enter your custom ad-hoc prompt here, press enter to send.";
constructor(app: App, onSubmit: (result: string) => void) {
super(app);
this.onSubmit = onSubmit;
}
onOpen() {
const { contentEl } = this;
const promptDescFragment = createFragment((frag) => {
frag.createEl("strong", { text: "- {} represents the selected text (not required). " });
frag.createEl("br");
frag.createEl("strong", { text: "- {[[Note Title]]} represents a note. " });
frag.createEl("br");
frag.createEl("strong", { text: "- {activeNote} represents the active note. " });
frag.createEl("br");
frag.createEl("strong", { text: "- {FolderPath} represents a folder of notes. " });
frag.createEl("br");
frag.createEl("strong", {
text: "- {#tag1, #tag2} represents ALL notes with ANY of the specified tags in their property (an OR operation). ",
});
frag.createEl("br");
frag.createEl("br");
frag.appendText("Tip: turn on debug mode to show the processed prompt in the chat window.");
frag.createEl("br");
frag.createEl("br");
});
contentEl.appendChild(promptDescFragment);
const textareaEl = contentEl.createEl("textarea", {
attr: { placeholder: this.placeholderText },
});
textareaEl.style.width = "100%";
textareaEl.style.height = "100px"; // Set the desired height
textareaEl.style.padding = "10px";
textareaEl.style.resize = "vertical"; // Allow vertical resizing
textareaEl.addEventListener("input", (evt) => {
this.result = (evt.target as HTMLTextAreaElement).value;
});
textareaEl.addEventListener("keydown", (evt) => {
if (evt.key === "Enter" && !evt.shiftKey) {
evt.preventDefault(); // Prevent line break unless Shift key is pressed
this.close();
this.onSubmit(this.result);
}
});
}
onClose() {
const { contentEl } = this;
contentEl.empty();
}
}

View file

@ -0,0 +1,66 @@
import { App, FuzzySuggestModal, MarkdownView } from "obsidian";
import { getCachedCustomCommands } from "@/commands/state";
import { CustomCommand } from "@/commands/type";
import { sortSlashCommands } from "@/commands/customCommandUtils";
import { CustomCommandChatModal } from "@/commands/CustomCommandChatModal";
import { CustomCommandManager } from "@/commands/customCommandManager";
export class ApplyCustomCommandModal extends FuzzySuggestModal<CustomCommand> {
private commands: CustomCommand[];
constructor(app: App) {
super(app);
this.setPlaceholder("Select a custom command to apply...");
// Get all custom commands and sort them by the user's selected ordering method
const allCommands = getCachedCustomCommands();
this.commands = sortSlashCommands(allCommands);
}
onOpen() {
super.onOpen();
// Check if there are no commands available
if (this.commands.length === 0) {
this.setInstructions([
{
command: "",
purpose: "No custom commands found. Create some custom commands first in the settings.",
},
]);
}
}
getItems(): CustomCommand[] {
return this.commands;
}
getItemText(command: CustomCommand): string {
return command.title;
}
onChooseItem(command: CustomCommand, evt: MouseEvent | KeyboardEvent) {
const activeView = this.app.workspace.getActiveViewOfType(MarkdownView);
if (!activeView || !activeView.editor) {
// If no active editor, use empty string as selected text
this.openCommandModal(command, "");
return;
}
const selectedText = activeView.editor.getSelection();
this.openCommandModal(command, selectedText);
}
private openCommandModal(command: CustomCommand, selectedText: string) {
// Record usage of the command
CustomCommandManager.getInstance().recordUsage(command);
// Open the CustomCommandChatModal with the selected command
const modal = new CustomCommandChatModal(this.app, {
selectedText,
command,
});
modal.open();
}
}

View file

@ -29,21 +29,30 @@ export class LoadChatHistoryModal extends FuzzySuggestModal<TFile> {
}
getItemText(file: TFile): string {
// First, remove project ID prefix if it exists (format: projectId__)
const basename = file.basename.replace(/^[a-zA-Z0-9-]+__/, "");
// Read the file's front matter
const frontmatter = this.app.metadataCache.getFileCache(file)?.frontmatter;
// Remove {$date} and {$time} parts from the filename
const title = basename
.replace(/\{\$date}|\d{8}/g, "") // Remove {$date} or date in format YYYYMMDD
.replace(/\{\$time}|\d{6}/g, "") // Remove {$time} or time in format HHMMSS
.replace(/[@_]/g, " ") // Replace @ and _ with spaces
.replace(/\s+/g, " ") // Replace multiple spaces with single space
.trim();
let title: string;
// First check if there's a custom topic in frontmatter
if (frontmatter?.topic && typeof frontmatter.topic === "string" && frontmatter.topic.trim()) {
title = frontmatter.topic.trim();
} else {
// Fallback to extracting from filename
// First, remove project ID prefix if it exists (format: projectId__)
const basename = file.basename.replace(/^[a-zA-Z0-9-]+__/, "");
// Remove {$date} and {$time} parts from the filename
title = basename
.replace(/\{\$date}|\d{8}/g, "") // Remove {$date} or date in format YYYYMMDD
.replace(/\{\$time}|\d{6}/g, "") // Remove {$time} or time in format HHMMSS
.replace(/[@_]/g, " ") // Replace @ and _ with spaces
.replace(/\s+/g, " ") // Replace multiple spaces with single space
.trim();
}
let formattedDateTime: FormattedDateTime;
// Read the file's front matter
const frontmatter = this.app.metadataCache.getFileCache(file)?.frontmatter;
if (frontmatter && frontmatter.epoch) {
// Use the epoch from front matter if available
formattedDateTime = formatDateTime(new Date(frontmatter.epoch));

View file

@ -3,9 +3,9 @@ import { CONTEXT_SCORE_THRESHOLD } from "@/constants";
import { App, Modal } from "obsidian";
export class SourcesModal extends Modal {
sources: { title: string; score: number }[];
sources: { title: string; path: string; score: number }[];
constructor(app: App, sources: { title: string; score: number }[]) {
constructor(app: App, sources: { title: string; path: string; score: number }[]) {
super(app);
this.sources = sources;
}
@ -31,7 +31,10 @@ export class SourcesModal extends Modal {
}
}
private createSourceList(container: HTMLElement, sources: { title: string; score: number }[]) {
private createSourceList(
container: HTMLElement,
sources: { title: string; path: string; score: number }[]
) {
const list = container.createEl("ul");
list.style.listStyleType = "none";
list.style.padding = "0";
@ -40,13 +43,20 @@ export class SourcesModal extends Modal {
const item = list.createEl("li");
item.style.marginBottom = "1em";
// Display title, but show path in parentheses if there are duplicates
const displayText =
source.path && source.path !== source.title
? `${source.title} (${source.path})`
: source.title;
const link = item.createEl("a", {
href: `obsidian://open?vault=${encodeURIComponent(this.app.vault.getName())}&file=${encodeURIComponent(source.title)}`,
text: source.title,
href: `obsidian://open?vault=${encodeURIComponent(this.app.vault.getName())}&file=${encodeURIComponent(source.path || source.title)}`,
text: displayText,
});
link.addEventListener("click", (e) => {
e.preventDefault();
this.app.workspace.openLinkText(source.title, "");
// Use the path if available, otherwise fall back to title
this.app.workspace.openLinkText(source.path || source.title, "");
});
if (source.score && source.score <= 1) {
item.appendChild(document.createTextNode(` - Relevance score: ${source.score.toFixed(3)}`));

View file

@ -220,6 +220,56 @@ function ItemCard({ item, viewMode, onDelete }: ItemCardProps) {
);
}
function CategoryItemCard({
item,
onClick,
}: {
item: CategoryItem;
onClick: (item: CategoryItem) => void;
}) {
let IconComponent;
let iconColorClassName;
switch (item.type) {
case "tag":
IconComponent = TagIcon;
iconColorClassName = "tw-text-context-manager-orange";
break;
case "folder":
IconComponent = FolderIcon;
iconColorClassName = "tw-text-context-manager-yellow";
break;
case "files":
IconComponent = FileText;
iconColorClassName = "tw-text-context-manager-blue";
break;
case "ignoreFiles":
IconComponent = XIcon;
iconColorClassName = "tw-text-context-manager-red";
break;
}
return (
<div
className="tw-group tw-flex tw-cursor-pointer tw-items-center tw-rounded-lg tw-border tw-border-solid tw-border-border tw-p-2 tw-transition-shadow hover:tw-shadow-md"
onClick={() => onClick(item)}
>
<div className="tw-mr-2 tw-shrink-0">
<IconComponent className={`tw-size-6 ${iconColorClassName}`} />
</div>
<div className="tw-flex tw-min-w-0 tw-flex-1 tw-flex-col">
<TruncatedText className="tw-flex-1 tw-text-sm tw-font-medium">
{item.type === "tag" && <span className="tw-mr-2 tw-text-faint">#</span>}
{item.name}
</TruncatedText>
<TruncatedText className="tw-flex-1 tw-text-xs tw-text-faint">
{item.count} {item.count === 1 ? "item" : "items"}
</TruncatedText>
</div>
</div>
);
}
interface ContextManageProps {
initialProject: ProjectConfig;
onSave: (project: ProjectConfig) => void;
@ -244,6 +294,20 @@ interface IgnoreItems {
files: Set<TFile>;
}
interface CategoryItem {
id: string;
name: string;
type: "tag" | "folder" | "files" | "ignoreFiles";
originalId?: string;
count: number;
}
type DisplayItem = GroupItem | CategoryItem;
function isCategoryItem(item: DisplayItem): item is CategoryItem {
return "type" in item;
}
function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageProps) {
const { inclusions: inclusionPatterns, exclusions: exclusionPatterns } = useMemo(() => {
return getMatchingPatterns({
@ -265,7 +329,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
exclusionPatterns: PatternCategory | null
): GroupListItem => {
const projectAllFiles = appFiles.filter((file) =>
shouldIndexFile(file, inclusionPatterns, exclusionPatterns)
shouldIndexFile(file, inclusionPatterns, exclusionPatterns, true)
);
const processPatternGroup = (
@ -278,7 +342,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
patterns.forEach((pattern) => {
const singlePatternConfig = { [patternType]: [pattern] };
if (
shouldIndexFile(file, singlePatternConfig, null) &&
shouldIndexFile(file, singlePatternConfig, null, true) &&
!targetGroup[pattern].some((item) => item.id === file.path)
) {
targetGroup[pattern].push({
@ -325,7 +389,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
// note/file
if (
inclusionPatterns?.notePatterns &&
shouldIndexFile(file, { notePatterns: inclusionPatterns.notePatterns }, null) &&
shouldIndexFile(file, { notePatterns: inclusionPatterns.notePatterns }, null, true) &&
!notes.some((item) => item.id === file.path)
) {
notes.push({
@ -352,7 +416,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
const [ignoreItems, setIgnoreItems] = useState<IgnoreItems>(() => {
// init exclude files
const excludeFiles = appAllFiles.filter(
(file) => exclusionPatterns && shouldIndexFile(file, exclusionPatterns, null)
(file) => exclusionPatterns && shouldIndexFile(file, exclusionPatterns, null, true)
);
return {
files: new Set<TFile>(excludeFiles),
@ -460,7 +524,13 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
return { tags, titles, extensions };
}, []);
const getDisplayItems = useMemo(() => {
const sortItems = useCallback((items: DisplayItem[]) => {
return [...items].sort((a, b) => {
return a.name.toLowerCase().localeCompare(b.name.toLowerCase());
});
}, []);
const getDisplayItems = useMemo<DisplayItem[]>(() => {
if (searchTerm) {
// Custom search
const parsedQuery = parseSearchQuery(searchTerm);
@ -547,6 +617,55 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
}));
}
// When no part is selected, return all items
if (!activeSection) {
const tagItems = sortItems(
Object.entries(groupList.tags).map(([tagId, files]) => ({
id: `tag:${tagId}`,
name: tagId.slice(1),
type: "tag",
originalId: tagId,
count: files.length,
}))
);
const folderItems = sortItems(
Object.entries(groupList.folders).map(([folderId, files]) => ({
id: `folder:${folderId}`,
name: folderId,
type: "folder",
originalId: folderId,
count: files.length,
}))
);
const filesItem =
groupList.notes.length > 0
? [
{
id: "files:all",
name: "Files",
type: "files",
count: groupList.notes.length,
},
]
: [];
const ignoreFilesItem =
ignoreItems.files.size > 0
? [
{
id: "ignoreFiles:all",
name: "Ignore Files",
type: "ignoreFiles",
count: ignoreItems.files.size,
},
]
: [];
return [...tagItems, ...folderItems, ...filesItem, ...ignoreFilesItem];
}
return [];
}, [
searchTerm,
@ -560,14 +679,9 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
groupList.notes,
groupList.extensions,
ignoreItems.files,
sortItems,
]);
const sortItems = useCallback((items: GroupItem[]) => {
return [...items].sort((a, b) => {
return a.name.toLowerCase().localeCompare(b.name.toLowerCase());
});
}, []);
const makeSectionItem = useCallback(
(
groupData: Record<string, Array<GroupItem>>,
@ -592,7 +706,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
) => {
const getMatchingFilesFromApp = (patterns: PatternCategory): GroupItem[] => {
return appAllFiles
.filter((file) => shouldIndexFile(file, patterns, null))
.filter((file) => shouldIndexFile(file, patterns, null, true))
.map((file) => ({
id: file.path,
name: file.basename,
@ -783,6 +897,21 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
setActiveState,
]);
const handleCategoryItemClick = useCallback(
(item: CategoryItem) => {
if (item.type === "tag" && item.originalId) {
groupHandlers.click.tag(item.originalId);
} else if (item.type === "folder" && item.originalId) {
groupHandlers.click.folder(item.originalId);
} else if (item.type === "files") {
groupHandlers.click.files();
} else if (item.type === "ignoreFiles") {
groupHandlers.click.ignoreFiles();
}
},
[groupHandlers]
);
const getDisplayTitle = () => {
if (searchTerm) return `Search Results for: "${searchTerm}"`;
if (activeSection === "tags" && activeItem) {
@ -796,7 +925,7 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
return `Extension: ${activeItem}`;
}
if (activeSection === "ignoreFiles") return "Ignore Files";
return "Select a category to view items";
return "All Categories";
};
const handleDeleteItem = (e: React.MouseEvent, item: GroupItem) => {
@ -1003,22 +1132,40 @@ function ContextManage({ initialProject, onSave, onCancel, app }: ContextManageP
<div className="tw-mt-10 tw-text-center tw-text-muted">
{activeSection
? "No items found."
: "Select a category from the sidebar to view items."}
: "No categories found. Add tags, folders, or files using the sidebar."}
</div>
) : (
<div className="tw-space-y-2" style={{ display: "block" }}>
{sortItems(getDisplayItems).map((item) => (
<ItemCard
key={item.id}
item={item}
viewMode="list"
onDelete={
activeSection === "ignoreFiles" || item.isIgnored
? handleDeleteIgnoreItem
: handleDeleteItem
}
/>
))}
{activeSection || searchTerm
? // When a category is selected or a search is performed, display the normal item list.
sortItems(getDisplayItems)
.map((item) =>
!isCategoryItem(item) ? (
<ItemCard
key={item.id}
item={item}
viewMode="list"
onDelete={
activeSection === "ignoreFiles" || item.isIgnored
? handleDeleteIgnoreItem
: handleDeleteItem
}
/>
) : null
)
.filter(Boolean)
: // When no category is selected and no search, display the grouped category list.
getDisplayItems
.map((item) =>
isCategoryItem(item) ? (
<CategoryItemCard
key={item.id}
item={item}
onClick={handleCategoryItemClick}
/>
) : null
)
.filter(Boolean)}
</div>
)}
</ScrollArea>

View file

@ -0,0 +1,95 @@
import React, { useState } from "react";
import { Notice } from "obsidian";
import { Button } from "@/components/ui/button";
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { ModelDisplay } from "@/components/ui/model-display";
import { useSettingsValue, getModelKeyFromModel } from "@/settings/model";
import { checkModelApiKey, err2String } from "@/utils";
import { ChevronDown } from "lucide-react";
interface ModelSelectorProps {
disabled?: boolean;
size?: "sm" | "fit" | "default" | "lg" | "icon";
variant?: "default" | "destructive" | "secondary" | "ghost" | "ghost2" | "link" | "success";
className?: string;
// Always controlled
value: string;
onChange: (modelKey: string) => void;
}
export function ModelSelector({
disabled = false,
size = "fit",
variant = "ghost2",
className,
value,
onChange,
}: ModelSelectorProps) {
const [modelError, setModelError] = useState<string | null>(null);
const settings = useSettingsValue();
const currentModel = settings.activeModels.find(
(model) => model.enabled && getModelKeyFromModel(model) === value
);
return (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant={variant} size={size} disabled={disabled} className={className}>
{modelError ? (
<span className="tw-text-error">Model Load Failed</span>
) : currentModel ? (
<ModelDisplay model={currentModel} iconSize={8} />
) : (
"Select Model"
)}
{!disabled && <ChevronDown className="tw-mt-0.5 tw-size-5" />}
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start">
{settings.activeModels
.filter((model) => model.enabled)
.map((model) => {
const { hasApiKey, errorNotice } = checkModelApiKey(model, settings);
return (
<DropdownMenuItem
key={getModelKeyFromModel(model)}
onSelect={async (event) => {
if (!hasApiKey && errorNotice) {
event.preventDefault();
new Notice(errorNotice);
return;
}
try {
setModelError(null);
onChange(getModelKeyFromModel(model));
} catch (error) {
const msg = `Model switch failed: ` + err2String(error);
setModelError(msg);
new Notice(msg);
// Restore to the last valid model
const lastValidModel = settings.activeModels.find(
(m) => m.enabled && getModelKeyFromModel(m) === value
);
if (lastValidModel) {
onChange(getModelKeyFromModel(lastValidModel));
}
}
}}
className={!hasApiKey ? "tw-cursor-not-allowed tw-opacity-50" : ""}
>
<ModelDisplay model={model} iconSize={12} />
</DropdownMenuItem>
);
})}
</DropdownMenuContent>
</DropdownMenu>
);
}

View file

@ -23,74 +23,77 @@ export const DEFAULT_SYSTEM_PROMPT = `You are Obsidian Copilot, a helpful assist
7. When showing note titles, use [[title]] format and do not wrap them in \` \`.
8. When showing **Obsidian internal** image links, use ![[link]] format and do not wrap them in \` \`.
9. When showing **web** image links, use ![link](url) format and do not wrap them in \` \`.
10. When generating a table, use compact formatting without excessive whitespace.
10. When generating a table, format as github markdown tables, however, for table headings, immediately add ' |' after the table heading.
11. Always respond in the language of the user's query.
12. Do NOT mention the additional context provided such as getCurrentTime and getTimeRangeMs if it's irrelevant to the user message.
13. If the user mentions "tags", it most likely means tags in Obsidian note properties.`;
13. If the user mentions "tags", it most likely means tags in Obsidian note properties.
14. YouTube URLs: If the user provides YouTube URLs in their message, transcriptions will be automatically fetched and provided to you. You don't need to do anything special - just use the transcription content if available.`;
export const COMPOSER_OUTPUT_INSTRUCTIONS = `Return the new note content or canvas JSON in a special JSON format.
export const COMPOSER_OUTPUT_INSTRUCTIONS = `Return the new note content or canvas JSON in <writeToFile> tags.
# Steps to find the the target notes
1. Extract the target note information from user message and find out the note path from the context below.
2. If target note is not specified, use the <active_note> as the target note.
3. If still failed to find the target note or the note path, ask the user to specify the target note.
# JSON Format
Provide the content in JSON format and wrap it in a code block with the following structure:
# Examples
For a single markdown file:
\`\`\`json
{
"type": "composer",
"path": "path/to/file.md",
"content": "The FULL CONTENT of the md note goes here"
}
\`\`\`
Input: Add a new section to note A
Output:
<writeToFile>
<path>path/to/file.md</path>
<content>The FULL CONTENT of the note A with added section goes here</content>
</writeToFile>
For a canvas file:
\`\`\`json
Input: Create a new canvas with "Hello, world!"
Output:
<writeToFile>
<path>path/to/file.canvas</path>
<content>
{
"type": "composer",
"path": "path/to/file.canvas",
"canvas_json": {
"nodes": [
{
"id": "1",
"type": "text",
"text": "Hello, world!",
"x": 0,
"y": 0,
"width": 200,
"height": 50
}
],
"edges": [
{
"id": "e1-2",
"fromNode": "1",
"toNode": "2",
"label": "connects to"
}
]
}
"nodes": [
{
"id": "1",
"type": "text",
"text": "Hello, world!",
"x": 0,
"y": 0,
"width": 200,
"height": 50
}
],
"edges": [
{
"id": "e1-2",
"fromNode": "1",
"toNode": "2",
"label": "connects to"
}
]
}
\`\`\`
</content>
</writeToFile>
# Important
* ALL JSON objects must be complete and valid - ensure all arrays and objects have matching closing brackets
# The content within the <content> tags for canvas files is in JSON format.
* For canvas files, both 'nodes' and 'edges' arrays must be properly closed with ]
* Properly escape all special characters in the content field, especially backticks and quotes
* Prefer to create new files in existing folders or root folder unless the user's request specifies otherwise
* File paths must end with a .md or .canvas extension
* When generating changes on multiple files, output multiple JSON objects
* Each JSON object must be parseable independently
* When generating changes on multiple files, output multiple <writeToFile> tags
* For canvas files:
- Every node must have: id, type, x, y, width, height
- Every edge must have: id, fromNode, toNode
- All IDs must be unique
- Edge fromNode and toNode must reference existing node IDs`;
- Edge fromNode and toNode must reference existing node IDs.
`;
export const NOTE_CONTEXT_PROMPT_TAG = "note_context";
export const SELECTED_TEXT_TAG = "selected_text";
export const VARIABLE_TAG = "variable";
export const VARIABLE_NOTE_TAG = "variable_note";
export const EMBEDDED_PDF_TAG = "embedded_pdf";
export const VAULT_NOTE_TAG = "vault_note";
export const RETRIEVED_DOCUMENT_TAG = "retrieved_document";
export const EMPTY_INDEX_ERROR_MESSAGE =
"Copilot index does not exist. Please index your vault first!\n\n1. Set a working embedding model in QA settings. If it's not a local model, don't forget to set the API key. \n\n2. Click 'Refresh Index for Vault' and wait for indexing to complete. If you encounter the rate limiting error, please turn your request per second down in QA setting.";
export const CHUNK_SIZE = 6000;
@ -587,7 +590,7 @@ export enum DEFAULT_OPEN_AREA {
}
export const COMMAND_IDS = {
APPLY_ADHOC_PROMPT: "apply-adhoc-prompt",
TRIGGER_QUICK_COMMAND: "trigger-quick-command",
CLEAR_LOCAL_COPILOT_INDEX: "clear-local-copilot-index",
CLEAR_COPILOT_CACHE: "clear-copilot-cache",
COUNT_WORD_AND_TOKENS_SELECTION: "count-word-and-tokens-selection",
@ -607,10 +610,11 @@ export const COMMAND_IDS = {
TOGGLE_AUTOCOMPLETE: "toggle-autocomplete",
ADD_SELECTION_TO_CHAT_CONTEXT: "add-selection-to-chat-context",
ADD_CUSTOM_COMMAND: "add-custom-command",
APPLY_CUSTOM_COMMAND: "apply-custom-command",
} as const;
export const COMMAND_NAMES: Record<CommandId, string> = {
[COMMAND_IDS.APPLY_ADHOC_PROMPT]: "Apply ad-hoc custom prompt",
[COMMAND_IDS.TRIGGER_QUICK_COMMAND]: "Trigger quick command",
[COMMAND_IDS.CLEAR_LOCAL_COPILOT_INDEX]: "Clear local Copilot index",
[COMMAND_IDS.CLEAR_COPILOT_CACHE]: "Clear Copilot cache",
[COMMAND_IDS.COUNT_TOTAL_VAULT_TOKENS]: "Count total tokens in your vault",
@ -631,6 +635,7 @@ export const COMMAND_NAMES: Record<CommandId, string> = {
[COMMAND_IDS.TOGGLE_AUTOCOMPLETE]: "Toggle autocomplete",
[COMMAND_IDS.ADD_SELECTION_TO_CHAT_CONTEXT]: "Add selection to chat context",
[COMMAND_IDS.ADD_CUSTOM_COMMAND]: "Add new custom command",
[COMMAND_IDS.APPLY_CUSTOM_COMMAND]: "Apply custom command",
};
export type CommandId = (typeof COMMAND_IDS)[keyof typeof COMMAND_IDS];
@ -707,8 +712,18 @@ export const DEFAULT_SETTINGS: CopilotSettings = {
enableWordCompletion: false,
lastDismissedVersion: null,
passMarkdownImages: true,
enableAutonomousAgent: false,
enableCustomPromptTemplating: true,
suggestedDefaultCommands: false,
autonomousAgentMaxIterations: 4,
autonomousAgentEnabledToolIds: [
"localSearch",
"webSearch",
"pomodoro",
"youtubeTranscription",
"writeToFile",
"replaceInFile",
],
};
export const EVENT_NAMES = {

View file

@ -2,7 +2,7 @@ import { getSelectedTextContexts } from "@/aiParams";
import { ChainType } from "@/chainFactory";
import { FileParserManager } from "@/tools/FileParserManager";
import { TFile, Vault } from "obsidian";
import { NOTE_CONTEXT_PROMPT_TAG } from "./constants";
import { NOTE_CONTEXT_PROMPT_TAG, EMBEDDED_PDF_TAG, SELECTED_TEXT_TAG } from "./constants";
export class ContextProcessor {
private static instance: ContextProcessor;
@ -31,12 +31,15 @@ export class ContextProcessor {
if (pdfFile instanceof TFile) {
try {
const pdfContent = await fileParserManager.parseFile(pdfFile, vault);
content = content.replace(match[0], `\n\nEmbedded PDF (${pdfName}):\n${pdfContent}\n\n`);
content = content.replace(
match[0],
`\n\n<${EMBEDDED_PDF_TAG}>\n<name>${pdfName}</name>\n<content>\n${pdfContent}\n</content>\n</${EMBEDDED_PDF_TAG}>\n\n`
);
} catch (error) {
console.error(`Error processing embedded PDF ${pdfName}:`, error);
content = content.replace(
match[0],
`\n\nEmbedded PDF (${pdfName}): [Error: Could not process PDF]\n\n`
`\n\n<${EMBEDDED_PDF_TAG}>\n<name>${pdfName}</name>\n<error>Could not process PDF</error>\n</${EMBEDDED_PDF_TAG}>\n\n`
);
}
}
@ -106,10 +109,15 @@ export class ContextProcessor {
content = await this.processEmbeddedPDFs(content, vault, fileParserManager);
}
additionalContext += `\n\n <${prompt_tag}> \n Title: [[${note.basename}]]\nPath: ${note.path}\n\n${content}\n</${prompt_tag}>`;
// Get file metadata
const stats = await vault.adapter.stat(note.path);
const ctime = stats ? new Date(stats.ctime).toISOString() : "Unknown";
const mtime = stats ? new Date(stats.mtime).toISOString() : "Unknown";
additionalContext += `\n\n<${prompt_tag}>\n<title>${note.basename}</title>\n<path>${note.path}</path>\n<ctime>${ctime}</ctime>\n<mtime>${mtime}</mtime>\n<content>\n${content}\n</content>\n</${prompt_tag}>`;
} catch (error) {
console.error(`Error processing file ${note.path}:`, error);
additionalContext += `\n\n <${prompt_tag}_error> \n Title: [[${note.basename}]]\nPath: ${note.path}\n\n[Error: Could not process file]\n</${prompt_tag}_error>`;
additionalContext += `\n\n<${prompt_tag}_error>\n<title>${note.basename}</title>\n<path>${note.path}</path>\n<error>[Error: Could not process file]</error>\n</${prompt_tag}_error>`;
}
};
@ -180,12 +188,7 @@ export class ContextProcessor {
let additionalContext = "";
for (const selectedText of selectedTextContexts) {
const lineRange =
selectedText.startLine === selectedText.endLine
? `L${selectedText.startLine}`
: `L${selectedText.startLine}-${selectedText.endLine}`;
additionalContext += `\n\n <selected_text> \n Title: [[${selectedText.noteTitle}]]\nPath: ${selectedText.notePath}\nLines: ${lineRange}\n\n${selectedText.content}\n</selected_text>`;
additionalContext += `\n\n<${SELECTED_TEXT_TAG}>\n<title>${selectedText.noteTitle}</title>\n<path>${selectedText.notePath}</path>\n<start_line>${selectedText.startLine.toString()}</start_line>\n<end_line>${selectedText.endLine.toString()}</end_line>\n<content>\n${selectedText.content}\n</content>\n</${SELECTED_TEXT_TAG}>`;
}
return additionalContext;

View file

@ -22,7 +22,6 @@ jest.mock("./ChatPersistenceManager", () => ({
saveChat: jest.fn().mockResolvedValue({ success: true, path: "/test/path.md" }),
})),
}));
import { ChatManager } from "./ChatManager";
import { MessageRepository } from "./MessageRepository";
import { ContextManager } from "./ContextManager";
@ -133,7 +132,8 @@ describe("ChatManager", () => {
"Hello",
"Hello",
USER_SENDER,
context
context,
undefined
);
expect(mockContextManager.processMessageContext).toHaveBeenCalledWith(
mockMessage,
@ -167,11 +167,17 @@ describe("ChatManager", () => {
await chatManager.sendMessage("Hello", context, ChainType.LLM_CHAIN, true);
// Should have called addMessage with updated context that includes active note
expect(mockMessageRepo.addMessage).toHaveBeenCalledWith("Hello", "Hello", USER_SENDER, {
notes: [mockActiveFile],
urls: [],
selectedTextContexts: [],
});
expect(mockMessageRepo.addMessage).toHaveBeenCalledWith(
"Hello",
"Hello",
USER_SENDER,
{
notes: [mockActiveFile],
urls: [],
selectedTextContexts: [],
},
undefined
);
});
it("should handle case when no active file exists", async () => {
@ -196,7 +202,8 @@ describe("ChatManager", () => {
"Hello",
"Hello",
USER_SENDER,
context
context,
undefined
);
});
@ -515,11 +522,17 @@ describe("ChatManager", () => {
await chatManager.sendMessage("Hello", context, ChainType.LLM_CHAIN, true);
// Verify that active note was added to context
expect(mockMessageRepo.addMessage).toHaveBeenCalledWith("Hello", "Hello", USER_SENDER, {
notes: [mockActiveFile],
urls: [],
selectedTextContexts: [],
});
expect(mockMessageRepo.addMessage).toHaveBeenCalledWith(
"Hello",
"Hello",
USER_SENDER,
{
notes: [mockActiveFile],
urls: [],
selectedTextContexts: [],
},
undefined
);
});
});

View file

@ -24,6 +24,7 @@ export class ChatManager {
private defaultProjectKey = "defaultProjectKey";
private lastKnownProjectId: string | null = null;
private persistenceManager: ChatPersistenceManager;
private onMessageCreatedCallback?: (messageId: string) => void;
constructor(
private messageRepo: MessageRepository,
@ -35,7 +36,7 @@ export class ChatManager {
// Initialize default project repository
this.projectMessageRepos.set(this.defaultProjectKey, messageRepo);
// Initialize persistence manager with default repository
this.persistenceManager = new ChatPersistenceManager(plugin.app, messageRepo);
this.persistenceManager = new ChatPersistenceManager(plugin.app, messageRepo, chainManager);
}
/**
@ -64,11 +65,22 @@ export class ChatManager {
const currentRepo = this.projectMessageRepos.get(projectKey)!;
// Update persistence manager to use current repository
this.persistenceManager = new ChatPersistenceManager(this.plugin.app, currentRepo);
this.persistenceManager = new ChatPersistenceManager(
this.plugin.app,
currentRepo,
this.chainManager
);
return currentRepo;
}
/**
* Set callback for when a message is created (before context processing)
*/
setOnMessageCreatedCallback(callback: (messageId: string) => void): void {
this.onMessageCreatedCallback = callback;
}
/**
* Send a new message with context processing
*/
@ -76,7 +88,8 @@ export class ChatManager {
displayText: string,
context: MessageContext,
chainType: ChainType,
includeActiveNote: boolean = false
includeActiveNote: boolean = false,
content?: any[]
): Promise<string> {
try {
logInfo(`[ChatManager] Sending message: "${displayText}"`);
@ -99,9 +112,15 @@ export class ChatManager {
displayText,
displayText, // Will be updated with processed content
USER_SENDER,
updatedContext
updatedContext,
content
);
// Notify that message was created (for immediate UI update)
if (this.onMessageCreatedCallback) {
this.onMessageCreatedCallback(messageId);
}
// Get the message for context processing
const message = currentRepo.getMessage(messageId);
if (!message) {
@ -177,7 +196,8 @@ export class ChatManager {
async regenerateMessage(
messageId: string,
onUpdateCurrentMessage: (message: string) => void,
onAddMessage: (message: ChatMessage) => void
onAddMessage: (message: ChatMessage) => void,
onTruncate?: () => void
): Promise<boolean> {
try {
logInfo(`[ChatManager] Regenerating message ${messageId}`);
@ -208,6 +228,11 @@ export class ChatManager {
// Truncate messages after the user message
currentRepo.truncateAfter(messageIndex - 1);
// Notify that truncation happened
if (onTruncate) {
onTruncate();
}
// Update chain memory
await this.updateChainMemory();

View file

@ -6,6 +6,7 @@ import { formatDateTime } from "@/utils";
import { USER_SENDER, AI_SENDER } from "@/constants";
import { getSettings } from "@/settings/model";
import { getCurrentProject } from "@/aiParams";
import ChainManager from "@/LLMProviders/chainManager";
/**
* ChatPersistenceManager - Handles saving and loading chat messages
@ -19,7 +20,8 @@ import { getCurrentProject } from "@/aiParams";
export class ChatPersistenceManager {
constructor(
private app: App,
private messageRepo: MessageRepository
private messageRepo: MessageRepository,
private chainManager?: ChainManager
) {}
/**
@ -43,15 +45,26 @@ export class ChatPersistenceManager {
await this.app.vault.createFolder(settings.defaultSaveFolder);
}
const fileName = this.generateFileName(messages, firstMessageEpoch);
const noteContent = this.generateNoteContent(chatContent, firstMessageEpoch, modelKey);
// Check if a file with this epoch already exists
const existingFile = await this.findFileByEpoch(firstMessageEpoch);
let topic: string | undefined;
// Check if the file already exists
const existingFile = this.app.vault.getAbstractFileByPath(fileName);
if (existingFile instanceof TFile) {
if (existingFile) {
// If file exists, preserve the existing topic
const frontmatter = this.app.metadataCache.getFileCache(existingFile)?.frontmatter;
topic = frontmatter?.topic;
} else {
// If new file, generate AI topic
topic = await this.generateAITopic(messages);
}
const fileName = this.generateFileName(messages, firstMessageEpoch, topic);
const noteContent = this.generateNoteContent(chatContent, firstMessageEpoch, modelKey, topic);
if (existingFile) {
// If the file exists, update its content
await this.app.vault.modify(existingFile, noteContent);
logInfo(`[ChatPersistenceManager] Updated existing chat file: ${fileName}`);
logInfo(`[ChatPersistenceManager] Updated existing chat file: ${existingFile.path}`);
} else {
// If the file doesn't exist, create a new one
await this.app.vault.create(fileName, noteContent);
@ -185,33 +198,107 @@ export class ChatPersistenceManager {
return messages;
}
/**
* Find a file by its epoch in the frontmatter
*/
private async findFileByEpoch(epoch: number): Promise<TFile | null> {
const files = await this.getChatHistoryFiles();
for (const file of files) {
const frontmatter = this.app.metadataCache.getFileCache(file)?.frontmatter;
if (frontmatter?.epoch === epoch) {
return file;
}
}
return null;
}
/**
* Generate AI topic for the conversation
*/
private async generateAITopic(messages: ChatMessage[]): Promise<string | undefined> {
if (!this.chainManager) {
return undefined;
}
try {
const chatModel = this.chainManager.chatModelManager.getChatModel();
if (!chatModel) {
return undefined;
}
// Constants for topic generation
const TOPIC_GENERATION_MESSAGE_LIMIT = 6;
const TOPIC_GENERATION_CHAR_LIMIT = 200;
// Get conversation content for topic generation - using reduce for efficiency
const conversationSummary = messages.reduce((acc, m, i) => {
if (i >= TOPIC_GENERATION_MESSAGE_LIMIT) return acc;
return (
acc +
(acc ? "\n" : "") +
`${m.sender}: ${m.message.slice(0, TOPIC_GENERATION_CHAR_LIMIT)}`
);
}, "");
const prompt = `Generate a concise title (max 5 words) for this conversation based on its content. Return only the title without any explanation or quotes.
Conversation:
${conversationSummary}`;
const response = await chatModel.invoke(prompt);
const topic = response.content
.toString()
.trim()
.replace(/^["']|["']$/g, "") // Remove quotes if present
.replace(/[\\/:*?"<>|]/g, "") // Remove invalid filename characters
.slice(0, 50); // Limit length
return topic || undefined;
} catch (error) {
logError("[ChatPersistenceManager] Error generating AI topic:", error);
return undefined;
}
}
/**
* Generate a file name for the chat
*/
private generateFileName(messages: ChatMessage[], firstMessageEpoch: number): string {
private generateFileName(
messages: ChatMessage[],
firstMessageEpoch: number,
topic?: string
): string {
const settings = getSettings();
const formattedDateTime = formatDateTime(new Date(firstMessageEpoch));
const timestampFileName = formattedDateTime.fileName;
// Get the first user message
const firstUserMessage = messages.find((message) => message.sender === USER_SENDER);
// Use provided topic or fall back to first 10 words
let topicForFilename: string;
if (topic) {
topicForFilename = topic;
} else {
// Get the first user message
const firstUserMessage = messages.find((message) => message.sender === USER_SENDER);
// Get the first 10 words from the first user message and sanitize them
const firstTenWords = firstUserMessage
? firstUserMessage.message
.split(/\s+/)
.slice(0, 10)
.join(" ")
.replace(/[\\/:*?"<>|]/g, "") // Remove invalid filename characters
.trim()
: "Untitled Chat";
// Get the first 10 words from the first user message and sanitize them
topicForFilename = firstUserMessage
? firstUserMessage.message
.split(/\s+/)
.slice(0, 10)
.join(" ")
.replace(/[\\/:*?"<>|]/g, "") // Remove invalid filename characters
.trim()
: "Untitled Chat";
}
// Parse the custom format and replace variables
let customFileName = settings.defaultConversationNoteName || "{$date}_{$time}__{$topic}";
// Create the file name (limit to 100 characters to avoid excessively long names)
customFileName = customFileName
.replace("{$topic}", firstTenWords.slice(0, 100).replace(/\s+/g, "_"))
.replace("{$topic}", topicForFilename.slice(0, 100).replace(/\s+/g, "_"))
.replace("{$date}", timestampFileName.split("_")[0])
.replace("{$time}", timestampFileName.split("_")[1]);
@ -231,7 +318,8 @@ export class ChatPersistenceManager {
private generateNoteContent(
chatContent: string,
firstMessageEpoch: number,
modelKey: string
modelKey: string,
topic?: string
): string {
const settings = getSettings();
const currentProject = getCurrentProject();
@ -239,6 +327,7 @@ export class ChatPersistenceManager {
return `---
epoch: ${firstMessageEpoch}
modelKey: ${modelKey}
${topic ? `topic: "${topic}"` : ""}
${currentProject ? `projectId: ${currentProject.id}` : ""}
${currentProject ? `projectName: ${currentProject.name}` : ""}
tags:

View file

@ -0,0 +1,283 @@
import { AI_SENDER, USER_SENDER } from "@/constants";
import { MessageContext } from "@/types/message";
import { TFile } from "obsidian";
import { MessageRepository } from "./MessageRepository";
// Mock the settings module
jest.mock("@/settings/model", () => ({
getSettings: jest.fn(() => ({ debug: false })),
}));
/**
* This test file demonstrates the complete message lifecycle with context notes.
* It serves as both a test and documentation of how messages flow through the system.
*/
describe("Message Lifecycle with Context Notes - Complete Example", () => {
let messageRepository: MessageRepository;
beforeEach(() => {
messageRepository = new MessageRepository();
});
it("should demonstrate complete message lifecycle with context note", () => {
// Step 1: User types a message and attaches a note
const userDisplayText = "Please summarize the key points";
const attachedNote: TFile = {
path: "meeting-notes-2024-01-15.md",
name: "meeting-notes-2024-01-15.md",
basename: "meeting-notes-2024-01-15",
extension: "md",
} as TFile;
const context: MessageContext = {
notes: [attachedNote],
urls: [],
selectedTextContexts: [],
};
// Step 2: Message is stored with basic display text
const messageId = messageRepository.addMessage(
userDisplayText,
userDisplayText, // Initially same as display
USER_SENDER,
context
);
// Verify initial storage
let displayMessages = messageRepository.getDisplayMessages();
expect(displayMessages).toHaveLength(1);
expect(displayMessages[0]).toMatchObject({
message: "Please summarize the key points",
sender: USER_SENDER,
context: {
notes: [
expect.objectContaining({
basename: "meeting-notes-2024-01-15",
}),
],
},
});
// Step 3: Context Manager processes the note and updates processed text
const processedTextWithContext = `Please summarize the key points
<note_context>
<title>meeting-notes-2024-01-15</title>
<path>meeting-notes-2024-01-15.md</path>
<ctime>2024-01-15T10:00:00.000Z</ctime>
<mtime>2024-01-15T14:30:00.000Z</mtime>
<content>
# Team Meeting - January 15, 2024
## Attendees
- John (Product Manager)
- Sarah (Tech Lead)
- Mike (Designer)
## Key Decisions
1. Launch date moved to Q2 2024
2. MVP features: Auth, Dashboard, Analytics
3. Tech stack: React + Node.js + PostgreSQL
## Action Items
- Sarah: Set up CI/CD pipeline by Jan 20
- Mike: Complete dashboard mockups by Jan 22
- John: Finalize user stories by Jan 18
</content>
</note_context>`;
messageRepository.updateProcessedText(messageId, processedTextWithContext);
// Step 4: Verify different views for UI vs LLM
// UI View - shows only what user typed
displayMessages = messageRepository.getDisplayMessages();
expect(displayMessages[0].message).toBe("Please summarize the key points");
// LLM View - includes full context
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toBe(processedTextWithContext);
expect(llmMessages[0].message).toContain("Team Meeting - January 15, 2024");
expect(llmMessages[0].message).toContain("Launch date moved to Q2 2024");
// Step 5: AI responds based on the context
const aiResponse = `Based on the meeting notes, here are the key points:
**Main Decisions:**
Project launch postponed to Q2 2024
MVP will include authentication, dashboard, and analytics features
Technology choices: React frontend, Node.js backend, PostgreSQL database
**Team Responsibilities:**
Sarah (Tech Lead): CI/CD pipeline setup - Due Jan 20
Mike (Designer): Dashboard mockup designs - Due Jan 22
John (Product Manager): User story finalization - Due Jan 18
The team appears to be taking a pragmatic approach with a focused MVP scope and clear task delegation.`;
messageRepository.addMessage(
aiResponse,
aiResponse, // AI messages have same display and processed text
AI_SENDER
);
// Step 6: Verify complete conversation
const finalDisplayMessages = messageRepository.getDisplayMessages();
expect(finalDisplayMessages).toHaveLength(2);
// User message with context badge
expect(finalDisplayMessages[0]).toMatchObject({
message: "Please summarize the key points",
sender: USER_SENDER,
context: {
notes: expect.arrayContaining([
expect.objectContaining({ basename: "meeting-notes-2024-01-15" }),
]),
},
});
// AI response
expect(finalDisplayMessages[1]).toMatchObject({
message: expect.stringContaining("Based on the meeting notes"),
sender: AI_SENDER,
});
// Step 7: Verify what LLM sees for potential follow-up
const llmView = messageRepository.getLLMMessages();
expect(llmView).toHaveLength(2);
// LLM sees user message with full context
expect(llmView[0].message).toContain("Please summarize the key points");
expect(llmView[0].message).toContain("Team Meeting - January 15, 2024");
// LLM sees its own response
expect(llmView[1].message).toContain("Based on the meeting notes");
});
it("should handle message edit with context reprocessing", () => {
// Initial message with context
const initialText = "List the attendees";
const note: TFile = {
path: "meeting.md",
name: "meeting.md",
basename: "meeting",
extension: "md",
} as TFile;
const context: MessageContext = {
notes: [note],
urls: [],
selectedTextContexts: [],
};
// Add initial message with properly formatted context
const messageId = messageRepository.addMessage(
initialText,
`${initialText}
<note_context>
<title>meeting</title>
<path>meeting.md</path>
<ctime>2024-01-10T09:00:00.000Z</ctime>
<mtime>2024-01-10T10:00:00.000Z</mtime>
<content>
Attendees: Alice, Bob, Charlie
</content>
</note_context>`,
USER_SENDER,
context
);
// User edits the message
const editedText = "List the attendees and their roles";
messageRepository.editMessage(messageId, editedText);
// Context is reprocessed (simulated)
const reprocessedText = `${editedText}
<note_context>
<title>meeting</title>
<path>meeting.md</path>
<ctime>2024-01-10T09:00:00.000Z</ctime>
<mtime>2024-01-10T10:30:00.000Z</mtime>
<content>
Attendees: Alice (PM), Bob (Dev), Charlie (QA)
</content>
</note_context>`;
messageRepository.updateProcessedText(messageId, reprocessedText);
// Verify the edit
const displayMessages = messageRepository.getDisplayMessages();
expect(displayMessages[0].message).toBe("List the attendees and their roles");
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain("List the attendees and their roles");
expect(llmMessages[0].message).toContain("Alice (PM), Bob (Dev), Charlie (QA)");
});
it("should maintain context through conversation", () => {
// User asks initial question with context
const context: MessageContext = {
notes: [
{
path: "budget.md",
name: "budget.md",
basename: "budget",
extension: "md",
} as TFile,
],
urls: [],
selectedTextContexts: [],
};
messageRepository.addMessage(
"What is our total budget?",
`What is our total budget?
<note_context>
<title>budget</title>
<path>budget.md</path>
<ctime>2024-01-01T08:00:00.000Z</ctime>
<mtime>2024-01-05T16:00:00.000Z</mtime>
<content>
Q1: $100k
Q2: $150k
Q3: $200k
Q4: $250k
</content>
</note_context>`,
USER_SENDER,
context
);
// AI responds
messageRepository.addMessage(
"Based on the budget document, your total budget for the year is $700k ($100k + $150k + $200k + $250k).",
"Based on the budget document, your total budget for the year is $700k ($100k + $150k + $200k + $250k).",
AI_SENDER
);
// User asks follow-up (no new context needed)
messageRepository.addMessage(
"What percentage increase is Q4 over Q1?",
"What percentage increase is Q4 over Q1?",
USER_SENDER
);
// Verify conversation flow
const messages = messageRepository.getDisplayMessages();
expect(messages).toHaveLength(3);
// First message has context
expect(messages[0].context?.notes).toHaveLength(1);
// Follow-up messages don't need context repeated
expect(messages[1].context).toBeUndefined();
expect(messages[2].context).toBeUndefined();
// But LLM still has access to full conversation
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain("<note_context>");
expect(llmMessages[0].message).toContain("<title>budget</title>");
});
});

View file

@ -0,0 +1,268 @@
import { MessageRepository } from "./MessageRepository";
import { USER_SENDER, SELECTED_TEXT_TAG } from "@/constants";
import { MessageContext } from "@/types/message";
import { TFile } from "obsidian";
/**
* Tests specifically for proper XML tag formatting in context processing
*/
describe("Message Context XML Tag Formatting", () => {
let messageRepository: MessageRepository;
beforeEach(() => {
messageRepository = new MessageRepository();
});
it("should format note context with proper XML tags including metadata", () => {
const userMessage = "Analyze this document";
const note: TFile = {
path: "reports/quarterly-review.md",
name: "quarterly-review.md",
basename: "quarterly-review",
extension: "md",
} as TFile;
const context: MessageContext = {
notes: [note],
urls: [],
selectedTextContexts: [],
};
const processedText = `Analyze this document
<note_context>
<title>quarterly-review</title>
<path>reports/quarterly-review.md</path>
<ctime>2024-03-15T09:00:00.000Z</ctime>
<mtime>2024-03-20T15:30:00.000Z</mtime>
<content>
# Q1 2024 Quarterly Review
## Performance Metrics
- Revenue: $2.5M (110% of target)
- Customer Acquisition: 450 new customers
- Churn Rate: 2.3% (improved from 3.1%)
## Key Achievements
1. Launched new product feature X
2. Expanded to 3 new markets
3. Improved customer satisfaction score to 92%
## Challenges
- Supply chain delays in March
- Increased competition in segment B
</content>
</note_context>`;
messageRepository.addMessage(userMessage, processedText, USER_SENDER, context);
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain("<note_context>");
expect(llmMessages[0].message).toContain("<title>quarterly-review</title>");
expect(llmMessages[0].message).toContain("<path>reports/quarterly-review.md</path>");
expect(llmMessages[0].message).toContain("<ctime>");
expect(llmMessages[0].message).toContain("<mtime>");
expect(llmMessages[0].message).toContain("<content>");
expect(llmMessages[0].message).toContain("</content>");
expect(llmMessages[0].message).toContain("</note_context>");
});
it("should format URL content with proper XML tags", () => {
const userMessage = "Summarize this article";
const context: MessageContext = {
notes: [],
urls: ["https://example.com/ai-trends-2024"],
selectedTextContexts: [],
};
const processedText = `Summarize this article
<url_content>
<url>https://example.com/ai-trends-2024</url>
<content>
# AI Trends for 2024
The landscape of artificial intelligence continues to evolve rapidly. Here are the key trends:
1. **Multimodal AI**: Systems that can process and generate content across text, images, audio, and video.
2. **Edge AI**: Moving AI processing closer to where data is generated, reducing latency and improving privacy.
3. **Explainable AI**: Growing demand for AI systems that can explain their decision-making process.
4. **AI Governance**: Increased focus on ethical AI development and regulatory frameworks.
5. **Specialized AI Chips**: Custom hardware designed specifically for AI workloads becoming mainstream.
</content>
</url_content>`;
messageRepository.addMessage(userMessage, processedText, USER_SENDER, context);
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain("<url_content>");
expect(llmMessages[0].message).toContain("<url>https://example.com/ai-trends-2024</url>");
expect(llmMessages[0].message).toContain("<content>");
expect(llmMessages[0].message).toContain("AI Trends for 2024");
expect(llmMessages[0].message).toContain("</content>");
expect(llmMessages[0].message).toContain("</url_content>");
});
it("should format selected text with proper XML tags", () => {
const userMessage = "Explain this code snippet";
const context: MessageContext = {
notes: [],
urls: [],
selectedTextContexts: [
{
id: "sel-1",
content: `function fibonacci(n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}`,
noteTitle: "Recursion Examples",
notePath: "algorithms/recursion.md",
startLine: 45,
endLine: 49,
},
],
};
const processedText = `Explain this code snippet
<selected_text>
<title>Recursion Examples</title>
<path>algorithms/recursion.md</path>
<start_line>45</start_line>
<end_line>49</end_line>
<content>
function fibonacci(n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}
</content>
</selected_text>`;
messageRepository.addMessage(userMessage, processedText, USER_SENDER, context);
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain(`<${SELECTED_TEXT_TAG}>`);
expect(llmMessages[0].message).toContain("<title>Recursion Examples</title>");
expect(llmMessages[0].message).toContain("<path>algorithms/recursion.md</path>");
expect(llmMessages[0].message).toContain("<start_line>45</start_line>");
expect(llmMessages[0].message).toContain("<end_line>49</end_line>");
expect(llmMessages[0].message).toContain("<content>");
expect(llmMessages[0].message).toContain("function fibonacci");
expect(llmMessages[0].message).toContain("</content>");
expect(llmMessages[0].message).toContain(`</${SELECTED_TEXT_TAG}>`);
});
it("should handle multiple contexts with proper XML structure", () => {
const userMessage = "Compare these resources";
const note: TFile = {
path: "design-patterns.md",
name: "design-patterns.md",
basename: "design-patterns",
extension: "md",
} as TFile;
const context: MessageContext = {
notes: [note],
urls: ["https://patterns.dev/solid-principles"],
selectedTextContexts: [
{
id: "sel-2",
content:
"The Single Responsibility Principle states that a class should have only one reason to change.",
noteTitle: "SOLID Principles",
notePath: "solid/srp.md",
startLine: 10,
endLine: 11,
},
],
};
const processedText = `Compare these resources
<note_context>
<title>design-patterns</title>
<path>design-patterns.md</path>
<ctime>2024-02-01T10:00:00.000Z</ctime>
<mtime>2024-02-15T14:00:00.000Z</mtime>
<content>
# Design Patterns Overview
Common solutions to recurring problems in software design.
</content>
</note_context>
<url_content>
<url>https://patterns.dev/solid-principles</url>
<content>
SOLID Principles Guide - comprehensive overview of all five principles...
</content>
</url_content>
<selected_text>
<title>SOLID Principles</title>
<path>solid/srp.md</path>
<start_line>10</start_line>
<end_line>11</end_line>
<content>
The Single Responsibility Principle states that a class should have only one reason to change.
</content>
</selected_text>`;
messageRepository.addMessage(userMessage, processedText, USER_SENDER, context);
const llmMessages = messageRepository.getLLMMessages();
const message = llmMessages[0].message;
// Verify all three context types are present with proper tags
expect(message).toContain("<note_context>");
expect(message).toContain("</note_context>");
expect(message).toContain("<url_content>");
expect(message).toContain("</url_content>");
expect(message).toContain(`<${SELECTED_TEXT_TAG}>`);
expect(message).toContain(`</${SELECTED_TEXT_TAG}>`);
// Verify proper nesting and structure
expect(message.indexOf("<note_context>")).toBeLessThan(message.indexOf("</note_context>"));
expect(message.indexOf("<url_content>")).toBeLessThan(message.indexOf("</url_content>"));
expect(message.indexOf(`<${SELECTED_TEXT_TAG}>`)).toBeLessThan(
message.indexOf(`</${SELECTED_TEXT_TAG}>`)
);
});
it("should handle error cases with proper XML tags", () => {
const userMessage = "Process this file";
const corruptedNote: TFile = {
path: "corrupted-file.pdf",
name: "corrupted-file.pdf",
basename: "corrupted-file",
extension: "pdf",
} as TFile;
const context: MessageContext = {
notes: [corruptedNote],
urls: [],
selectedTextContexts: [],
};
const processedText = `Process this file
<note_context_error>
<title>corrupted-file</title>
<path>corrupted-file.pdf</path>
<error>[Error: Could not process file]</error>
</note_context_error>`;
messageRepository.addMessage(userMessage, processedText, USER_SENDER, context);
const llmMessages = messageRepository.getLLMMessages();
expect(llmMessages[0].message).toContain("<note_context_error>");
expect(llmMessages[0].message).toContain("<title>corrupted-file</title>");
expect(llmMessages[0].message).toContain("<path>corrupted-file.pdf</path>");
expect(llmMessages[0].message).toContain("<error>[Error: Could not process file]</error>");
expect(llmMessages[0].message).toContain("</note_context_error>");
});
});

View file

@ -30,7 +30,8 @@ export class MessageRepository {
displayText: string,
processedText: string,
sender: string,
context?: MessageContext
context?: MessageContext,
content?: any[]
): string {
const id = this.generateId();
const timestamp = formatDateTime(new Date());
@ -44,6 +45,7 @@ export class MessageRepository {
context,
isVisible: true,
isErrorMessage: false,
content,
};
this.messages.push(message);

View file

@ -0,0 +1,149 @@
import { USER_SENDER } from "@/constants";
import { ChatMessage } from "@/types/message";
import { useCallback, useRef, useState, useEffect } from "react";
interface UseChatScrollingOptions {
chatHistory: ChatMessage[];
}
interface UseChatScrollingReturn {
containerMinHeight: number;
scrollContainerCallbackRef: (node: HTMLDivElement | null) => void;
getMessageKey: (message: ChatMessage, index: number) => string;
}
export const useChatScrolling = ({
chatHistory,
}: UseChatScrollingOptions): UseChatScrollingReturn => {
const [containerMinHeight, setContainerMinHeight] = useState(0);
const scrollContainerRef = useRef<HTMLDivElement | null>(null);
const resizeObserverRef = useRef<ResizeObserver | null>(null);
// Generate consistent message key for DOM identification
const getMessageKey = useCallback((message: ChatMessage, index: number): string => {
return `message-${message.timestamp?.epoch || index}`;
}, []);
// Calculate min-height based on actual last user message size
const calculateDynamicMinHeight = useCallback(() => {
if (!scrollContainerRef.current) return 0;
const messagesContainer = scrollContainerRef.current;
const containerHeight = messagesContainer.clientHeight;
// Find the last user message element to measure its actual height
const lastUserMessageIndex = chatHistory
.map((msg, idx) => ({ msg, idx }))
.filter(({ msg }) => msg.isVisible && msg.sender === USER_SENDER)
.pop()?.idx;
let lastUserMessageHeight = 0;
if (lastUserMessageIndex !== undefined) {
// Try to find the corresponding DOM element
const lastUserMessageKey = getMessageKey(
chatHistory[lastUserMessageIndex],
lastUserMessageIndex
);
const lastUserMessageElement = messagesContainer.querySelector(
`[data-message-key="${lastUserMessageKey}"]`
);
if (lastUserMessageElement) {
lastUserMessageHeight = lastUserMessageElement.getBoundingClientRect().height;
} else {
// Fallback: estimate based on message length (rough approximation)
const messageLength = chatHistory[lastUserMessageIndex].message.length;
const estimatedLines = Math.ceil(messageLength / 80); // ~80 chars per line
lastUserMessageHeight = Math.max(60, estimatedLines * 24); // ~24px per line + padding
}
}
const minHeight = Math.max(100, containerHeight - lastUserMessageHeight);
return minHeight;
}, [chatHistory, getMessageKey]);
// Memoized callback ref that gets called only when the DOM element actually changes
const scrollContainerCallbackRef = useCallback(
(node: HTMLDivElement | null) => {
// Only proceed if the node actually changed
if (node === scrollContainerRef.current) {
return; // Same node, nothing to do
}
// Clean up previous observer if it exists
if (resizeObserverRef.current) {
resizeObserverRef.current.disconnect();
resizeObserverRef.current = null;
}
// Update the ref
scrollContainerRef.current = node;
if (node) {
// Calculate initial height using dynamic measurement
const calculatedMinHeight = calculateDynamicMinHeight();
setContainerMinHeight(calculatedMinHeight);
// Set up ResizeObserver on the messages container
const resizeObserver = new ResizeObserver(() => {
if (scrollContainerRef.current) {
// Recalculate min-height dynamically based on current messages
const newCalculatedMinHeight = calculateDynamicMinHeight();
setContainerMinHeight(newCalculatedMinHeight);
}
});
// Observe the messages container for size changes
resizeObserver.observe(node);
resizeObserverRef.current = resizeObserver;
}
},
[calculateDynamicMinHeight]
);
// Recalculate min-height when chat history changes (new messages)
useEffect(() => {
if (scrollContainerRef.current && chatHistory.length > 0) {
const newCalculatedMinHeight = calculateDynamicMinHeight();
setContainerMinHeight(newCalculatedMinHeight);
}
}, [chatHistory, calculateDynamicMinHeight]);
// Cleanup ResizeObserver on unmount
useEffect(() => {
return () => {
if (resizeObserverRef.current) {
resizeObserverRef.current.disconnect();
}
};
}, []);
// Scroll to bottom function
const scrollToBottom = useCallback((behavior: "smooth" | "instant" = "smooth") => {
if (scrollContainerRef.current) {
scrollContainerRef.current.scrollTo({
top: scrollContainerRef.current.scrollHeight,
behavior,
});
}
}, []);
// Scroll to bottom when component mounts (instant to avoid initial animation)
useEffect(() => {
scrollToBottom("instant");
}, [scrollToBottom]);
// Scroll to bottom when new messages are added
useEffect(() => {
scrollToBottom();
}, [chatHistory.length, scrollToBottom]);
return {
containerMinHeight,
scrollContainerCallbackRef,
getMessageKey,
};
};

View file

@ -77,7 +77,7 @@ describe("Composer Instructions - Integration Tests", () => {
});
});
// Helper function to run a test with a given prompt and check for composer blocks
// Helper function to run a test with a given prompt and check for writeToFile blocks
const testComposerResponse = async (
testName: string,
userPrompt: string,
@ -91,40 +91,41 @@ describe("Composer Instructions - Integration Tests", () => {
userPrompt + "\n\n<output_format>\n" + COMPOSER_OUTPUT_INSTRUCTIONS + "\n</output_format>"
);
const content = result.response.text();
if (expectedBlocks == 0) {
expect(content).not.toContain('"type": "composer"');
expect(content).not.toContain("<writeToFile>");
return;
} else {
let composerBlocks = [];
// When only one block is expected, find the first { and last } and parse the JSON
if (expectedBlocks == 1) {
const start = content.indexOf("{");
const end = content.lastIndexOf("}");
composerBlocks.push(content.substring(start, end + 1));
} else {
const blocks = content.match(
/{\s*"type":\s*"composer",\s*"path":\s*"[^"]+\.(md|canvas)"[\s\S]*?}/g
);
expect(blocks).toBeTruthy();
composerBlocks = blocks!;
expect(composerBlocks.length).toBe(expectedBlocks);
}
// Extract writeToFile blocks
const writeToFileRegex =
/<writeToFile>\s*<path>(.*?)<\/path>\s*<content>([\s\S]*?)<\/content>\s*<\/writeToFile>/g;
const matches = [...content.matchAll(writeToFileRegex)];
// Validate each block is valid JSON
composerBlocks!.forEach((block, index) => {
try {
const json = JSON.parse(block);
expect(json).toHaveProperty("type", "composer");
expect(json).toHaveProperty("path");
expect(json.path).toMatch(/\.(md|canvas)$/);
if (json.path.endsWith(".canvas")) {
expect(json).toHaveProperty("canvas_json");
expect(json.canvas_json).toHaveProperty("nodes");
} else {
expect(json).toHaveProperty("content");
expect(matches.length).toBe(expectedBlocks);
// Validate each block
matches.forEach((match, index) => {
const path = match[1].trim();
const contentStr = match[2].trim();
// Check path ends with .md or .canvas
expect(path).toMatch(/\.(md|canvas)$/);
// For canvas files, validate JSON structure
if (path.endsWith(".canvas")) {
try {
const canvasJson = JSON.parse(contentStr);
expect(canvasJson).toHaveProperty("nodes");
expect(Array.isArray(canvasJson.nodes)).toBe(true);
if (canvasJson.edges) {
expect(Array.isArray(canvasJson.edges)).toBe(true);
}
} catch (e) {
throw new Error(`Invalid canvas JSON in block: ${e.message}`);
}
} catch (e) {
throw new Error(`Invalid JSON in block ${block}: ${e.message}`);
} else {
// For markdown files, just check it has content
expect(contentStr.length).toBeGreaterThan(0);
}
});
}

View file

@ -24,6 +24,7 @@ import {
subscribeToSettingsChange,
} from "@/settings/model";
import { FileParserManager } from "@/tools/FileParserManager";
import { initializeBuiltinTools } from "@/tools/builtinTools";
import {
Editor,
MarkdownView,
@ -40,6 +41,8 @@ import { migrateCommands, suggestDefaultCommands } from "@/commands/migrator";
import { ChatManager } from "@/core/ChatManager";
import { MessageRepository } from "@/core/MessageRepository";
import { ChatUIState } from "@/state/ChatUIState";
import { createQuickCommandContainer } from "@/components/QuickCommand";
import { QUICK_COMMAND_CODE_BLOCK } from "@/commands/constants";
export default class CopilotPlugin extends Plugin {
// Plugin components
@ -67,6 +70,9 @@ export default class CopilotPlugin extends Plugin {
// Core plugin initialization
// Initialize built-in tools with vault access
initializeBuiltinTools(this.app.vault);
this.vectorStoreManager = VectorStoreManager.getInstance();
// Initialize BrevilabsClient
@ -97,6 +103,18 @@ export default class CopilotPlugin extends Plugin {
registerCommands(this, undefined, getSettings());
this.registerMarkdownCodeBlockProcessor(QUICK_COMMAND_CODE_BLOCK, (_, el) => {
createQuickCommandContainer({
plugin: this,
element: el,
});
// Remove parent element class names to clear default code block styling
if (el.parentElement) {
el.parentElement.className = "";
}
});
IntentAnalyzer.initTools(this.app.vault);
this.registerEvent(

View file

@ -85,7 +85,7 @@ export class Mention {
// Append all processed content
processedUrls.forEach((urlData) => {
if (urlData?.processed) {
urlContext += `\n\nContent from ${urlData.original}:\n${urlData.processed}`;
urlContext += `\n\n<url_content>\n<url>${urlData.original}</url>\n<content>\n${urlData.processed}\n</content>\n</url_content>`;
}
});

View file

@ -201,7 +201,7 @@ export async function findRelevantNotes({
return sortedHits
.map(([path, score]) => {
const file = app.vault.getAbstractFileByPath(path);
if (!(file instanceof TFile)) {
if (!(file instanceof TFile) || file.extension !== "md") {
return null;
}
return {

View file

@ -131,12 +131,14 @@ export function getMatchingPatterns(options?: {
* @param file - The file to check.
* @param inclusions - The inclusions patterns.
* @param exclusions - The exclusions patterns.
* @param isProject - Project: Only the included files need to be processed, setting vault embedding All files not excluded need to be processed.
* @returns True if the file should be indexed, false otherwise.
*/
export function shouldIndexFile(
file: TFile,
inclusions: PatternCategory | null,
exclusions: PatternCategory | null
exclusions: PatternCategory | null,
isProject?: boolean
): boolean {
if (exclusions && matchFilePathWithPatterns(file.path, exclusions)) {
return false;
@ -144,6 +146,12 @@ export function shouldIndexFile(
if (inclusions && !matchFilePathWithPatterns(file.path, inclusions)) {
return false;
}
// ProjectOnly the included files need to be processed.
if (isProject && !inclusions) {
return false;
}
return true;
}

View file

@ -7,7 +7,6 @@ import { type ChainType } from "@/chainFactory";
import {
BUILTIN_CHAT_MODELS,
BUILTIN_EMBEDDING_MODELS,
COMPOSER_OUTPUT_INSTRUCTIONS,
DEFAULT_OPEN_AREA,
DEFAULT_SETTINGS,
DEFAULT_SYSTEM_PROMPT,
@ -109,9 +108,12 @@ export interface CopilotSettings {
enableWordCompletion: boolean;
projectList: Array<ProjectConfig>;
passMarkdownImages: boolean;
enableAutonomousAgent: boolean;
enableCustomPromptTemplating: boolean;
/** Whether we have suggested built-in default commands to the user once. */
suggestedDefaultCommands: boolean;
autonomousAgentMaxIterations: number;
autonomousAgentEnabledToolIds: string[];
}
export const settingsStore = createStore();
@ -254,15 +256,27 @@ export function sanitizeSettings(settings: CopilotSettings): CopilotSettings {
sanitizedSettings.enableWordCompletion = DEFAULT_SETTINGS.enableWordCompletion;
}
// Ensure autonomousAgentMaxIterations has a valid value
const autonomousAgentMaxIterations = Number(settingsToSanitize.autonomousAgentMaxIterations);
if (
isNaN(autonomousAgentMaxIterations) ||
autonomousAgentMaxIterations < 4 ||
autonomousAgentMaxIterations > 8
) {
sanitizedSettings.autonomousAgentMaxIterations = DEFAULT_SETTINGS.autonomousAgentMaxIterations;
} else {
sanitizedSettings.autonomousAgentMaxIterations = autonomousAgentMaxIterations;
}
// Ensure autonomousAgentEnabledToolIds is an array
if (!Array.isArray(sanitizedSettings.autonomousAgentEnabledToolIds)) {
sanitizedSettings.autonomousAgentEnabledToolIds =
DEFAULT_SETTINGS.autonomousAgentEnabledToolIds;
}
return sanitizedSettings;
}
export function getComposerOutputPrompt(): string {
const isPlusUser = getSettings().isPlusUser;
return isPlusUser ? COMPOSER_OUTPUT_INSTRUCTIONS : "";
}
export function getSystemPrompt(): string {
const userPrompt = getSettings().userSystemPrompt;
const basePrompt = DEFAULT_SYSTEM_PROMPT;

View file

@ -11,6 +11,7 @@ import {
} from "@/components/ui/select";
import { SettingItem } from "@/components/ui/setting-item";
import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from "@/components/ui/tooltip";
import { ToolSettingsSection } from "./ToolSettingsSection";
import { AUTOCOMPLETE_CONFIG } from "@/constants";
import { cn } from "@/lib/utils";
import { updateSetting, useSettingsValue } from "@/settings/model";
@ -97,6 +98,24 @@ export const CopilotPlusSettings: React.FC = () => {
}}
/>
<div className="tw-pt-4 tw-text-xl tw-font-semibold">Autonomous Agent</div>
<SettingItem
type="switch"
title="Enable Autonomous Agent"
description="Enable autonomous agent mode in Plus chat. The AI will reason step-by-step and decide which tools to use automatically, improving response quality for complex queries."
checked={settings.enableAutonomousAgent}
onCheckedChange={(checked) => {
updateSetting("enableAutonomousAgent", checked);
}}
/>
{settings.enableAutonomousAgent && (
<>
<ToolSettingsSection />
</>
)}
<div className="tw-pt-4 tw-text-xl tw-font-semibold">Autocomplete</div>
<SettingItem

View file

@ -0,0 +1,79 @@
import React from "react";
import { SettingItem } from "@/components/ui/setting-item";
import { ToolRegistry } from "@/tools/ToolRegistry";
import { updateSetting, useSettingsValue } from "@/settings/model";
export const ToolSettingsSection: React.FC = () => {
const settings = useSettingsValue();
const registry = ToolRegistry.getInstance();
const enabledToolIds = new Set(settings.autonomousAgentEnabledToolIds || []);
// Get configurable tools grouped by category
const toolsByCategory = registry.getToolsByCategory();
const configurableTools = registry.getConfigurableTools();
const handleToolToggle = (toolId: string, enabled: boolean) => {
const newEnabledIds = new Set(enabledToolIds);
if (enabled) {
newEnabledIds.add(toolId);
} else {
newEnabledIds.delete(toolId);
}
updateSetting("autonomousAgentEnabledToolIds", Array.from(newEnabledIds));
};
const renderToolsByCategory = () => {
const categories = Array.from(toolsByCategory.entries()).filter(([_, tools]) =>
tools.some((t) => configurableTools.includes(t))
);
return categories.map(([category, tools]) => {
const configurableInCategory = tools.filter((t) => configurableTools.includes(t));
if (configurableInCategory.length === 0) return null;
return (
<React.Fragment key={category}>
{configurableInCategory.map(({ metadata }) => (
<SettingItem
key={metadata.id}
type="switch"
title={metadata.displayName}
description={metadata.description}
checked={enabledToolIds.has(metadata.id)}
onCheckedChange={(checked) => handleToolToggle(metadata.id, checked)}
/>
))}
</React.Fragment>
);
});
};
return (
<>
<SettingItem
type="slider"
title="Max Iterations"
description="Maximum number of reasoning iterations the autonomous agent can perform. Higher values allow for more complex reasoning but may take longer."
value={settings.autonomousAgentMaxIterations ?? 4}
onChange={(value) => {
updateSetting("autonomousAgentMaxIterations", value);
}}
min={4}
max={8}
step={1}
/>
<div className="tw-mt-4 tw-rounded-lg tw-bg-secondary tw-p-4">
<div className="tw-mb-2 tw-text-sm tw-font-medium">Agent Accessible Tools</div>
<div className="tw-mb-4 tw-text-xs tw-text-muted">
Toggle which tools the autonomous agent can use
</div>
<div className="tw-flex tw-flex-col tw-gap-2">{renderToolsByCategory()}</div>
</div>
</>
);
};

View file

@ -15,7 +15,12 @@ import { ChatMessage, MessageContext } from "@/types/message";
export class ChatUIState {
private listeners: Set<() => void> = new Set();
constructor(private chatManager: ChatManager) {}
constructor(private chatManager: ChatManager) {
// Set up callback for immediate UI updates when messages are created
this.chatManager.setOnMessageCreatedCallback(() => {
this.notifyListeners();
});
}
// ================================
// UI STATE MANAGEMENT
@ -55,13 +60,15 @@ export class ChatUIState {
displayText: string,
context: MessageContext,
chainType: ChainType,
includeActiveNote: boolean = false
includeActiveNote: boolean = false,
content?: any[]
): Promise<string> {
const messageId = await this.chatManager.sendMessage(
displayText,
context,
chainType,
includeActiveNote
includeActiveNote,
content
);
this.notifyListeners();
return messageId;
@ -102,6 +109,10 @@ export class ChatUIState {
(message) => {
onAddMessage(message);
this.notifyListeners();
},
() => {
// Notify immediately after truncation
this.notifyListeners();
}
);
if (success) {
@ -203,7 +214,12 @@ export class ChatUIState {
*/
addMessage(message: ChatMessage): void {
if (message.isVisible) {
this.addDisplayMessage(message.message, message.sender, message.id);
// If the message has sources or other metadata, use addFullMessage to preserve them
if (message.sources || message.content) {
this.addFullMessage(message);
} else {
this.addDisplayMessage(message.message, message.sender, message.id);
}
} else {
this.addFullMessage(message);
}

View file

@ -440,6 +440,10 @@ If your plugin does not need CSS, delete this file.
margin: 10px;
}
.tool-call-container {
margin: 0.75rem 0;
}
.message-image-content {
width: 100%;
max-width: 400px;
@ -461,6 +465,16 @@ If your plugin does not need CSS, delete this file.
gap: 10px;
}
/* Custom animations */
@keyframes shimmer {
0% {
transform: translateX(-100%);
}
100% {
transform: translateX(100%);
}
}
.copilot-command-input-container {
width: 90%;
margin: auto;
@ -772,6 +786,11 @@ If your plugin does not need CSS, delete this file.
.chain-select-content [role="menuitem"]:hover {
background: var(--background-modifier-hover);
}
/* Tool call banner responsive styles */
.tool-call-container {
margin: 0.5rem 0;
}
}
.model-card {

View file

@ -76,7 +76,7 @@ const createAndPopulateGroupList = (
exclusionPatterns: PatternCategory | null
): GroupListItem => {
const projectAllFiles = appFiles.filter((file) =>
shouldIndexFile(file, inclusionPatterns, exclusionPatterns)
shouldIndexFile(file, inclusionPatterns, exclusionPatterns, true)
);
// Initialize groups

View file

@ -0,0 +1,475 @@
import {
parseSearchReplaceBlocks,
normalizeLineEndings,
replaceWithLineEndingAwareness,
} from "./ComposerTools";
describe("parseSearchReplaceBlocks", () => {
describe("Standard format", () => {
test("should parse basic SEARCH/REPLACE block with newlines", () => {
const diff = `------- SEARCH
old text here
=======
new text here
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text here",
replaceText: "new text here",
});
});
test("should parse multiline content", () => {
const diff = `------- SEARCH
function oldFunction() {
return "old";
}
=======
function newFunction() {
return "new";
}
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0].searchText).toBe(`function oldFunction() {
return "old";
}`);
expect(result[0].replaceText).toBe(`function newFunction() {
return "new";
}`);
});
});
describe("Flexible format without newlines", () => {
test("should parse compact format without newlines", () => {
const diff = "-------SEARCHold text=======new text+++++++REPLACE";
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
test("should handle minimal markers", () => {
const diff = "---SEARCHtest===replacement+++REPLACE";
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "test",
replaceText: "replacement",
});
});
});
describe("Different line endings", () => {
test("should handle Windows line endings (\\r\\n)", () => {
const diff = "------- SEARCH\r\nold text\r\n=======\r\nnew text\r\n+++++++ REPLACE";
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
test("should handle mixed line endings", () => {
const diff = "------- SEARCH\nold text\r\n=======\nnew text\r\n+++++++ REPLACE";
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
});
describe("Multiple blocks", () => {
test("should parse multiple SEARCH/REPLACE blocks", () => {
const diff = `------- SEARCH
first old text
=======
first new text
+++++++ REPLACE
------- SEARCH
second old text
=======
second new text
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(2);
expect(result[0]).toEqual({
searchText: "first old text",
replaceText: "first new text",
});
expect(result[1]).toEqual({
searchText: "second old text",
replaceText: "second new text",
});
});
test("should parse multiple blocks with different formatting", () => {
const diff = `------- SEARCH
block1 search
=======
block1 replace
+++++++ REPLACE
-------SEARCHblock2 search=======block2 replace+++++++REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(2);
expect(result[0]).toEqual({
searchText: "block1 search",
replaceText: "block1 replace",
});
expect(result[1]).toEqual({
searchText: "block2 search",
replaceText: "block2 replace",
});
});
});
describe("Whitespace handling", () => {
test("should trim whitespace from search and replace text", () => {
const diff = `------- SEARCH
old text with spaces
=======
new text with spaces
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text with spaces",
replaceText: "new text with spaces",
});
});
test("should handle spaces in markers", () => {
const diff = `------- SEARCH
old text
=======
new text
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
});
describe("Special characters", () => {
test("should handle special regex characters in content", () => {
const diff = `------- SEARCH
function test() { return /regex.*pattern/g; }
=======
function test() { return /new.*pattern/g; }
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0].searchText).toBe("function test() { return /regex.*pattern/g; }");
expect(result[0].replaceText).toBe("function test() { return /new.*pattern/g; }");
});
test("should handle Unicode characters", () => {
const diff = `------- SEARCH
=======
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "这是测试文本",
replaceText: "这是新文本",
});
});
});
describe("Empty content", () => {
test("should handle empty search text", () => {
const diff = `------- SEARCH
=======
new content to add
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "",
replaceText: "new content to add",
});
});
test("should handle empty replace text (deletion)", () => {
const diff = `------- SEARCH
content to delete
=======
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "content to delete",
replaceText: "",
});
});
});
describe("Variable marker lengths", () => {
test("should handle different numbers of dashes", () => {
const diff = `----------- SEARCH
old text
=======
new text
+++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
test("should handle different numbers of equals and plus signs", () => {
const diff = `------- SEARCH
old text
===========
new text
+++++++++++ REPLACE`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({
searchText: "old text",
replaceText: "new text",
});
});
});
describe("Edge cases and invalid formats", () => {
test("should return empty array for invalid format", () => {
const diff = "invalid format without proper markers";
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(0);
});
test("should return empty array for incomplete block", () => {
const diff = `------- SEARCH
old text
=======
new text`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(0);
});
test("should handle empty diff string", () => {
const result = parseSearchReplaceBlocks("");
expect(result).toHaveLength(0);
});
test("should handle case sensitivity in markers", () => {
const diff = `------- search
old text
=======
new text
+++++++ replace`;
const result = parseSearchReplaceBlocks(diff);
expect(result).toHaveLength(0);
});
});
});
describe("normalizeLineEndings", () => {
test("should normalize CRLF to LF", () => {
const text = "line1\r\nline2\r\nline3";
const result = normalizeLineEndings(text);
expect(result).toBe("line1\nline2\nline3");
});
test("should normalize CR to LF", () => {
const text = "line1\rline2\rline3";
const result = normalizeLineEndings(text);
expect(result).toBe("line1\nline2\nline3");
});
test("should leave LF unchanged", () => {
const text = "line1\nline2\nline3";
const result = normalizeLineEndings(text);
expect(result).toBe("line1\nline2\nline3");
});
test("should handle mixed line endings", () => {
const text = "line1\r\nline2\nline3\rline4";
const result = normalizeLineEndings(text);
expect(result).toBe("line1\nline2\nline3\nline4");
});
test("should handle empty string", () => {
const result = normalizeLineEndings("");
expect(result).toBe("");
});
test("should handle string without line endings", () => {
const text = "single line text";
const result = normalizeLineEndings(text);
expect(result).toBe("single line text");
});
});
describe("replaceWithLineEndingAwareness", () => {
describe("CRLF files", () => {
test("should preserve CRLF when file predominantly uses CRLF", () => {
const content = "line1\r\nold text\r\nline3\r\n";
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\r\nnew text\r\nline3\r\n");
});
test("should work when search text has different line endings than file", () => {
const content = "line1\r\nold\r\ntext\r\nline3\r\n";
const searchText = "old\ntext"; // LF in search text
const replaceText = "new\ntext"; // LF in replace text
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\r\nnew\r\ntext\r\nline3\r\n"); // Result preserves CRLF
});
test("should handle multiline replacement with CRLF", () => {
const content = "function test() {\r\n return 'old';\r\n}\r\n";
const searchText = "function test() {\n return 'old';\n}";
const replaceText = "function test() {\n return 'new';\n}";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("function test() {\r\n return 'new';\r\n}\r\n");
});
});
describe("LF files", () => {
test("should preserve LF when file predominantly uses LF", () => {
const content = "line1\nold text\nline3\n";
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\nnew text\nline3\n");
});
test("should work when search text has different line endings than file", () => {
const content = "line1\nold\ntext\nline3\n";
const searchText = "old\r\ntext"; // CRLF in search text
const replaceText = "new\r\ntext"; // CRLF in replace text
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\nnew\ntext\nline3\n"); // Result preserves LF
});
});
describe("Mixed line ending handling", () => {
test("should choose CRLF when CRLF is more common", () => {
const content = "line1\r\nline2\r\nold text\nline4\r\n"; // 3 CRLF, 1 LF
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\r\nline2\r\nnew text\r\nline4\r\n");
});
test("should choose LF when LF is more common", () => {
const content = "line1\nline2\nold text\r\nline4\n"; // 3 LF, 1 CRLF
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\nline2\nnew text\nline4\n");
});
test("should default to LF when counts are equal", () => {
const content = "line1\r\nold text\nline3"; // 1 CRLF, 1 LF
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\nnew text\nline3"); // Defaults to LF
});
});
describe("Edge cases", () => {
test("should handle empty search text (edge case behavior)", () => {
const content = "ab";
const searchText = "";
const replaceText = "x";
// Empty string replacement inserts between every character (JavaScript's replaceAll behavior)
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("xaxbx");
});
test("should handle empty replace text (deletion)", () => {
const content = "line1\r\nto delete\r\nline3\r\n";
const searchText = "to delete\r\n";
const replaceText = "";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("line1\r\nline3\r\n");
});
test("should handle no line endings in content", () => {
const content = "old text without line endings";
const searchText = "old text";
const replaceText = "new text";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("new text without line endings");
});
test("should handle multiple occurrences", () => {
const content = "old\r\ntest old\r\nmore old\r\n";
const searchText = "old";
const replaceText = "new";
const result = replaceWithLineEndingAwareness(content, searchText, replaceText);
expect(result).toBe("new\r\ntest new\r\nmore new\r\n");
});
});
});

321
src/tools/ComposerTools.ts Normal file
View file

@ -0,0 +1,321 @@
import { logWarn } from "@/logger";
import { Notice, TFile } from "obsidian";
import { APPLY_VIEW_TYPE } from "@/components/composer/ApplyView";
import { diffTrimmedLines } from "diff";
import { ApplyViewResult } from "@/types";
import { z } from "zod";
import { createTool } from "./SimpleTool";
async function show_preview(file_path: string, content: string): Promise<ApplyViewResult> {
const file = app.vault.getAbstractFileByPath(file_path);
// Check if the current active note is the same as the target note
const activeFile = app.workspace.getActiveFile();
if (file && (!activeFile || activeFile.path !== file_path)) {
// If not, open the target file in the current leaf
await app.workspace.getLeaf().openFile(file as TFile);
new Notice(`Switched to ${file.name}`);
}
let originalContent = "";
if (file) {
originalContent = await app.vault.read(file as TFile);
}
const changes = diffTrimmedLines(originalContent, content, {
newlineIsToken: true,
});
// Return a promise that resolves when the user makes a decision
return new Promise((resolve) => {
// Open the Apply View in a new leaf with the processed content and the callback
const leaf = app.workspace.getLeaf(true);
leaf.setViewState({
type: APPLY_VIEW_TYPE,
active: true,
state: {
changes: changes,
path: file_path,
resultCallback: (result: ApplyViewResult) => {
resolve(result);
},
},
});
});
}
// Define Zod schema for writeToFile
const writeToFileSchema = z.object({
path: z.string().describe(`(Required) The path to the file to write to.
The path must end with explicit file extension, such as .md or .canvas .
Prefer to create new files in existing folders or root folder unless the user's request specifies otherwise.
The path must be relative to the root of the vault.`),
content: z.string().describe(`(Required) The content to write to the file.
ALWAYS provide the COMPLETE intended content of the file, without any truncation or omissions.
You MUST include ALL parts of the file, even if they haven't been modified.
# Rules for Obsidian Canvas content
* For canvas files, both 'nodes' and 'edges' arrays must be properly closed with ]
* Every node must have: id, type, x, y, width, height
* Every edge must have: id, fromNode, toNode
* All IDs must be unique
* Edge fromNode and toNode must reference existing node IDs
# Example content of a canvas file
{
"nodes": [
{
"id": "1",
"type": "text",
"text": "Hello, world!",
"x": 0,
"y": 0,
"width": 200,
"height": 50
}
],
"edges": [
{
"id": "e1-2",
"fromNode": "1",
"toNode": "2",
"label": "connects to"
}
]
}`),
});
const writeToFileTool = createTool({
name: "writeToFile",
description: `Request to write content to a file at the specified path and show the changes in a Change Preview UI.
# Steps to find the the target path
1. Extract the target file information from user message and find out the file path from the context.
2. If target file is not specified, use the active note as the target file.
3. If still failed to find the target file or the file path, ask the user to specify the target file.
`,
schema: writeToFileSchema,
handler: async ({ path, content }) => {
const result = await show_preview(path, content);
// Return tool result and also instruct the model do not retry this tool call for failed result.
return `File change result: ${result}. Do not retry or attempt alternative approaches to modify this file in response to the current user request.`;
},
timeoutMs: 0, // no timeout
});
const replaceInFileSchema = z.object({
path: z
.string()
.describe(
`(Required) The path of the file to modify (relative to the root of the vault and include the file extension).`
),
diff: z.string()
.describe(`(Required) One or more SEARCH/REPLACE blocks. Each block MUST follow this exact format with these exact markers:
\`\`\`
------- SEARCH
[exact content to find, including all whitespace and indentation]
=======
[new content to replace with]
+++++++ REPLACE
\`\`\`
WHEN TO USE THIS TOOL vs writeToFile:
- Use replaceInFile for: small edits, fixing typos, updating specific sections, targeted changes
- Use writeToFile for: creating new files, major rewrites, when you can't identify specific text to replace
CRITICAL RULES:
1. SEARCH content must match EXACTLY - every character, space, and line break
2. Always wrap blocks in triple backticks (\`\`\`)
3. Use the exact markers: "------- SEARCH", "=======", "+++++++ REPLACE"
4. For multiple changes, include multiple SEARCH/REPLACE blocks in order
5. Keep blocks concise - include only the lines being changed plus minimal context
COMMON MISTAKES TO AVOID:
- Wrong: Using different markers like "---- SEARCH" or "SEARCH -------"
- Wrong: Forgetting the triple backticks
- Wrong: Including too many unchanged lines
- Wrong: Not matching whitespace/indentation exactly`),
});
/**
* Normalizes line endings to LF (\n) for consistent string matching.
* This helps avoid issues with mixed line endings (CRLF vs LF).
*/
function normalizeLineEndings(text: string): string {
return text.replace(/\r\n/g, "\n").replace(/\r/g, "\n");
}
/**
* Performs line ending aware text replacement.
* Normalizes line endings for matching but preserves the original line ending style.
*/
function replaceWithLineEndingAwareness(
content: string,
searchText: string,
replaceText: string
): string {
// Detect the predominant line ending style in the original content
const crlfCount = (content.match(/\r\n/g) || []).length;
const lfCount = (content.match(/(?<!\r)\n/g) || []).length;
const usesCrlf = crlfCount > lfCount;
// Normalize for matching
const normalizedContent = normalizeLineEndings(content);
const normalizedSearchText = normalizeLineEndings(searchText);
const normalizedReplaceText = normalizeLineEndings(replaceText);
// Perform replacement on normalized content
const resultNormalized = normalizedContent.replaceAll(
normalizedSearchText,
normalizedReplaceText
);
// Convert back to original line ending style if CRLF was predominant
if (usesCrlf) {
return resultNormalized.replace(/\n/g, "\r\n");
}
return resultNormalized;
}
const replaceInFileTool = createTool({
name: "replaceInFile",
description: `Request to replace sections of content in an existing file using SEARCH/REPLACE blocks that define exact changes to specific parts of the file. This tool should be used when you need to make targeted changes to specific parts of a file.`,
schema: replaceInFileSchema,
handler: async ({ path, diff }: { path: string; diff: string }) => {
const file = app.vault.getAbstractFileByPath(path);
if (!file || !(file instanceof TFile)) {
return `File not found at path: ${path}. Please check the file path and try again.`;
}
try {
const originalContent = await app.vault.read(file);
let modifiedContent = originalContent;
// Parse SEARCH/REPLACE blocks from diff
const searchReplaceBlocks = parseSearchReplaceBlocks(diff);
if (searchReplaceBlocks.length === 0) {
return `No valid SEARCH/REPLACE blocks found in diff. Please use the correct format with ------- SEARCH, =======, and +++++++ REPLACE markers. \n diff: ${diff}`;
}
let changesApplied = 0;
// Apply each SEARCH/REPLACE block in order
for (const block of searchReplaceBlocks) {
const { searchText, replaceText } = block;
// Check if the search text exists in the current content (with line ending normalization)
const normalizedContent = normalizeLineEndings(modifiedContent);
const normalizedSearchText = normalizeLineEndings(searchText);
if (!normalizedContent.includes(normalizedSearchText)) {
logWarn(
`Search text not found in file ${path}. Block ${changesApplied + 1}: "${searchText}".`
);
continue;
}
// Replace all occurrences using line ending aware replacement
const beforeReplace = modifiedContent;
modifiedContent = replaceWithLineEndingAwareness(modifiedContent, searchText, replaceText);
// Check if any replacements were made
if (modifiedContent !== beforeReplace) {
changesApplied++;
}
}
if (originalContent === modifiedContent) {
return `No changes made to ${path}. The search text was not found or replacement resulted in identical content. Call writeToFile instead`;
}
// Show preview of changes
const result = await show_preview(path, modifiedContent);
return `Applied ${changesApplied} SEARCH/REPLACE block(s) (replacing all occurrences). Result: ${result}. Do not call this tool again to modify this file in response to the current user request.`;
} catch (error) {
return `Error performing SEARCH/REPLACE on ${path}: ${error}. Please check the file path and diff format and try again.`;
}
},
timeoutMs: 0, // no timeout
});
/**
* Helper function to parse SEARCH/REPLACE blocks from diff string.
*
* Supports flexible formatting with various line endings and optional newlines.
*
* @param diff - The diff string containing SEARCH/REPLACE blocks
* @returns Array of parsed search/replace text pairs
*
* @example
* // Standard format with newlines:
* const diff1 = `------- SEARCH
* old text here
* =======
* new text here
* +++++++ REPLACE`;
*
* @example
* // Flexible format without newlines:
* const diff2 = `-------SEARCHold text=======new text+++++++REPLACE`;
*
* @example
* // Windows line endings:
* const diff3 = `------- SEARCH\r\nold text\r\n=======\r\nnew text\r\n+++++++ REPLACE`;
*
* @example
* // Multiple blocks:
* const diff4 = `------- SEARCH
* first old text
* =======
* first new text
* +++++++ REPLACE
*
* ------- SEARCH
* second old text
* =======
* second new text
* +++++++ REPLACE`;
*
* Regex patterns match:
* - SEARCH_MARKER: /-{3,}\s*SEARCH\s*(?:\r?\n)?/ "---SEARCH" to "----------- SEARCH\n"
* - SEPARATOR: /(?:\r?\n)?={3,}\s*(?:\r?\n)?/ "===" to "\n========\n"
* - REPLACE_MARKER: /(?:\r?\n)?\+{3,}\s*REPLACE/ "+++REPLACE" to "\n+++++++ REPLACE"
*/
function parseSearchReplaceBlocks(
diff: string
): Array<{ searchText: string; replaceText: string }> {
const blocks: Array<{ searchText: string; replaceText: string }> = [];
const SEARCH_MARKER = /-{3,}\s*SEARCH\s*(?:\r?\n)?/;
const SEPARATOR = /(?:\r?\n)?={3,}\s*(?:\r?\n)?/;
const REPLACE_MARKER = /(?:\r?\n)?\+{3,}\s*REPLACE/;
const blockRegex = new RegExp(
SEARCH_MARKER.source +
"([\\s\\S]*?)" +
SEPARATOR.source +
"([\\s\\S]*?)" +
REPLACE_MARKER.source,
"g"
);
let match;
while ((match = blockRegex.exec(diff)) !== null) {
const searchText = match[1].trim();
const replaceText = match[2].trim();
blocks.push({ searchText, replaceText });
}
return blocks;
}
export {
writeToFileTool,
replaceInFileTool,
parseSearchReplaceBlocks,
normalizeLineEndings,
replaceWithLineEndingAwareness,
};

View file

@ -1,7 +1,7 @@
import { tool } from "@langchain/core/tools";
import { TFile, TFolder } from "obsidian";
import { z } from "zod";
import { getMatchingPatterns, shouldIndexFile } from "@/search/searchUtils";
import { z } from "zod";
import { createTool } from "./SimpleTool";
interface FileTreeNode {
files?: string[];
@ -94,8 +94,11 @@ function buildFileTree(
}
const createGetFileTreeTool = (root: TFolder) =>
tool(
async () => {
createTool({
name: "getFileTree",
description: "Get the file tree as a nested structure of folders and files",
schema: z.void(),
handler: async () => {
// First try building the tree with files included
const tree = buildFileTree(root, true);
@ -118,11 +121,7 @@ const createGetFileTreeTool = (root: TFolder) =>
return prompt + jsonResult;
},
{
name: "getFileTree",
description: "Get the file tree as a nested structure of folders and files",
schema: z.void(),
}
);
isBackground: true,
});
export { createGetFileTreeTool, buildFileTree, type FileTreeNode };

381
src/tools/README.md Normal file
View file

@ -0,0 +1,381 @@
# Tool System Documentation
## Overview
The Copilot tool system uses a centralized registry pattern that makes it easy to add new tools, including future MCP (Model Context Protocol) tools. All tools are managed through a singleton `ToolRegistry` that provides a unified interface for tool discovery, configuration, and execution.
## Tool Prompt Architecture
### How Tool Instructions Flow to the LLM
The system uses a three-layer approach for providing tool instructions to LLMs:
1. **Tool Schema Descriptions** (in tool implementations like `ComposerTools.ts`)
- Defines parameter formats, rules, and validation
- NO XML examples - focuses on data contract only
2. **Custom Prompt Instructions** (in `builtinTools.ts`)
- Contains XML `<use_tool>` invocation examples
- Shows when and how to call the tool
3. **Model-Specific Adaptations** (in `modelAdapter.ts`)
- Last resort for model-specific quirks
### Why Two Layers: Schema vs Custom Instructions
**Key Difference**: Schema descriptions document parameters, while custom instructions provide XML invocation examples.
1. **Clear Separation**
- **Schema**: Parameter documentation (types, formats, rules) - NO XML examples
- **Custom Instructions**: XML `<use_tool>` examples showing how to invoke
2. **MCP Compatibility**
- External tools provide immutable schemas
- We add custom instructions without modifying their code
3. **Example**
```typescript
// Schema (parameter documentation only)
salientTerms: z.array(z.string()).describe("Keywords to find in notes");
// Custom Instructions (XML invocation examples)
customPromptInstructions: `
Example usage:
<use_tool>
<name>localSearch</name>
<query>piano learning</query>
<salientTerms>["piano", "learning"]</salientTerms>
</use_tool>`;
```
### Best Practices
1. **Schema Descriptions: Parameter Documentation Only**
- Document parameter types, formats, and validation rules
- NO XML examples - those belong in custom instructions
- Focus on the data contract
2. **Custom Instructions: XML Examples & Usage Patterns**
- Provide XML `<use_tool>` invocation examples
- Show common usage patterns and edge cases
- Include behavioral guidance (when to use vs other tools)
3. **Model Adapters: Model-Specific Fixes**
- Only for persistent model-specific failures
- Keep minimal and targeted
## Current Implementation
### Core Files
- `src/tools/ToolRegistry.ts` - Central registry for all tools
- `src/tools/builtinTools.ts` - Built-in tool definitions and initialization
- `src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts` - Tool execution in the agent
- `src/settings/v2/components/ToolSettingsSection.tsx` - Settings UI for tool configuration
### Tool Registry Pattern
The `ToolRegistry` is a singleton that manages all tools:
```typescript
class ToolRegistry {
static getInstance(): ToolRegistry;
register(definition: ToolDefinition): void;
registerAll(definitions: ToolDefinition[]): void;
getAllTools(): ToolDefinition[];
getEnabledTools(enabledToolIds: Set<string>, vaultAvailable: boolean): SimpleTool<any, any>[];
getToolsByCategory(): Map<string, ToolDefinition[]>;
getConfigurableTools(): ToolDefinition[];
getToolMetadata(id: string): ToolMetadata | undefined;
clear(): void;
}
```
### Tool Definition Structure
```typescript
interface ToolDefinition {
tool: SimpleTool<any, any>; // The actual tool implementation
metadata: ToolMetadata; // UI and configuration metadata
}
interface ToolMetadata {
id: string; // Unique identifier
displayName: string; // Shown in UI
description: string; // Help text
category: "search" | "time" | "file" | "media" | "mcp" | "custom";
isAlwaysEnabled?: boolean; // If true, not configurable (e.g., time tools)
requiresVault?: boolean; // Needs vault access
customPromptInstructions?: string; // Tool-specific prompts
}
```
## Adding a New Built-in Tool
### 1. Implement the Tool
Create your tool following the `SimpleTool` interface:
```typescript
// Example: New built-in tool
import { z } from "zod";
import { SimpleTool } from "./SimpleTool";
export const myNewTool: SimpleTool<{ input: string }, { result: string }> = {
name: "myNewTool",
description: "Description for the LLM to understand when to use this tool",
schema: z.object({
input: z.string().describe("The input parameter description"),
}),
func: async (params) => {
// Tool implementation
const result = await performOperation(params.input);
return { result };
},
};
```
### 2. Add to Built-in Tools
Update `src/tools/builtinTools.ts`:
```typescript
export const BUILTIN_TOOLS: ToolDefinition[] = [
// ... existing tools ...
{
tool: myNewTool,
metadata: {
id: "myNewTool",
displayName: "My New Tool",
description: "User-friendly description for settings UI",
category: "custom", // Choose appropriate category
// Optional flags:
isAlwaysEnabled: false, // Set true if tool should always be available
requiresVault: true, // Set true if tool needs vault access
customPromptInstructions: "Special instructions for the AI when using this tool",
},
},
];
```
### 3. Update Default Settings (if configurable)
If the tool is configurable (not always-enabled), add its ID to the default enabled tools in `src/constants.ts`:
```typescript
autonomousAgentEnabledToolIds: [
"localSearch",
"webSearch",
"pomodoro",
"youtubeTranscription",
"writeToFile",
"myNewTool" // Add your tool ID here
],
```
## Adding MCP Tools (Future Implementation)
### 1. MCP Tool Wrapper
Create a wrapper to convert MCP tools to the SimpleTool interface:
```typescript
function createMcpToolWrapper(serverName: string, mcpTool: McpTool): SimpleTool<any, any> {
return {
name: `${serverName}_${mcpTool.name}`,
description: mcpTool.description || `MCP tool from ${serverName}`,
schema: convertMcpSchemaToZod(mcpTool.inputSchema),
func: async (params) => {
// Call the MCP server
const result = await mcpHub.callTool(serverName, mcpTool.name, params);
// Convert MCP response to expected format
return {
result: formatMcpResponse(result),
};
},
};
}
```
### 2. Dynamic MCP Tool Registration
Register MCP tools when servers connect:
```typescript
// In your MCP initialization code
export async function registerMcpServerTools(serverName: string, mcpTools: McpTool[]) {
const registry = ToolRegistry.getInstance();
for (const mcpTool of mcpTools) {
registry.register({
tool: createMcpToolWrapper(serverName, mcpTool),
metadata: {
id: `mcp_${serverName}_${mcpTool.name}`,
displayName: mcpTool.displayName || mcpTool.name,
description: mcpTool.description || `MCP tool from ${serverName}`,
category: "mcp",
// MCP tools are user-configurable by default
isAlwaysEnabled: false,
// Add any MCP-specific prompt instructions
customPromptInstructions: mcpTool.systemPrompt,
},
});
}
}
// When MCP server disconnects
export function unregisterMcpServerTools(serverName: string) {
const registry = ToolRegistry.getInstance();
const allTools = registry.getAllTools();
// Remove tools from this server
const toolsToKeep = allTools.filter((t) => !t.metadata.id.startsWith(`mcp_${serverName}_`));
registry.clear();
registry.registerAll(toolsToKeep);
// Re-initialize built-in tools
initializeBuiltinTools(app.vault);
}
```
### 3. Schema Conversion Helper
Convert MCP JSON Schema to Zod schema:
```typescript
function convertMcpSchemaToZod(jsonSchema: any): z.ZodSchema {
// Basic implementation - extend as needed
if (jsonSchema.type === "object") {
const shape: any = {};
for (const [key, prop] of Object.entries(jsonSchema.properties || {})) {
const propSchema = prop as any;
if (propSchema.type === "string") {
shape[key] = z.string();
if (propSchema.description) {
shape[key] = shape[key].describe(propSchema.description);
}
} else if (propSchema.type === "number") {
shape[key] = z.number();
} else if (propSchema.type === "boolean") {
shape[key] = z.boolean();
} else if (propSchema.type === "array") {
shape[key] = z.array(z.any()); // Simplification
} else if (propSchema.type === "object") {
shape[key] = z.object({});
}
// Handle optional properties
if (!jsonSchema.required?.includes(key)) {
shape[key] = shape[key].optional();
}
}
return z.object(shape);
}
// Fallback for other types
return z.any();
}
```
### 4. Settings Storage for MCP Tools
MCP tool preferences are stored in the same array as built-in tools:
```typescript
// When enabling/disabling MCP tools:
function updateMcpToolSetting(toolId: string, enabled: boolean) {
const settings = getSettings();
const enabledIds = new Set(settings.autonomousAgentEnabledToolIds || []);
if (enabled) {
enabledIds.add(toolId);
} else {
enabledIds.delete(toolId);
}
updateSetting("autonomousAgentEnabledToolIds", Array.from(enabledIds));
}
```
## How the System Works
### Tool Discovery Flow
1. **Initialization**: `initializeBuiltinTools()` registers all built-in tools
2. **MCP Connection**: When MCP servers connect, their tools are dynamically registered
3. **Settings UI**: `ToolSettingsSection` component reads from the registry to generate UI
4. **Tool Execution**: `AutonomousAgentChainRunner.getAvailableTools()` filters tools based on settings
### Tool Execution Flow
1. Agent calls `getAvailableTools()` which:
- Gets enabled tool IDs from settings array (`autonomousAgentEnabledToolIds`)
- Calls `registry.getEnabledTools()` to get actual tool implementations
- Filters based on vault availability and user preferences
2. Model adapter receives tool list and:
- Generates tool descriptions for the system prompt
- Includes tool-specific instructions based on enabled tools
3. When tool is called:
- XML parsing extracts tool name and parameters
- Tool is executed via its `func` implementation
- Results are formatted and returned to the agent
## Benefits of This Architecture
1. **Modularity**: Each tool is self-contained with metadata
2. **Extensibility**: New tools can be added without core changes
3. **Backward Compatibility**: Settings structure preserved while supporting new tools
4. **Dynamic UI**: Settings automatically adapt to registered tools
5. **Smart Prompts**: System prompts include only relevant tool instructions
6. **MCP Ready**: Architecture supports dynamic tool registration from external sources
## Testing Your Tools
```typescript
// Test tool registration
const registry = ToolRegistry.getInstance();
registry.clear();
initializeBuiltinTools(vault);
// Verify tool is registered
const allTools = registry.getAllTools();
console.log(
"Registered tools:",
allTools.map((t) => t.metadata.id)
);
// Test with settings
const enabledIds = new Set(["myNewTool", "localSearch"]);
const enabledTools = registry.getEnabledTools(enabledIds, true);
console.log(
"Enabled tools:",
enabledTools.map((t) => t.name)
);
```
This architecture provides a clean, extensible foundation for the tool system while maintaining simplicity and backward compatibility.
## Summary
The tool system's layered approach allows for:
- Clear, comprehensive tool documentation at the schema level
- Model-agnostic instructions that work for most LLMs
- Targeted model-specific adaptations when necessary
- Easy extension for new tools without modifying core infrastructure

View file

@ -0,0 +1,301 @@
import { z } from "zod";
/**
* Schema validation tests for SearchTools
* These tests ensure that tool schemas match expected handler interfaces
* without requiring complex runtime mocks
*/
describe("SearchTools Schema Validation", () => {
describe("localSearchTool schema", () => {
// Define the expected schema based on handler requirements
const localSearchSchema = z.object({
query: z.string().min(1).describe("The search query"),
salientTerms: z.array(z.string()).describe("List of salient terms extracted from the query"),
timeRange: z
.object({
startTime: z.any(), // TimeInfo type
endTime: z.any(), // TimeInfo type
})
.optional()
.describe("Time range for search"),
});
test("validates correct input structure", () => {
const validInput = {
query: "test query",
salientTerms: ["test", "query"],
};
const result = localSearchSchema.safeParse(validInput);
expect(result.success).toBe(true);
});
test("accepts empty salientTerms array", () => {
const validInput = {
query: "what did I do last week",
salientTerms: [],
};
const result = localSearchSchema.safeParse(validInput);
expect(result.success).toBe(true);
});
test("validates input with timeRange", () => {
const validInput = {
query: "meetings last week",
salientTerms: ["meetings"],
timeRange: {
startTime: {
epoch: 1234567890000,
isoString: "2009-02-13T23:31:30.000Z",
userLocaleString: "2/13/2009, 11:31:30 PM",
localDateString: "2009-02-13",
timezoneOffset: 0,
timezone: "UTC",
},
endTime: {
epoch: 1234567900000,
isoString: "2009-02-13T23:31:40.000Z",
userLocaleString: "2/13/2009, 11:31:40 PM",
localDateString: "2009-02-13",
timezoneOffset: 0,
timezone: "UTC",
},
},
};
const result = localSearchSchema.safeParse(validInput);
expect(result.success).toBe(true);
});
test("rejects empty query", () => {
const invalidInput = {
query: "",
salientTerms: ["test"],
};
const result = localSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
if (!result.success) {
expect(result.error.issues[0].path).toContain("query");
}
});
test("rejects missing salientTerms", () => {
const invalidInput = {
query: "test query",
};
const result = localSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
});
test("rejects non-array salientTerms", () => {
const invalidInput = {
query: "test query",
salientTerms: "not an array",
};
const result = localSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
});
});
describe("webSearchTool schema", () => {
// This should match ChatHistoryEntry interface
const webSearchSchema = z.object({
query: z.string().min(1).describe("The search query"),
chatHistory: z
.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
})
)
.describe("Previous conversation turns"),
});
test("validates correct input with proper chatHistory", () => {
const validInput = {
query: "search for TypeScript tutorials",
chatHistory: [
{ role: "user" as const, content: "I want to learn TypeScript" },
{ role: "assistant" as const, content: "I can help you with that!" },
],
};
const result = webSearchSchema.safeParse(validInput);
expect(result.success).toBe(true);
});
test("accepts empty chatHistory", () => {
const validInput = {
query: "TypeScript tutorials",
chatHistory: [],
};
const result = webSearchSchema.safeParse(validInput);
expect(result.success).toBe(true);
});
test("rejects invalid role in chatHistory", () => {
const invalidInput = {
query: "search query",
chatHistory: [
{ role: "system", content: "System message" }, // 'system' not allowed
],
};
const result = webSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
if (!result.success) {
expect(result.error.issues[0].path).toContain("chatHistory");
}
});
test("rejects missing content in chatHistory", () => {
const invalidInput = {
query: "search query",
chatHistory: [{ role: "user" }], // missing content
};
const result = webSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
});
test("rejects non-string content", () => {
const invalidInput = {
query: "search query",
chatHistory: [
{ role: "user", content: 123 }, // content must be string
],
};
const result = webSearchSchema.safeParse(invalidInput);
expect(result.success).toBe(false);
});
test("rejects z.any() chatHistory (regression test)", () => {
// This tests that we're NOT using z.any()
const weakSchema = z.object({
query: z.string(),
chatHistory: z.array(z.any()), // This is what we want to avoid
});
const malformedInput = {
query: "test",
chatHistory: ["just", "strings", 123, null], // Should fail with proper schema
};
// Weak schema would accept this
expect(weakSchema.safeParse(malformedInput).success).toBe(true);
// But our proper schema should reject it
expect(webSearchSchema.safeParse(malformedInput).success).toBe(false);
});
});
describe("indexTool schema", () => {
const indexSchema = z.void();
test("accepts undefined", () => {
const result = indexSchema.safeParse(undefined);
expect(result.success).toBe(true);
});
test("rejects any parameters", () => {
const result = indexSchema.safeParse({ someParam: "value" });
expect(result.success).toBe(false);
});
test("rejects string input", () => {
const result = indexSchema.safeParse("string");
expect(result.success).toBe(false);
});
test("handles empty object with SimpleTool fix", () => {
// This tests the SimpleTool fix for empty objects on void schemas
// The fix converts {} to undefined before validation
const emptyObj = {};
// Direct parse would fail
const directResult = indexSchema.safeParse(emptyObj);
expect(directResult.success).toBe(false);
// But with the SimpleTool fix (simulated here), it should work
const fixedInput = Object.keys(emptyObj).length === 0 ? undefined : emptyObj;
const fixedResult = indexSchema.safeParse(fixedInput);
expect(fixedResult.success).toBe(true);
});
});
describe("Schema type inference", () => {
test("localSearch schema infers correct types", () => {
const localSchema = z.object({
query: z.string().min(1),
salientTerms: z.array(z.string()),
timeRange: z
.object({
startTime: z.any(),
endTime: z.any(),
})
.optional(),
});
type InferredType = z.infer<typeof localSchema>;
// This is a compile-time test - if it compiles, types are correct
const testValue: InferredType = {
query: "test",
salientTerms: ["test"],
// timeRange is optional
};
// TypeScript should enforce these types
const query: string = testValue.query;
const terms: string[] = testValue.salientTerms;
const timeRange = testValue.timeRange;
expect(query).toBe("test");
expect(terms).toEqual(["test"]);
expect(timeRange).toBeUndefined();
// Also validate the schema works
expect(localSchema.safeParse(testValue).success).toBe(true);
});
test("webSearch schema matches ChatHistoryEntry interface", () => {
// Define the expected interface
interface ChatHistoryEntry {
role: "user" | "assistant";
content: string;
}
const webSchema = z.object({
query: z.string().min(1),
chatHistory: z.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
})
),
});
type InferredType = z.infer<typeof webSchema>;
// This ensures the inferred type matches ChatHistoryEntry[]
const testValue: InferredType = {
query: "test",
chatHistory: [] as ChatHistoryEntry[],
};
// Should be assignable to ChatHistoryEntry[]
const history: ChatHistoryEntry[] = testValue.chatHistory;
expect(history).toEqual([]);
// Also validate the schema works
expect(webSchema.safeParse(testValue).success).toBe(true);
});
});
});

View file

@ -9,21 +9,27 @@ import { BrevilabsClient } from "@/LLMProviders/brevilabsClient";
import { HybridRetriever } from "@/search/hybridRetriever";
import VectorStoreManager from "@/search/vectorStoreManager";
import { getSettings } from "@/settings/model";
import { TimeInfo } from "@/tools/TimeTools";
import { ChatHistoryEntry } from "@/utils";
import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { createTool, SimpleTool } from "./SimpleTool";
const localSearchTool = tool(
async ({
timeRange,
query,
salientTerms,
}: {
timeRange?: { startTime: TimeInfo; endTime: TimeInfo };
query: string;
salientTerms: string[];
}) => {
// Define Zod schema for localSearch
const localSearchSchema = z.object({
query: z.string().min(1).describe("The search query"),
salientTerms: z.array(z.string()).describe("List of salient terms extracted from the query"),
timeRange: z
.object({
startTime: z.any(), // TimeInfo type
endTime: z.any(), // TimeInfo type
})
.optional()
.describe("Time range for search"),
});
const localSearchTool = createTool({
name: "localSearch",
description: "Search for notes based on the time range and query",
schema: localSearchSchema,
handler: async ({ timeRange, query, salientTerms }) => {
const indexEmpty = await VectorStoreManager.getInstance().isIndexEmpty();
if (indexEmpty) {
throw new CustomError(EMPTY_INDEX_ERROR_MESSAGE);
@ -71,24 +77,13 @@ const localSearchTool = tool(
return JSON.stringify(formattedResults);
},
{
name: "localSearch",
description: "Search for notes based on the time range and query",
schema: z.object({
timeRange: z
.object({
startTime: z.any(),
endTime: z.any(),
})
.optional(),
query: z.string().describe("The search query"),
salientTerms: z.array(z.string()).describe("List of salient terms extracted from the query"),
}),
}
);
});
const indexTool = tool(
async () => {
const indexTool = createTool({
name: "indexVault",
description: "Index the vault to the Copilot index",
schema: z.void(), // No parameters
handler: async () => {
try {
const indexedCount = await VectorStoreManager.getInstance().indexVaultToVectorStore();
const indexResultPrompt = `Please report whether the indexing was successful.\nIf success is true, just say it is successful. If 0 files is indexed, say there are no new files to index.`;
@ -110,15 +105,29 @@ const indexTool = tool(
});
}
},
{
name: "indexVault",
description: "Index the vault to the Copilot index",
}
);
isBackground: true,
});
// Define Zod schema for webSearch
const webSearchSchema = z.object({
query: z.string().min(1).describe("The search query"),
chatHistory: z
.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
})
)
.describe("Previous conversation turns"),
});
// Add new web search tool
const webSearchTool = tool(
async ({ query, chatHistory }: { query: string; chatHistory: ChatHistoryEntry[] }) => {
const webSearchTool = createTool({
name: "webSearch",
description: "Search the web for information",
schema: webSearchSchema,
isPlusOnly: true,
handler: async ({ query, chatHistory }) => {
try {
// Get standalone question considering chat history
const standaloneQuestion = await getStandaloneQuestion(query, chatHistory);
@ -140,21 +149,7 @@ const webSearchTool = tool(
return "";
}
},
{
name: "webSearch",
description: "Search the web for information",
schema: z.object({
query: z.string().describe("The search query"),
chatHistory: z
.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
})
)
.describe("Previous conversation turns"),
}),
}
);
});
export { indexTool, localSearchTool, webSearchTool };
export type { SimpleTool };

View file

@ -0,0 +1,322 @@
import { z } from "zod";
import { CommonSchemas, createAsyncTool, createTool, extractParametersFromZod } from "./SimpleTool";
describe("SimpleTool Tests", () => {
describe("createTool", () => {
test("creates a basic tool with void schema", async () => {
const tool = createTool({
name: "testTool",
description: "A test tool",
schema: z.void(),
handler: async () => "success",
});
expect(tool.name).toBe("testTool");
expect(tool.description).toBe("A test tool");
const result = await tool.call(undefined);
expect(result).toBe("success");
});
test("creates a tool with object schema", async () => {
const tool = createTool({
name: "greetTool",
description: "Greets a user",
schema: z.object({
name: z.string().describe("User's name"),
age: z.number().optional().describe("User's age"),
}),
handler: async ({ name, age }) => `Hello ${name}, age: ${age ?? "unknown"}`,
});
const result = await tool.call({ name: "John", age: 30 });
expect(result).toBe("Hello John, age: 30");
});
test("validates input parameters", async () => {
const tool = createTool({
name: "strictTool",
description: "A tool with strict validation",
schema: z.object({
email: z.string().email(),
count: z.number().min(1).max(10),
}),
handler: async ({ email, count }) => `${email}: ${count}`,
});
// Valid input
const result = await tool.call({ email: "test@example.com", count: 5 });
expect(result).toBe("test@example.com: 5");
// Invalid email
await expect(tool.call({ email: "invalid", count: 5 })).rejects.toThrow(
"Tool strictTool validation failed"
);
// Count out of range
await expect(tool.call({ email: "test@example.com", count: 15 })).rejects.toThrow(
"Tool strictTool validation failed"
);
});
test("handles empty object for void schema", async () => {
const tool = createTool({
name: "voidTool",
description: "A tool expecting no parameters",
schema: z.void(),
handler: async () => "no params needed",
});
// Should handle empty object gracefully
const result = await tool.call({} as any);
expect(result).toBe("no params needed");
// Should also handle undefined
const result2 = await tool.call(undefined);
expect(result2).toBe("no params needed");
});
test("includes optional properties", async () => {
const tool = createTool({
name: "metaTool",
description: "A tool with metadata",
schema: z.void(),
handler: async () => "done",
timeoutMs: 5000,
isBackground: true,
version: "1.0.0",
deprecated: false,
metadata: { category: "utility" },
});
expect(tool.timeoutMs).toBe(5000);
expect(tool.isBackground).toBe(true);
expect(tool.version).toBe("1.0.0");
expect(tool.deprecated).toBe(false);
expect(tool.metadata).toEqual({ category: "utility" });
});
test("formats Zod validation errors properly", async () => {
const tool = createTool({
name: "complexTool",
description: "A tool with complex validation",
schema: z.object({
user: z.object({
name: z.string().min(2),
email: z.string().email(),
}),
items: z.array(z.string()).min(1),
}),
handler: async () => "success",
});
try {
await tool.call({
user: { name: "J", email: "invalid" },
items: [],
});
fail("Should have thrown validation error");
} catch (error) {
expect(error).toBeInstanceOf(Error);
const message = (error as Error).message;
expect(message).toContain("Tool complexTool validation failed");
expect(message).toContain("user.name:");
expect(message).toContain("user.email:");
expect(message).toContain("items:");
}
});
test("propagates handler errors", async () => {
const tool = createTool({
name: "errorTool",
description: "A tool that throws errors",
schema: z.object({ shouldFail: z.boolean() }),
handler: async ({ shouldFail }) => {
if (shouldFail) {
throw new Error("Handler failed as requested");
}
return "success";
},
});
const result = await tool.call({ shouldFail: false });
expect(result).toBe("success");
await expect(tool.call({ shouldFail: true })).rejects.toThrow("Handler failed as requested");
});
});
describe("CommonSchemas", () => {
test("provides common schema helpers", () => {
expect(CommonSchemas.emptyParams).toBeInstanceOf(z.ZodVoid);
const stringSchema = CommonSchemas.stringParam("Test string");
expect(stringSchema).toBeInstanceOf(z.ZodString);
expect(stringSchema._def.description).toBe("Test string");
const numberSchema = CommonSchemas.numberParam("Test number");
expect(numberSchema).toBeInstanceOf(z.ZodNumber);
const urlSchema = CommonSchemas.url("Website URL");
expect(urlSchema).toBeInstanceOf(z.ZodString);
// Test URL validation
expect(() => urlSchema.parse("https://example.com")).not.toThrow();
expect(() => urlSchema.parse("not-a-url")).toThrow();
});
test("optional string schema works correctly", () => {
const schema = CommonSchemas.optionalString("Optional field");
expect(schema.parse(undefined)).toBeUndefined();
expect(schema.parse("value")).toBe("value");
});
test("non-empty string schema validates minimum length", () => {
const schema = CommonSchemas.nonEmptyString("Required field");
expect(() => schema.parse("")).toThrow();
expect(schema.parse("value")).toBe("value");
});
});
describe("extractParametersFromZod", () => {
test("extracts parameters from object schema", () => {
const schema = z.object({
name: z.string().describe("User's name"),
age: z.number().describe("User's age"),
email: z.string().email(),
});
const params = extractParametersFromZod(schema);
expect(params).toEqual({
name: "User's name",
age: "User's age",
email: "No description",
});
});
test("returns empty object for void schema", () => {
const params = extractParametersFromZod(z.void());
expect(params).toEqual({});
});
test("handles optional fields", () => {
const schema = z.object({
required: z.string().describe("Required field"),
// Note: For optional/nullable/default wrappers, descriptions need to be on the inner type
optional: z.string().describe("Optional field").optional(),
nullable: z.string().describe("Nullable field").nullable(),
withDefault: z.string().describe("Field with default").default("default"),
});
const params = extractParametersFromZod(schema);
expect(params).toEqual({
required: "Required field",
optional: "Optional field",
nullable: "Nullable field",
withDefault: "Field with default",
});
});
test("handles union schemas", () => {
const schema = z.union([
z.object({ type: z.literal("a"), value: z.string() }),
z.object({ type: z.literal("b"), value: z.number() }),
]);
const params = extractParametersFromZod(schema);
expect(params).toEqual({
_union: "Multiple parameter formats supported",
});
});
});
describe("createAsyncTool", () => {
test("creates tool with async validation", async () => {
let validationCalled = false;
const tool = createAsyncTool({
name: "asyncTool",
description: "Tool with async validation",
schema: z.object({
path: z.string(),
}),
asyncValidator: async ({ path }) => {
validationCalled = true;
if (!path.startsWith("/valid/")) {
throw new Error("Invalid path");
}
},
handler: async ({ path }) => `Path ${path} is valid`,
});
// Valid path
const result = await tool.call({ path: "/valid/file.txt" });
expect(result).toBe("Path /valid/file.txt is valid");
expect(validationCalled).toBe(true);
// Invalid path
validationCalled = false;
await expect(tool.call({ path: "/invalid/file.txt" })).rejects.toThrow("Invalid path");
expect(validationCalled).toBe(true);
});
test("works without async validator", async () => {
const tool = createAsyncTool({
name: "simpleTool",
description: "Tool without async validation",
schema: z.object({ value: z.string() }),
handler: async ({ value }) => value.toUpperCase(),
});
const result = await tool.call({ value: "hello" });
expect(result).toBe("HELLO");
});
});
describe("TypeScript type inference", () => {
test("infers correct types for tool handlers", async () => {
// This test mainly ensures TypeScript compilation works correctly
const tool = createTool({
name: "typedTool",
description: "A tool with typed parameters",
schema: z.object({
stringField: z.string(),
numberField: z.number(),
booleanField: z.boolean(),
arrayField: z.array(z.string()),
optionalField: z.string().optional(),
}),
handler: async (args) => {
// TypeScript should infer the correct types here
const upperString: string = args.stringField.toUpperCase();
const doubled: number = args.numberField * 2;
const negated: boolean = !args.booleanField;
const joined: string = args.arrayField.join(",");
const optional: string | undefined = args.optionalField;
return {
upperString,
doubled,
negated,
joined,
optional,
};
},
});
const result = await tool.call({
stringField: "hello",
numberField: 5,
booleanField: true,
arrayField: ["a", "b", "c"],
});
expect(result).toEqual({
upperString: "HELLO",
doubled: 10,
negated: false,
joined: "a,b,c",
optional: undefined,
});
});
});
});

170
src/tools/SimpleTool.ts Normal file
View file

@ -0,0 +1,170 @@
/**
* Simple tool interface to replace LangChain tool wrapper
* Provides a minimal, clean interface for tool definitions
*/
import { z } from "zod";
// Common schema types for reuse
export const CommonSchemas = {
emptyParams: z.void(),
stringParam: (description: string) => z.string().describe(description),
numberParam: (description: string) => z.number().describe(description),
booleanParam: (description: string) => z.boolean().describe(description),
optionalString: (description: string) => z.string().optional().describe(description),
nonEmptyString: (description: string) => z.string().min(1).describe(description),
url: (description: string) => z.string().url().describe(description),
email: (description: string) => z.string().email().describe(description),
} as const;
export interface SimpleTool<TSchema extends z.ZodType = z.ZodVoid, TOutput = any> {
name: string;
description: string;
schema: TSchema;
call: (args: z.infer<TSchema>) => Promise<TOutput>;
timeoutMs?: number;
isBackground?: boolean; // If true, tool execution is not shown to user
isPlusOnly?: boolean; // If true, tool requires Plus subscription
requiresUserMessageContent?: boolean; // If true, tool receives original user message for URL extraction
// Future extensibility fields
version?: string; // Tool version for compatibility
deprecated?: boolean; // Mark tools for future removal
metadata?: Record<string, unknown>; // Additional tool metadata
}
export interface CreateToolOptions<TSchema extends z.ZodType, TOutput = any> {
name: string;
description: string;
schema: TSchema;
handler: (args: z.infer<TSchema>) => Promise<TOutput>;
timeoutMs?: number;
isBackground?: boolean;
isPlusOnly?: boolean;
requiresUserMessageContent?: boolean;
version?: string;
deprecated?: boolean;
metadata?: Record<string, unknown>;
}
/**
* Create a tool with Zod schema validation
* This is the only way to create tools - ensures type safety and validation
*/
export function createTool<TSchema extends z.ZodType, TOutput = any>(
options: CreateToolOptions<TSchema, TOutput>
): SimpleTool<TSchema, TOutput> {
return {
name: options.name,
description: options.description,
schema: options.schema,
call: async (args: any) => {
try {
// Handle empty objects for void schemas
if (
options.schema instanceof z.ZodVoid &&
args &&
typeof args === "object" &&
Object.keys(args).length === 0
) {
args = undefined;
}
// Handle injected parameters for tools that require user message content
let injectedParams = {};
if (options.requiresUserMessageContent && args?._userMessageContent) {
// Extract _userMessageContent before validation
injectedParams = { _userMessageContent: args._userMessageContent };
// Remove it from args to prevent validation errors
args = Object.fromEntries(
Object.entries(args).filter(([key]) => key !== "_userMessageContent")
);
}
// Validate at runtime with better error handling
const validated = options.schema.parse(args);
// Merge validated args with injected params for handler
const handlerArgs = { ...validated, ...injectedParams };
return await options.handler(handlerArgs);
} catch (error) {
if (error instanceof z.ZodError) {
// Format Zod errors for better readability
const formattedErrors = error.errors
.map((e) => `${e.path.join(".")}: ${e.message}`)
.join(", ");
throw new Error(`Tool ${options.name} validation failed: ${formattedErrors}`);
}
throw error;
}
},
timeoutMs: options.timeoutMs,
isBackground: options.isBackground,
isPlusOnly: options.isPlusOnly,
requiresUserMessageContent: options.requiresUserMessageContent,
version: options.version,
deprecated: options.deprecated,
metadata: options.metadata,
};
}
/**
* Extract parameter descriptions from Zod schema for tool description generation
*/
export function extractParametersFromZod(schema: z.ZodType): Record<string, string> {
const descriptions: Record<string, string> = {};
if (schema instanceof z.ZodObject) {
const shape = schema.shape;
for (const [key, value] of Object.entries(shape)) {
const zodField = value as z.ZodType;
descriptions[key] = getZodDescription(zodField) || "No description";
}
} else if (schema instanceof z.ZodVoid) {
// No parameters for void schema
return {};
} else if (schema instanceof z.ZodUnion || schema instanceof z.ZodDiscriminatedUnion) {
// For unions, we could extract common fields or return a special description
descriptions._union = "Multiple parameter formats supported";
}
// Add more schema types as needed
return descriptions;
}
function getZodDescription(schema: z.ZodType): string {
// Handle optional/nullable/default wrappers
if (
schema instanceof z.ZodOptional ||
schema instanceof z.ZodNullable ||
schema instanceof z.ZodDefault
) {
return getZodDescription(schema._def.innerType);
}
// @ts-ignore - accessing private _def property
return schema._def.description || "";
}
/**
* Helper to create a tool that validates input asynchronously
* Useful when validation requires external checks (e.g., file existence)
*/
export function createAsyncTool<TSchema extends z.ZodType, TOutput = any>(
options: CreateToolOptions<TSchema, TOutput> & {
asyncValidator?: (args: z.infer<TSchema>) => Promise<void>;
}
): SimpleTool<TSchema, TOutput> {
const tool = createTool(options);
if (options.asyncValidator) {
const originalCall = tool.call;
tool.call = async (args: any) => {
const validated = options.schema.parse(args);
await options.asyncValidator!(validated);
return originalCall(args);
};
}
return tool;
}

View file

@ -3,7 +3,7 @@ import { getTimeRangeMsTool } from "./TimeTools";
// Helper function to extract the tool function
const getTimeRangeMs = async (timeExpression: string) => {
return await getTimeRangeMsTool.func({ timeExpression });
return await getTimeRangeMsTool.call({ timeExpression });
};
// Helper to verify date ranges
@ -351,11 +351,27 @@ describe("Time Expression Tests", () => {
});
describe("Invalid Expressions", () => {
let consoleWarnSpy: jest.SpyInstance;
beforeEach(() => {
// Mock console.warn to suppress expected warnings
consoleWarnSpy = jest.spyOn(console, "warn").mockImplementation(() => {});
});
afterEach(() => {
// Restore console.warn after each test
consoleWarnSpy.mockRestore();
});
test.each(["invalid time", "", "random text", "week of invalid"])(
"invalid expression: %s",
async (expression) => {
const result = await getTimeRangeMs(expression);
expect(result).toBeUndefined();
// Verify that console.warn was called with the expected message
expect(consoleWarnSpy).toHaveBeenCalledWith(
`Unable to parse time expression: ${expression}`
);
}
);
});

View file

@ -0,0 +1,192 @@
import { DateTime } from "luxon";
import { getCurrentTimeTool, convertTimeBetweenTimezonesTool } from "./TimeTools";
describe("TimeTools Timezone Tests", () => {
// Mock the current date
const mockNow = DateTime.fromObject({
year: 2024,
month: 1,
day: 15,
hour: 14, // 2 PM
minute: 30,
}).setZone("America/Los_Angeles");
beforeAll(() => {
jest.spyOn(DateTime, "now").mockImplementation(() => mockNow as DateTime<true>);
});
afterAll(() => {
jest.restoreAllMocks();
});
describe("getCurrentTimeTool with timezone", () => {
it("should return local time when no timezone is provided", async () => {
const result = await getCurrentTimeTool.call({});
expect(result.timezone).toBeTruthy();
expect(result.epoch).toBeGreaterThan(0);
});
it("should return time at UTC+9 offset (Tokyo)", async () => {
const result = await getCurrentTimeTool.call({ timezoneOffset: "+9" });
expect(result.timezoneOffset).toBe(540); // 9 * 60 minutes
expect(["GMT+9", "UTC+9"]).toContain(result.timezone);
});
it("should handle UTC+0", async () => {
const result = await getCurrentTimeTool.call({ timezoneOffset: "UTC+0" });
expect(result.timezone).toBe("UTC");
expect(result.timezoneOffset).toBe(0);
});
it("should throw error for invalid timezone offset", async () => {
await expect(getCurrentTimeTool.call({ timezoneOffset: "Asia/Tokyo" })).rejects.toThrow(
"Invalid timezone offset format"
);
});
it("should throw error for out of range offset", async () => {
await expect(getCurrentTimeTool.call({ timezoneOffset: "+25" })).rejects.toThrow(
"Invalid timezone offset"
);
});
it("should handle UTC+8 format", async () => {
const result = await getCurrentTimeTool.call({ timezoneOffset: "UTC+8" });
expect(result.timezoneOffset).toBe(480); // 8 * 60 minutes
expect(["GMT+8", "UTC+8"]).toContain(result.timezone);
});
it("should handle negative UTC offset format (GMT-5)", async () => {
const result = await getCurrentTimeTool.call({ timezoneOffset: "GMT-5" });
expect(result.timezoneOffset).toBe(-300); // -5 * 60 minutes
expect(["GMT-5", "UTC-5"]).toContain(result.timezone);
});
it("should handle UTC offset with minutes (+5:30)", async () => {
const result = await getCurrentTimeTool.call({ timezoneOffset: "+5:30" });
expect(result.timezoneOffset).toBe(330); // 5.5 * 60 minutes
expect(["GMT+5:30", "UTC+5:30", "+05:30"]).toContain(result.timezone);
});
});
describe("convertTimeBetweenTimezonesTool", () => {
it("should convert times between timezones correctly", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "18:00", // Use 24-hour format for deterministic parsing
fromOffset: "-8",
toOffset: "+9",
});
// Just verify the conversion happened and timezone is correct
expect(result.originalTime).toBeDefined();
expect(result.convertedTime).toBeDefined();
expect(["GMT+9", "UTC+9"]).toContain(result.timezone);
// Verify the timezone offset is correct (9 hours = 540 minutes)
expect(result.timezoneOffset).toBe(540);
});
it("should convert 9am UTC-5 to UTC+0 (London)", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "9:00 AM",
fromOffset: "-5",
toOffset: "+0",
});
// Just verify the conversion happened
expect(result.originalTime).toBeDefined();
expect(result.convertedTime).toBeDefined();
expect(result.originalTime).not.toEqual(result.convertedTime);
});
it("should handle 24-hour time format", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "18:30",
fromOffset: "UTC+0",
toOffset: "-5",
});
// Verify conversion happened
expect(result.originalTime).toBeDefined();
expect(result.convertedTime).toBeDefined();
// UTC to UTC-5 should show different times
expect(result.originalTime).not.toEqual(result.convertedTime);
});
it("should handle same offset conversion", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "12:00", // Use 24-hour format
fromOffset: "-5",
toOffset: "-5",
});
// When converting to same timezone, times should match
expect(result.originalTime).toBeDefined();
expect(result.convertedTime).toBeDefined();
// Both should have same timezone offset
expect(result.timezoneOffset).toBe(-300); // -5 hours = -300 minutes
});
it("should throw error for invalid time", async () => {
await expect(
convertTimeBetweenTimezonesTool.call({
time: "invalid time",
fromOffset: "-8",
toOffset: "+0",
})
).rejects.toThrow("Could not parse time");
});
it("should handle UTC+10 offset (Australia)", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "10:00 AM",
fromOffset: "-8",
toOffset: "+10",
});
expect(["GMT+10", "UTC+10"]).toContain(result.timezone);
expect(result.convertedTime).toBeDefined();
});
it("should convert times with large offset differences", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "06:00", // Use 24-hour format
fromOffset: "-8",
toOffset: "+9",
});
expect(result.originalTime).toBeDefined();
expect(result.convertedTime).toBeDefined();
// Verify the timezone offset is correct (9 hours = 540 minutes)
expect(result.timezoneOffset).toBe(540);
expect(["GMT+9", "UTC+9"]).toContain(result.timezone);
});
it("should convert between UTC offsets", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "12:00 PM",
fromOffset: "UTC+8",
toOffset: "UTC-5",
});
expect(result).toBeDefined();
expect(result.convertedTime).toBeDefined();
// Verify the offset is correct regardless of the parsed time
expect(result.timezoneOffset).toBe(-300); // UTC-5 is -300 minutes
});
it("should handle mixed offset formats", async () => {
const result = await convertTimeBetweenTimezonesTool.call({
time: "3:00 PM",
fromOffset: "GMT+8",
toOffset: "-5",
});
expect(result).toBeDefined();
expect(result.convertedTime).toBeDefined();
// UTC-5 is -300 minutes
expect(result.timezoneOffset).toBe(-300);
});
});
});

View file

@ -1,8 +1,8 @@
import { tool } from "@langchain/core/tools";
import * as chrono from "chrono-node";
import { DateTime } from "luxon";
import { Notice } from "obsidian";
import { z } from "zod";
import { createTool } from "./SimpleTool";
export interface TimeInfo {
epoch: number;
@ -13,32 +13,103 @@ export interface TimeInfo {
timezone: string;
}
async function getCurrentTime(): Promise<TimeInfo> {
const now = new Date();
const timezoneOffset = now.getTimezoneOffset();
const timezoneAbbr =
new Intl.DateTimeFormat("en", { timeZoneName: "short" })
.formatToParts(now)
.find((part) => part.type === "timeZoneName")?.value || "Unknown";
/**
* Parse timezone offset string to a valid UTC offset
* Supports formats like: "+8", "-5", "+08:00", "-05:30", "UTC+8", "GMT-5"
* Returns a normalized UTC offset string like "UTC+8" or "UTC-5:30"
*/
function parseTimezoneOffset(offset: string): string {
// Extract the numeric offset from various formats
const offsetMatch = offset.match(/^(?:UTC|GMT)?([-+]?\d{1,2})(?::(\d{2}))?$/i);
if (!offsetMatch) {
throw new Error(
`Invalid timezone offset format: ${offset}. Use formats like '+8', '-5', '+5:30', 'UTC+8', 'GMT-5'`
);
}
const hours = parseInt(offsetMatch[1]);
const minutes = parseInt(offsetMatch[2] || "0");
// Validate the offset range
if (Math.abs(hours) > 14 || minutes >= 60) {
throw new Error(
`Invalid timezone offset: ${offset}. Hours must be between -14 and +14, minutes must be less than 60`
);
}
// Create a normalized UTC offset string
const sign = hours >= 0 ? "+" : "";
const minutesStr = minutes > 0 ? `:${minutes.toString().padStart(2, "0")}` : "";
return `UTC${sign}${hours}${minutesStr}`;
}
async function getCurrentTime(timezoneOffset?: string): Promise<TimeInfo> {
let dt: DateTime = DateTime.now();
// If timezone offset is provided, convert to that timezone
if (timezoneOffset) {
try {
const parsedOffset = parseTimezoneOffset(timezoneOffset);
const newDt = dt.setZone(parsedOffset);
if (!newDt.isValid) {
throw new Error(`Failed to apply timezone offset: ${timezoneOffset}`);
}
dt = newDt;
} catch (error) {
throw new Error(`${error.message}`);
}
}
const jsDate = dt.toJSDate();
// Use Luxon's offset which is in minutes and already has the correct sign
const offsetMinutes = dt.offset;
const timezoneAbbr = dt.offsetNameShort || "Unknown";
return {
epoch: Math.floor(now.getTime()),
isoString: now.toISOString(),
userLocaleString: now.toLocaleString(),
localDateString: now.toLocaleDateString("en-CA", {
year: "numeric",
month: "2-digit",
day: "2-digit",
}),
timezoneOffset: -timezoneOffset, // Invert the offset to match common conventions
epoch: Math.floor(jsDate.getTime()),
isoString: jsDate.toISOString(),
userLocaleString: dt.toLocaleString(DateTime.DATETIME_FULL),
localDateString: dt.toISODate() || "",
timezoneOffset: offsetMinutes,
timezone: timezoneAbbr,
};
}
const getCurrentTimeTool = tool(async () => getCurrentTime(), {
const getCurrentTimeTool = createTool({
name: "getCurrentTime",
description: "Get the current time in various formats, including timezone information",
schema: z.object({}), // No input required
description:
"Get the current time in local timezone or at a specified UTC offset. Returns epoch time, ISO string, and formatted strings.",
schema: z.object({
timezoneOffset: z
.string()
.optional()
.describe(
`Optional UTC offset. IMPORTANT: Must be a numeric offset, NOT a timezone name.
EXAMPLES OF CORRECT USAGE:
- "what time is it" No parameter (uses local time)
- "what time is it in Tokyo" timezoneOffset: "+9"
- "what time is it in Beijing" timezoneOffset: "+8"
- "what time is it in New York" timezoneOffset: "-5" (or "-4" during DST)
- "what time is it in Mumbai" timezoneOffset: "+5:30"
SUPPORTED FORMATS:
- Simple: "+8", "-5", "+5:30"
- With prefix: "UTC+8", "GMT-5", "UTC+5:30"
COMMON TIMEZONE OFFSETS:
- Tokyo: UTC+9
- Beijing/Singapore: UTC+8
- Mumbai: UTC+5:30
- Dubai: UTC+4
- London: UTC+0 (UTC+1 during BST)
- New York: UTC-5 (UTC-4 during DST)
- Los Angeles: UTC-8 (UTC-7 during DST)`
),
}),
handler: async ({ timezoneOffset }) => getCurrentTime(timezoneOffset),
isBackground: true,
});
const monthNames = {
@ -387,40 +458,44 @@ function getTimeRangeMs(timeExpression: string) {
function convertToTimeInfo(dateTime: DateTime): TimeInfo {
const jsDate = dateTime.toJSDate();
const timezoneOffset = jsDate.getTimezoneOffset();
const timezoneAbbr =
new Intl.DateTimeFormat("en", { timeZoneName: "short" })
.formatToParts(jsDate)
.find((part) => part.type === "timeZoneName")?.value || "Unknown";
// Use Luxon's offset which is in minutes and already has the correct sign
const offsetMinutes = dateTime.offset;
const timezoneAbbr = dateTime.offsetNameShort || "Unknown";
return {
epoch: Math.floor(jsDate.getTime()),
isoString: jsDate.toISOString(),
userLocaleString: jsDate.toLocaleString(),
localDateString: jsDate.toLocaleDateString("en-CA", {
year: "numeric",
month: "2-digit",
day: "2-digit",
}),
timezoneOffset: -timezoneOffset,
userLocaleString: dateTime.toLocaleString(DateTime.DATETIME_FULL),
localDateString: dateTime.toISODate() || "",
timezoneOffset: offsetMinutes,
timezone: timezoneAbbr,
};
}
const getTimeRangeMsTool = tool(
async ({ timeExpression }: { timeExpression: string }) => getTimeRangeMs(timeExpression),
{
name: "getTimeRangeMs",
description: "Get a time range in milliseconds based on a natural language time expression",
schema: z.object({
timeExpression: z
.string()
.describe(
"A natural language time expression (e.g., 'last week', 'from July 1 to July 15')"
),
}),
}
);
const getTimeRangeMsTool = createTool({
name: "getTimeRangeMs",
description: "Convert natural language time expressions to date ranges for use with localSearch",
schema: z.object({
timeExpression: z.string()
.describe(`Natural language time expression to convert to a date range.
COMMON EXPRESSIONS:
- Relative past: "yesterday", "last week", "last month", "last year"
- Relative ranges: "this week", "this month", "this year"
- Specific dates: "July 1", "July 1 2023", "2023-07-01"
- Date ranges: "from July 1 to July 15", "between May and June"
- Time periods: "last 7 days", "past 30 days", "previous 3 months"
IMPORTANT: This tool is typically used as the first step before localSearch when searching notes by time.
EXAMPLE WORKFLOW:
1. User: "what did I do last week"
2. First call getTimeRangeMs with timeExpression: "last week"
3. Then use the returned time range with localSearch`),
}),
handler: async ({ timeExpression }) => getTimeRangeMs(timeExpression),
isBackground: true,
});
function getTimeInfoByEpoch(epoch: number): TimeInfo {
// Check if the epoch is in seconds (10 digits) or milliseconds (13 digits)
@ -429,17 +504,15 @@ function getTimeInfoByEpoch(epoch: number): TimeInfo {
return convertToTimeInfo(dateTime);
}
const getTimeInfoByEpochTool = tool(
async ({ epoch }: { epoch: number }) => getTimeInfoByEpoch(epoch),
{
name: "getTimeInfoByEpoch",
description:
"Convert a Unix timestamp (in seconds or milliseconds) to detailed time information",
schema: z.object({
epoch: z.number().describe("Unix timestamp in seconds or milliseconds"),
}),
}
);
const getTimeInfoByEpochTool = createTool({
name: "getTimeInfoByEpoch",
description: "Convert a Unix timestamp (in seconds or milliseconds) to detailed time information",
schema: z.object({
epoch: z.number().describe("Unix timestamp in seconds or milliseconds"),
}),
handler: async ({ epoch }) => getTimeInfoByEpoch(epoch),
isBackground: true,
});
function parseTimeInterval(interval: string): number {
const match = interval.match(/^(\d+)\s*(s|sec|seconds?|m|min|minutes?|h|hr|hours?)$/i);
@ -482,21 +555,102 @@ async function startPomodoro(interval = "25min"): Promise<void> {
});
}
const pomodoroTool = tool(
async ({ interval = "25min" }: { interval?: string }) => {
const pomodoroTool = createTool({
name: "startPomodoro",
description: "Start a Pomodoro timer with a customizable interval",
schema: z.object({
interval: z
.string()
.default("25min")
.describe("Time interval (e.g., '25min', '5s', '1h'). Default is 25min."),
}),
handler: async ({ interval }) => {
startPomodoro(interval);
return `Pomodoro timer started. It will end in ${interval}.`;
},
{
name: "startPomodoro",
description: "Start a Pomodoro timer with a customizable interval",
schema: z.object({
interval: z
.string()
.optional()
.describe("Time interval (e.g., '25min', '5s', '1h'). Default is 25min."),
}),
}
);
});
export { getCurrentTimeTool, getTimeInfoByEpochTool, getTimeRangeMsTool, pomodoroTool };
/**
* Convert a time from one UTC offset to another
* @param time - Time expression like "6pm", "18:00", "3:30 PM"
* @param fromOffset - Source UTC offset (e.g., "+8", "-5", "UTC+8")
* @param toOffset - Target UTC offset (e.g., "+9", "-5", "UTC+9")
* @returns Time information in the target timezone
*/
async function convertTimeBetweenTimezones(
time: string,
fromOffset: string,
toOffset: string
): Promise<TimeInfo & { originalTime: string; convertedTime: string }> {
// Parse timezone offsets
const sourceTz = parseTimezoneOffset(fromOffset);
const targetTz = parseTimezoneOffset(toOffset);
try {
// Parse the time string using chrono
const baseDate = DateTime.now().setZone(sourceTz);
const parsedDate = chrono.parseDate(time, baseDate.toJSDate());
if (!parsedDate) {
throw new Error(`Could not parse time: ${time}`);
}
// Create DateTime interpreting the parsed date as already being in source timezone
const sourceDt = DateTime.fromJSDate(parsedDate, { zone: sourceTz });
// Convert to target timezone
const targetDt = sourceDt.setZone(targetTz);
if (!targetDt.isValid) {
throw new Error(`Invalid timezone conversion`);
}
const jsDate = targetDt.toJSDate();
// Use Luxon's offset which is in minutes and already has the correct sign
const offsetMinutes = targetDt.offset;
return {
epoch: Math.floor(jsDate.getTime()),
isoString: jsDate.toISOString(),
userLocaleString: targetDt.toLocaleString(DateTime.DATETIME_FULL),
localDateString: targetDt.toISODate() || "",
timezoneOffset: offsetMinutes,
timezone: targetDt.offsetNameShort || targetTz,
originalTime: sourceDt.toLocaleString(DateTime.TIME_SIMPLE) + " " + sourceDt.offsetNameShort,
convertedTime: targetDt.toLocaleString(DateTime.TIME_SIMPLE) + " " + targetDt.offsetNameShort,
};
} catch (error) {
throw new Error(`Failed to convert time: ${error.message}`);
}
}
const convertTimeBetweenTimezonesTool = createTool({
name: "convertTimeBetweenTimezones",
description: "Convert a specific time from one timezone to another using UTC offsets",
schema: z.object({
time: z.string().describe(`Time to convert. Supports various formats:
- 12-hour: "6pm", "3:30 PM", "11:45 am"
- 24-hour: "18:00", "15:30", "23:45"
- Relative: "noon", "midnight"`),
fromOffset: z.string().describe(`Source UTC offset. Must be numeric, not timezone name.
Examples: "-8" for PT, "+0" for London, "+8" for Beijing`),
toOffset: z.string().describe(`Target UTC offset. Must be numeric, not timezone name.
Examples: "+9" for Tokyo, "-5" for NY, "+5:30" for Mumbai
EXAMPLE USAGE:
- "what time is 6pm PT in Tokyo" time: "6pm", fromOffset: "-8", toOffset: "+9"
- "convert 3:30 PM EST to London time" time: "3:30 PM", fromOffset: "-5", toOffset: "+0"
- "what is 9am Beijing time in New York" time: "9am", fromOffset: "+8", toOffset: "-5"`),
}),
handler: async ({ time, fromOffset, toOffset }) =>
convertTimeBetweenTimezones(time, fromOffset, toOffset),
isBackground: true,
});
export {
getCurrentTimeTool,
getTimeInfoByEpochTool,
getTimeRangeMsTool,
pomodoroTool,
convertTimeBetweenTimezonesTool,
};

128
src/tools/ToolRegistry.ts Normal file
View file

@ -0,0 +1,128 @@
import { SimpleTool } from "./SimpleTool";
/**
* Tool metadata for registration and UI display
*/
export interface ToolMetadata {
id: string;
displayName: string;
description: string;
category: "search" | "time" | "file" | "media" | "mcp" | "custom";
isAlwaysEnabled?: boolean; // Tools that are always available (e.g., time tools)
requiresVault?: boolean; // Tools that need vault access
customPromptInstructions?: string; // Optional custom instructions for this tool
}
/**
* Complete tool definition including implementation and metadata
*/
export interface ToolDefinition {
tool: SimpleTool<any, any>;
metadata: ToolMetadata;
}
/**
* Central registry for all tools available to the autonomous agent
*/
export class ToolRegistry {
private static instance: ToolRegistry;
private tools: Map<string, ToolDefinition> = new Map();
private constructor() {}
static getInstance(): ToolRegistry {
if (!ToolRegistry.instance) {
ToolRegistry.instance = new ToolRegistry();
}
return ToolRegistry.instance;
}
/**
* Register a tool with the registry
*/
register(definition: ToolDefinition): void {
this.tools.set(definition.metadata.id, definition);
}
/**
* Register multiple tools at once
*/
registerAll(definitions: ToolDefinition[]): void {
definitions.forEach((def) => this.register(def));
}
/**
* Get all registered tools
*/
getAllTools(): ToolDefinition[] {
return Array.from(this.tools.values());
}
/**
* Get tools filtered by enabled status
*/
getEnabledTools(enabledToolIds: Set<string>, vaultAvailable: boolean): SimpleTool<any, any>[] {
const enabledTools: SimpleTool<any, any>[] = [];
for (const [id, definition] of this.tools) {
const { metadata, tool } = definition;
// Always include tools marked as always enabled
if (metadata.isAlwaysEnabled) {
// Skip vault-required tools if vault is not available
if (!metadata.requiresVault || vaultAvailable) {
enabledTools.push(tool);
}
continue;
}
// Include user-enabled tools
if (enabledToolIds.has(id)) {
// Skip vault-required tools if vault is not available
if (!metadata.requiresVault || vaultAvailable) {
enabledTools.push(tool);
}
}
}
return enabledTools;
}
/**
* Get tool metadata by category for UI organization
*/
getToolsByCategory(): Map<string, ToolDefinition[]> {
const byCategory = new Map<string, ToolDefinition[]>();
for (const definition of this.tools.values()) {
const category = definition.metadata.category;
if (!byCategory.has(category)) {
byCategory.set(category, []);
}
byCategory.get(category)!.push(definition);
}
return byCategory;
}
/**
* Get configurable tools (excludes always-enabled tools)
*/
getConfigurableTools(): ToolDefinition[] {
return Array.from(this.tools.values()).filter((def) => !def.metadata.isAlwaysEnabled);
}
/**
* Get tool metadata by ID
*/
getToolMetadata(id: string): ToolMetadata | undefined {
return this.tools.get(id)?.metadata;
}
/**
* Clear the registry (useful for testing)
*/
clear(): void {
this.tools.clear();
}
}

View file

@ -0,0 +1,401 @@
/**
* Format tool results for display in the UI
* Each formatter should return a user-friendly representation of the tool result
*/
export class ToolResultFormatter {
/**
* Try to parse JSON string, returns array of parsed objects
* @param json JSON string to parse
* @returns Array containing parsed object(s), or empty array if parsing fails
*/
private static tryParseJson(json: string): any[] {
try {
const parsed = JSON.parse(json);
return Array.isArray(parsed) ? parsed : [parsed];
} catch {
return [];
}
}
static format(toolName: string, result: string): string {
try {
// Try to parse the result as JSON first
let parsedResult: any;
try {
parsedResult = JSON.parse(result);
} catch {
// If not JSON, use the raw string
parsedResult = result;
}
// Route to specific formatter based on tool name
switch (toolName) {
case "localSearch":
return this.formatLocalSearch(parsedResult);
case "webSearch":
return this.formatWebSearch(parsedResult);
case "simpleYoutubeTranscriptionTool":
case "youtubeTranscription":
return this.formatYoutubeTranscription(parsedResult);
case "writeToFile":
return this.formatWriteToFile(parsedResult);
case "replaceInFile":
return this.formatReplaceInFile(parsedResult);
default:
// For all other tools, return the raw result
return result;
}
} catch {
// If formatting fails, return the original result
return result;
}
}
private static formatLocalSearch(result: any): string {
// If already formatted or not a string, return as is
if (typeof result !== "string") {
return typeof result === "object" ? JSON.stringify(result, null, 2) : String(result);
}
// Check if it looks like JSON (array or object)
const trimmedResult = result.trim();
if (!trimmedResult.startsWith("[") && !trimmedResult.startsWith("{")) {
return result;
}
// Try standard JSON parsing first
let searchResults = this.tryParseJson(result);
// If parsing failed, try regex extraction for malformed JSON
try {
if (searchResults.length === 0) {
// Match individual JSON objects (works for arrays or malformed JSON)
const objectRegex = /\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}/g;
const matches = result.match(objectRegex);
if (matches) {
const regexResults: any[] = [];
for (const match of matches) {
try {
// Parse each object individually
const obj = JSON.parse(match);
regexResults.push(obj);
} catch {
// Skip malformed objects
}
}
searchResults = regexResults;
}
}
if (searchResults.length === 0) {
// Fallback: try to extract key information using regex
const titleRegex = /"title"\s*:\s*"([^"]+)"/g;
const pathRegex = /"path"\s*:\s*"([^"]+)"/g;
const scoreRegex = /"score"\s*:\s*([\d.]+)/g;
let titleMatch;
const results = [];
while ((titleMatch = titleRegex.exec(result))) {
const title = titleMatch[1];
pathRegex.lastIndex = titleMatch.index;
const pathMatch = pathRegex.exec(result);
scoreRegex.lastIndex = titleMatch.index;
const scoreMatch = scoreRegex.exec(result);
results.push({
title: title,
path: pathMatch ? pathMatch[1] : "",
score: scoreMatch ? parseFloat(scoreMatch[1]) : 0,
content: "", // Content is too complex to extract reliably
});
}
if (results.length > 0) {
searchResults = results;
}
}
} catch {
// If extraction fails, return original
return result;
}
// Check if it's an array of search results
if (Array.isArray(searchResults)) {
const output: string[] = [`📚 Found ${searchResults.length} relevant notes`];
if (searchResults.length === 0) {
output.push("\nNo matching notes found.");
return output.join("");
}
output.push("");
output.push("Top results:");
output.push("");
// Show top 10 results
searchResults.slice(0, 10).forEach((item, index) => {
const filename =
item.path?.split("/").pop()?.replace(/\.md$/, "") || item.title || "Untitled";
const score = item.rerank_score || item.score || 0;
const scoreDisplay = typeof score === "number" ? score.toFixed(3) : score;
output.push(`${index + 1}. ${filename}`);
output.push(` 📊 Relevance: ${scoreDisplay}${item.includeInContext ? " ✓" : ""}`);
// Show a snippet of content if available
if (item.content) {
// Extract actual content from the formatted string
let cleanContent = item.content;
// Remove NOTE TITLE section
cleanContent = cleanContent.replace(
/NOTE TITLE:[\s\S]*?(?=METADATA:|NOTE BLOCK CONTENT:)/,
""
);
// Remove METADATA section
cleanContent = cleanContent.replace(/METADATA:\{[\s\S]*?\}/, "");
// Extract content after NOTE BLOCK CONTENT:
const contentMatch = cleanContent.match(/NOTE BLOCK CONTENT:\s*([\s\S]*)/);
if (contentMatch) {
cleanContent = contentMatch[1];
}
// Clean up the content
const snippet = cleanContent
.substring(0, 150)
.replace(/\n+/g, " ")
.replace(/\s+/g, " ")
.trim();
if (snippet) {
output.push(` 💬 "${snippet}${cleanContent.length > 150 ? "..." : ""}"`);
}
}
// Show path if different from filename
if (item.path && !item.path.endsWith(`/${filename}.md`)) {
output.push(` 📁 ${item.path}`);
}
output.push("");
});
if (searchResults.length > 10) {
output.push(`... and ${searchResults.length - 10} more results`);
}
return output.join("\n");
}
// If not an array, return original
return typeof result === "string" ? result : JSON.stringify(result, null, 2);
}
private static formatWebSearch(result: any): string {
if (typeof result === "string") {
// Web search results include instructions and citations
// Extract the main content and citations
const lines = result.split("\n");
const output: string[] = ["🌐 Web Search Results"];
let inSources = false;
const mainContent: string[] = [];
const sources: string[] = [];
for (const line of lines) {
if (line.includes("Sources:")) {
inSources = true;
continue;
}
if (inSources) {
sources.push(line);
} else if (!line.includes("Here are the web search results")) {
mainContent.push(line);
}
}
// Add main content
if (mainContent.length > 0) {
output.push("");
output.push(...mainContent.filter((line) => line.trim()));
}
// Add sources
if (sources.length > 0) {
output.push("");
output.push("Sources:");
sources.forEach((source) => {
if (source.trim()) {
output.push(source);
}
});
}
return output.join("\n");
}
return result;
}
private static formatYoutubeTranscription(result: any): string {
// Handle both string and object results
let parsed: any;
if (typeof result === "string") {
try {
parsed = JSON.parse(result);
} catch {
// If not JSON, return as is
return result;
}
} else if (typeof result === "object") {
parsed = result;
} else {
return String(result);
}
// Handle error case
if (parsed.success === false) {
return `📺 YouTube Transcription Failed\n\n${parsed.message}`;
}
// Handle new multi-URL format
if (parsed.results && Array.isArray(parsed.results)) {
const output: string[] = [
`📺 YouTube Transcripts (${parsed.total_urls} video${parsed.total_urls > 1 ? "s" : ""})`,
];
output.push("");
for (const videoResult of parsed.results) {
if (videoResult.success) {
output.push(`📹 Video: ${videoResult.url}`);
output.push("");
// Format transcript
const lines = videoResult.transcript.split("\n");
let formattedLines = 0;
for (const line of lines) {
if (line.trim()) {
// Check if line starts with a timestamp pattern [MM:SS]
const timestampMatch = line.match(/^\[(\d+:\d+)\]/);
if (timestampMatch) {
if (formattedLines > 0) output.push(""); // Add spacing
output.push(`${line}`);
} else {
output.push(` ${line.trim()}`);
}
formattedLines++;
// Limit output to prevent overwhelming display
if (formattedLines > 30) {
output.push("");
output.push("... (transcript truncated for display)");
break;
}
}
}
if (videoResult.elapsed_time_ms) {
output.push("");
output.push(`Processing time: ${(videoResult.elapsed_time_ms / 1000).toFixed(1)}s`);
}
} else {
output.push(`❌ Failed to transcribe: ${videoResult.url}`);
output.push(` ${videoResult.message}`);
}
output.push("");
output.push("---");
output.push("");
}
return output.join("\n").trimEnd();
}
// Handle old single-video format
if (parsed.transcript) {
const output: string[] = ["📺 YouTube Transcript"];
output.push("");
// Split transcript into manageable chunks
const lines = parsed.transcript.split("\n");
let formattedLines = 0;
for (const line of lines) {
if (line.trim()) {
// Check if line starts with a timestamp pattern [MM:SS]
const timestampMatch = line.match(/^\[(\d+:\d+)\]/);
if (timestampMatch) {
if (formattedLines > 0) output.push(""); // Add spacing
output.push(`${line}`);
} else {
output.push(` ${line.trim()}`);
}
formattedLines++;
// Limit output to prevent overwhelming display
if (formattedLines > 50) {
output.push("");
output.push("... (transcript truncated for display)");
break;
}
}
}
if (parsed.elapsed_time_ms) {
output.push("");
output.push(`Processing time: ${(parsed.elapsed_time_ms / 1000).toFixed(1)}s`);
}
return output.join("\n");
}
// If we can't format it, return as string
return typeof result === "object" ? JSON.stringify(result, null, 2) : String(result);
}
private static formatWriteToFile(result: any): string {
if (typeof result === "string") {
// Check if it contains "accepted" or "rejected"
if (result.toLowerCase().includes("accepted")) {
return "✅ File change: accepted";
} else if (result.toLowerCase().includes("rejected")) {
return "❌ File change: rejected";
}
// Fallback for other messages
return result;
}
return result;
}
private static formatReplaceInFile(result: any): string {
if (typeof result === "string") {
// Extract the number of replacements from the result
const blockMatch = result.match(/Applied (\d+) SEARCH\/REPLACE block\(s\)/);
if (blockMatch) {
const blockCount = blockMatch[1];
const replacementText = blockCount === "1" ? "replacement" : "replacements";
if (result.toLowerCase().includes("accepted")) {
return `${blockCount} ${replacementText} accepted`;
}
}
// Check if it contains "accepted" or "rejected"
if (result.toLowerCase().includes("accepted")) {
return "✅ File replacements: accepted";
} else if (result.toLowerCase().includes("rejected")) {
return "❌ File replacements: rejected";
}
// Fallback for other messages
return result;
}
return result;
}
}

View file

@ -1,9 +1,105 @@
import { BrevilabsClient } from "@/LLMProviders/brevilabsClient";
import { tool } from "@langchain/core/tools";
import { extractAllYoutubeUrls } from "@/utils";
import { z } from "zod";
import { createTool } from "./SimpleTool";
const simpleYoutubeTranscriptionTool = tool(
async ({ url }: { url: string }) => {
// Maximum input length to prevent potential DoS attacks
const MAX_USER_MESSAGE_LENGTH = 50000; // Maximum number of characters
interface YouTubeHandlerArgs {
_userMessageContent: string;
}
const youtubeTranscriptionTool = createTool({
name: "youtubeTranscription",
description: "Get transcripts of YouTube videos when the user provides YouTube URLs",
schema: z.object({}), // Empty schema - the tool will receive _userMessageContent internally
isPlusOnly: true,
requiresUserMessageContent: true,
handler: async (args: YouTubeHandlerArgs) => {
// The _userMessageContent is injected by the tool execution system
const { _userMessageContent } = args;
// Input validation
if (typeof _userMessageContent !== "string") {
return JSON.stringify({
success: false,
message: "Invalid input: User message must be a string",
});
}
if (_userMessageContent.length > MAX_USER_MESSAGE_LENGTH) {
return JSON.stringify({
success: false,
message: `Input too long: Maximum allowed length is ${MAX_USER_MESSAGE_LENGTH} characters`,
});
}
// Extract YouTube URLs only from the user's message
const urls = extractAllYoutubeUrls(_userMessageContent);
if (urls.length === 0) {
return JSON.stringify({
success: false,
message:
"No YouTube URLs found in the user prompt. URLs must be in the user prompt instead of the context notes.",
});
}
// Process multiple URLs if present
const results = await Promise.all(
urls.map(async (url) => {
try {
const response = await BrevilabsClient.getInstance().youtube4llm(url);
// Check if transcript is empty
if (!response.response.transcript) {
return {
url,
success: false,
message:
"Transcript not available. Only English videos with auto transcript enabled are supported",
};
}
return {
url,
success: true,
transcript: response.response.transcript,
elapsed_time_ms: response.elapsed_time_ms,
};
} catch (error) {
console.error(`Error transcribing YouTube video ${url}:`, error);
return {
url,
success: false,
message: "An error occurred while transcribing the YouTube video",
};
}
})
);
// Check if at least one transcription was successful
const hasSuccessfulTranscriptions = results.some((result) => result.success);
return JSON.stringify({
success: hasSuccessfulTranscriptions,
results,
total_urls: urls.length,
});
},
});
// Legacy single-URL tool kept for backward compatibility with IntentAnalyzer
// IntentAnalyzer still uses this for non-agent based tool calls (e.g., @youtube command)
const simpleYoutubeTranscriptionTool = createTool({
name: "simpleYoutubeTranscription",
description: "Get the transcript of a YouTube video",
schema: z.object({
url: z.string().url().describe("The YouTube video URL"),
}),
isPlusOnly: true,
handler: async ({ url }) => {
try {
const response = await BrevilabsClient.getInstance().youtube4llm(url);
@ -29,14 +125,6 @@ const simpleYoutubeTranscriptionTool = tool(
});
}
},
{
name: "youtubeTranscription",
description: "Get the transcript of a YouTube video",
schema: z.object({
url: z.string().describe("The YouTube video URL"),
brevilabsClient: z.any().describe("The BrevilabsClient instance"),
}),
}
);
});
export { simpleYoutubeTranscriptionTool };
export { simpleYoutubeTranscriptionTool, youtubeTranscriptionTool };

View file

@ -0,0 +1,325 @@
import { z } from "zod";
import { SimpleTool } from "./SimpleTool";
/**
* Tool validation tests to ensure all tools follow best practices
* and have properly typed schemas that match their handlers
*/
// Helper to check if a tool schema uses z.any() which reduces type safety
function hasWeakTyping(schema: z.ZodType): boolean {
if (schema instanceof z.ZodAny) {
return true;
}
if (schema instanceof z.ZodObject) {
const shape = schema.shape;
for (const value of Object.values(shape)) {
if (hasWeakTyping(value as z.ZodType)) {
return true;
}
}
}
if (schema instanceof z.ZodArray) {
return hasWeakTyping(schema._def.type);
}
if (
schema instanceof z.ZodOptional ||
schema instanceof z.ZodNullable ||
schema instanceof z.ZodDefault
) {
return hasWeakTyping(schema._def.innerType);
}
return false;
}
// Helper to validate tool metadata
function validateToolMetadata(tool: SimpleTool<any, any>): string[] {
const issues: string[] = [];
if (!tool.name || tool.name.trim() === "") {
issues.push("Tool must have a non-empty name");
}
if (!tool.description || tool.description.trim() === "") {
issues.push("Tool must have a non-empty description");
}
if (tool.name && tool.name.includes(" ")) {
issues.push("Tool name should not contain spaces");
}
if (!tool.schema) {
issues.push("Tool must have a schema");
}
return issues;
}
describe("All Tools Validation", () => {
describe("SimpleTool tools validation", () => {
// We'll test tools individually since we can't easily import all at once due to circular deps
test("Tool validation helper functions work correctly", () => {
// Test weak typing detection
const weakSchema = z.object({
good: z.string(),
bad: z.any(), // This should be detected
});
expect(hasWeakTyping(weakSchema)).toBe(true);
const strongSchema = z.object({
query: z.string(),
count: z.number(),
items: z.array(z.string()),
});
expect(hasWeakTyping(strongSchema)).toBe(false);
});
test("Metadata validation works correctly", () => {
const goodTool: SimpleTool<any, any> = {
name: "testTool",
description: "A test tool",
schema: z.void(),
call: async () => "result",
};
expect(validateToolMetadata(goodTool)).toEqual([]);
const badTool: SimpleTool<any, any> = {
name: "",
description: " ",
schema: z.void(),
call: async () => "result",
};
const issues = validateToolMetadata(badTool);
expect(issues).toContain("Tool must have a non-empty name");
expect(issues).toContain("Tool must have a non-empty description");
});
});
describe("Schema best practices", () => {
test("Schemas should use specific types instead of z.any()", () => {
// Good practice
const goodSchema = z.object({
query: z.string().describe("Search query"),
options: z
.object({
caseSensitive: z.boolean().optional(),
limit: z.number().min(1).max(100).optional(),
})
.optional(),
});
expect(hasWeakTyping(goodSchema)).toBe(false);
// Bad practice - using z.any()
const badSchema = z.object({
query: z.string(),
data: z.any(), // Avoid this!
});
expect(hasWeakTyping(badSchema)).toBe(true);
});
test("Array schemas should specify element types", () => {
// Good - specific element type
const goodArraySchema = z.object({
items: z.array(z.string()),
history: z.array(
z.object({
timestamp: z.number(),
action: z.string(),
})
),
});
expect(hasWeakTyping(goodArraySchema)).toBe(false);
// Bad - array of any
const badArraySchema = z.object({
items: z.array(z.any()),
});
expect(hasWeakTyping(badArraySchema)).toBe(true);
});
test("Optional fields should be properly marked", () => {
const schema = z.object({
required: z.string(),
optional: z.string().optional(),
withDefault: z.string().default("default"),
nullable: z.string().nullable(),
});
// All valid inputs
expect(
schema.safeParse({ required: "test", withDefault: "value", nullable: "text" }).success
).toBe(true);
expect(
schema.safeParse({
required: "test",
optional: "opt",
withDefault: "value",
nullable: "text",
}).success
).toBe(true);
expect(
schema.safeParse({ required: "test", withDefault: "value", nullable: null }).success
).toBe(true);
// Test that default value is applied
const parsed = schema.parse({ required: "test", nullable: "text" });
expect(parsed.withDefault).toBe("default");
});
test("Descriptions should be provided for documentation", () => {
const wellDocumentedSchema = z.object({
query: z.string().describe("The search query to execute"),
limit: z.number().min(1).describe("Maximum number of results to return"),
includeArchived: z.boolean().describe("Whether to include archived items"),
});
// Check that descriptions are accessible
const shape = wellDocumentedSchema.shape;
expect((shape.query as any)._def.description).toBe("The search query to execute");
expect((shape.limit as any)._def.description).toBe("Maximum number of results to return");
});
});
describe("Common tool patterns", () => {
test("Search tools should have consistent parameter names", () => {
// Common pattern for search tools
const searchSchema = z.object({
query: z.string().min(1).describe("Search query"),
// Other tools might have: filters, limit, offset, etc.
});
// Validate the pattern
const result = searchSchema.safeParse({ query: "test search" });
expect(result.success).toBe(true);
});
test("Time-based tools should accept time parameters consistently", () => {
// TimeInfo pattern that's used across tools
const timeInfoSchema = z.object({
epoch: z.number(),
isoString: z.string(),
userLocaleString: z.string(),
localDateString: z.string(),
timezoneOffset: z.number(),
timezone: z.string(),
});
const timeRangeSchema = z.object({
startTime: timeInfoSchema,
endTime: timeInfoSchema,
});
// This pattern should be consistent across tools
const validTimeRange = {
startTime: {
epoch: 1234567890000,
isoString: "2009-02-13T23:31:30.000Z",
userLocaleString: "2/13/2009, 11:31:30 PM",
localDateString: "2009-02-13",
timezoneOffset: 0,
timezone: "UTC",
},
endTime: {
epoch: 1234567900000,
isoString: "2009-02-13T23:31:40.000Z",
userLocaleString: "2/13/2009, 11:31:40 PM",
localDateString: "2009-02-13",
timezoneOffset: 0,
timezone: "UTC",
},
};
expect(timeRangeSchema.safeParse(validTimeRange).success).toBe(true);
});
test("File operation tools should validate paths", () => {
const fileToolSchema = z.object({
path: z.string().min(1).describe("File path"),
content: z.string().optional().describe("File content"),
});
// Valid paths
expect(fileToolSchema.safeParse({ path: "/valid/path.md" }).success).toBe(true);
expect(fileToolSchema.safeParse({ path: "relative/path.md" }).success).toBe(true);
// Invalid - empty path
expect(fileToolSchema.safeParse({ path: "" }).success).toBe(false);
});
});
describe("Error handling patterns", () => {
test("Tools should provide meaningful validation error messages", () => {
const schema = z.object({
email: z.string().email("Invalid email format"),
age: z.number().min(0, "Age must be non-negative").max(150, "Age seems unrealistic"),
});
const result = schema.safeParse({ email: "not-an-email", age: -5 });
expect(result.success).toBe(false);
if (!result.success) {
const errors = result.error.issues;
expect(errors.some((e) => e.message === "Invalid email format")).toBe(true);
expect(errors.some((e) => e.message === "Age must be non-negative")).toBe(true);
}
});
test("Complex nested schemas should validate deeply", () => {
const complexSchema = z.object({
user: z.object({
name: z.string().min(1),
email: z.string().email(),
preferences: z.object({
theme: z.enum(["light", "dark"]),
notifications: z.boolean(),
}),
}),
items: z
.array(
z.object({
id: z.string(),
quantity: z.number().positive(),
})
)
.min(1, "At least one item required"),
});
// Invalid nested data
const invalidData = {
user: {
name: "",
email: "invalid",
preferences: {
theme: "blue", // Invalid enum value
notifications: true,
},
},
items: [],
};
const result = complexSchema.safeParse(invalidData);
expect(result.success).toBe(false);
if (!result.success) {
const paths = result.error.issues.map((issue) => issue.path.join("."));
expect(paths).toContain("user.name");
expect(paths).toContain("user.email");
expect(paths).toContain("user.preferences.theme");
expect(paths).toContain("items");
}
});
});
});

318
src/tools/builtinTools.ts Normal file
View file

@ -0,0 +1,318 @@
import { ToolDefinition, ToolRegistry } from "./ToolRegistry";
import { localSearchTool, webSearchTool } from "./SearchTools";
import {
getCurrentTimeTool,
getTimeInfoByEpochTool,
getTimeRangeMsTool,
pomodoroTool,
convertTimeBetweenTimezonesTool,
} from "./TimeTools";
import { youtubeTranscriptionTool } from "./YoutubeTools";
import { writeToFileTool, replaceInFileTool } from "./ComposerTools";
import { createGetFileTreeTool } from "./FileTreeTools";
import { Vault } from "obsidian";
/**
* Define all built-in tools with their metadata
*/
export const BUILTIN_TOOLS: ToolDefinition[] = [
// Search tools
{
tool: localSearchTool,
metadata: {
id: "localSearch",
displayName: "Vault Search",
description: "Search through your vault notes",
category: "search",
customPromptInstructions: `For localSearch (searching notes in the vault):
- You MUST always provide both "query" (string) and "salientTerms" (array of strings)
- salientTerms MUST be extracted from the user's original query - never invent new terms
- They are keywords used for BM25 full-text search to find notes containing those exact words
- Extract meaningful content words from the query (nouns, verbs, names, etc.)
- Exclude common words like "what", "I", "do", "the", "a", etc.
- Exclude time expressions like "last month", "yesterday", "last week"
- Preserve the original language - do NOT translate terms to English
Example usage:
<use_tool>
<name>localSearch</name>
<query>piano learning practice</query>
<salientTerms>["piano", "learning", "practice"]</salientTerms>
</use_tool>
For localSearch with time range (e.g., "what did I do last week"):
Step 1 - Get time range:
<use_tool>
<name>getTimeRangeMs</name>
<timeExpression>last week</timeExpression>
</use_tool>
Step 2 - Search with time range (after receiving time range result):
<use_tool>
<name>localSearch</name>
<query>what did I do</query>
<salientTerms>[]</salientTerms>
<timeRange>{"startTime": {...}, "endTime": {...}}</timeRange>
</use_tool>
For localSearch with meaningful terms (e.g., "python debugging notes from yesterday"):
Step 1 - Get time range:
<use_tool>
<name>getTimeRangeMs</name>
<timeExpression>yesterday</timeExpression>
</use_tool>
Step 2 - Search with time range:
<use_tool>
<name>localSearch</name>
<query>python debugging notes</query>
<salientTerms>["python", "debugging", "notes"]</salientTerms>
<timeRange>{"startTime": {...}, "endTime": {...}}</timeRange>
</use_tool>
For localSearch with non-English query (PRESERVE ORIGINAL LANGUAGE):
<use_tool>
<name>localSearch</name>
<query></query>
<salientTerms>["钢琴", "学习"]</salientTerms>
</use_tool>`,
},
},
{
tool: webSearchTool,
metadata: {
id: "webSearch",
displayName: "Web Search",
description: "Search the internet for information",
category: "search",
customPromptInstructions: `For webSearch:
- Only use when the user explicitly requests web/internet search
- Always provide an empty chatHistory array
Example usage:
<use_tool>
<name>webSearch</name>
<query>piano learning techniques</query>
<chatHistory>[]</chatHistory>
</use_tool>`,
},
},
// Time tools (always enabled)
{
tool: getCurrentTimeTool,
metadata: {
id: "getCurrentTime",
displayName: "Get Current Time",
description: "Get the current time in any timezone",
category: "time",
isAlwaysEnabled: true,
customPromptInstructions: `For time queries (IMPORTANT: Always use UTC offsets, not timezone names):
Example 1 - "what time is it" (local time):
<use_tool>
<name>getCurrentTime</name>
</use_tool>
Example 2 - "what time is it in Tokyo" (UTC+9):
<use_tool>
<name>getCurrentTime</name>
<timezoneOffset>+9</timezoneOffset>
</use_tool>
Example 3 - "what time is it in New York" (UTC-5 or UTC-4 depending on DST):
<use_tool>
<name>getCurrentTime</name>
<timezoneOffset>-5</timezoneOffset>
</use_tool>`,
},
},
{
tool: getTimeInfoByEpochTool,
metadata: {
id: "getTimeInfoByEpoch",
displayName: "Get Time Info",
description: "Convert epoch timestamp to human-readable format",
category: "time",
isAlwaysEnabled: true,
},
},
{
tool: getTimeRangeMsTool,
metadata: {
id: "getTimeRangeMs",
displayName: "Get Time Range",
description: "Convert time expressions to date ranges",
category: "time",
isAlwaysEnabled: true,
customPromptInstructions: `For time-based queries:
- Use this tool to convert time expressions like "last week", "yesterday", "last month" to proper time ranges
- This is typically the first step before using localSearch with a time range
Example:
<use_tool>
<name>getTimeRangeMs</name>
<timeExpression>last week</timeExpression>
</use_tool>`,
},
},
{
tool: convertTimeBetweenTimezonesTool,
metadata: {
id: "convertTimeBetweenTimezones",
displayName: "Convert Timezones",
description: "Convert time between different timezones",
category: "time",
isAlwaysEnabled: true,
customPromptInstructions: `For timezone conversions:
Example - "what time is 6pm PT in Tokyo" (PT is UTC-8 or UTC-7, Tokyo is UTC+9):
<use_tool>
<name>convertTimeBetweenTimezones</name>
<time>6pm</time>
<fromOffset>-8</fromOffset>
<toOffset>+9</toOffset>
</use_tool>`,
},
},
{
tool: pomodoroTool,
metadata: {
id: "pomodoro",
displayName: "Pomodoro Timer",
description: "Manage time with Pomodoro technique",
category: "time",
},
},
// File tools
{
tool: writeToFileTool,
metadata: {
id: "writeToFile",
displayName: "Write to File",
description: "Create or modify files in your vault",
category: "file",
requiresVault: true,
customPromptInstructions: `For writeToFile:
- NEVER display the file content directly in your response
- Always pass the complete file content to the tool
- Include the full path to the file
- You MUST explicitly call writeToFile for any intent of updating or creating files
- Do not call writeToFile tool again if the result is not accepted
- Do not call writeToFile tool if no change needs to be made
Example usage:
<use_tool>
<name>writeToFile</name>
<path>path/to/note.md</path>
<content>FULL CONTENT OF THE NOTE</content>
</use_tool>`,
},
},
{
tool: replaceInFileTool,
metadata: {
id: "replaceInFile",
displayName: "Replace in File",
description: "Make targeted changes to existing files using SEARCH/REPLACE blocks",
category: "file",
requiresVault: true,
customPromptInstructions: `For replaceInFile:
- Remember: Small edits replaceInFile, Major rewrites writeToFile
- SEARCH text must match EXACTLY including all whitespace
Example usage:
<use_tool>
<name>replaceInFile</name>
<path>notes/meeting.md</path>
<diff>\`\`\`
------- SEARCH
## Attendees
- John Smith
- Jane Doe
=======
## Attendees
- John Smith
- Jane Doe
- Bob Johnson
+++++++ REPLACE
\`\`\`</diff>
</use_tool>`,
},
},
// Media tools
{
tool: youtubeTranscriptionTool,
metadata: {
id: "youtubeTranscription",
displayName: "YouTube Transcription",
description: "Get transcripts from YouTube videos",
category: "media",
customPromptInstructions: `For youtubeTranscription:
- Use when user provides YouTube URLs
- No parameters needed - the tool will process URLs from the conversation
Example usage:
<use_tool>
<name>youtubeTranscription</name>
</use_tool>`,
},
},
];
/**
* Register the file tree tool separately as it needs vault access
*/
export function registerFileTreeTool(vault: Vault): void {
const registry = ToolRegistry.getInstance();
registry.register({
tool: createGetFileTreeTool(vault.getRoot()),
metadata: {
id: "getFileTree",
displayName: "File Tree",
description: "Browse vault file structure",
category: "file",
isAlwaysEnabled: true,
requiresVault: true,
customPromptInstructions: `For getFileTree:
- Use to browse the vault's file structure
- No parameters needed
Example usage:
<use_tool>
<name>getFileTree</name>
</use_tool>`,
},
});
}
/**
* Initialize all built-in tools in the registry.
* This function registers tool definitions, not user preferences.
* User-enabled tools are filtered dynamically when retrieved.
*
* @param vault - Optional Obsidian vault. When provided, enables registration of vault-dependent tools like file tree
*/
export function initializeBuiltinTools(vault?: Vault): void {
const registry = ToolRegistry.getInstance();
// Only reinitialize if tools have changed or vault status has changed
const hasFileTree = registry.getToolMetadata("getFileTree") !== undefined;
const shouldHaveFileTree = vault !== undefined;
if (registry.getAllTools().length === 0 || hasFileTree !== shouldHaveFileTree) {
// Clear any existing tools
registry.clear();
// Register all built-in tools
registry.registerAll(BUILTIN_TOOLS);
// Register vault-dependent tools if vault is available
if (vault) {
registerFileTreeTool(vault);
}
}
}

View file

@ -15,3 +15,5 @@ export enum PromptSortStrategy {
ALPHABETICAL = "alphabetical",
MANUAL = "manual",
}
export type ApplyViewResult = "accepted" | "rejected" | "aborted" | "failed";

View file

@ -54,7 +54,7 @@ export interface ChatMessage {
isVisible: boolean;
/** Sources cited in the response */
sources?: { title: string; score: number }[];
sources?: { title: string; path: string; score: number }[];
/** Rich content (images, etc.) */
content?: any[];
@ -87,6 +87,6 @@ export interface StoredMessage {
context?: MessageContext;
isVisible: boolean;
isErrorMessage?: boolean;
sources?: { title: string; score: number }[];
sources?: { title: string; path: string; score: number }[];
content?: any[];
}

View file

@ -0,0 +1,110 @@
import { cleanMessageForCopy } from "./utils";
describe("cleanMessageForCopy", () => {
it("should remove Think blocks", () => {
const input = "Before text\n<think>This is my thought process</think>\nAfter text";
const expected = "Before text\n\nAfter text";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should remove writeToFile blocks wrapped in XML codeblocks", () => {
const input = `Some text before
\`\`\`xml
<writeToFile>
<path>test.md</path>
<content>File content here</content>
</writeToFile>
\`\`\`
Some text after`;
const expected = "Some text before\n\nSome text after";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should remove standalone writeToFile blocks", () => {
const input = `Text before
<writeToFile>
<path>test.md</path>
<content>File content</content>
</writeToFile>
Text after`;
const expected = "Text before\n\nText after";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should remove tool call markers", () => {
const input =
"Before\n<!--TOOL_CALL_START:123:localSearch:Local Search:🔍::true-->Searching...<!--TOOL_CALL_END:123:Found 5 results-->\nAfter";
const expected = "Before\n\nAfter";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should handle multiple blocks in one message", () => {
const input = `Start of message
<think>First thought</think>
Middle part
<writeToFile><path>file.md</path><content>content</content></writeToFile>
<!--TOOL_CALL_START:456:webSearch:Web Search:🌐::false-->Searching web<!--TOOL_CALL_END:456:Results-->
End of message`;
const expected = "Start of message\n\nMiddle part\n\nEnd of message";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should clean up multiple consecutive newlines", () => {
const input = `Text\n\n\n\n\nMore text`;
const expected = "Text\n\nMore text";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should preserve normal content", () => {
const input = `# Heading
This is a normal message with:
- Bullet points
- Code blocks: \`const x = 1;\`
- **Bold** and *italic* text
\`\`\`javascript
function test() {
return true;
}
\`\`\`
More content here.`;
expect(cleanMessageForCopy(input)).toBe(input);
});
it("should handle nested think blocks", () => {
const input =
"Before\n<think>Outer thought <think>Inner thought</think> back to outer</think>\nAfter";
// Since we're not handling nested blocks, the outer block will be removed but inner content remains
const expected = "Before\n back to outer</think>\nAfter";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should handle multiline content in blocks", () => {
const input = `Start
<think>
Line 1 of thought
Line 2 of thought
Line 3 of thought
</think>
End`;
const expected = "Start\n\nEnd";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should trim leading and trailing whitespace", () => {
const input = "\n\n Content with spaces \n\n";
const expected = "Content with spaces";
expect(cleanMessageForCopy(input)).toBe(expected);
});
it("should handle empty message", () => {
expect(cleanMessageForCopy("")).toBe("");
});
it("should handle message with only blocks to remove", () => {
const input = "<think>Only a thought</think>";
const expected = "";
expect(cleanMessageForCopy(input)).toBe(expected);
});
});

View file

@ -291,16 +291,20 @@ export function isAllowedFileForContext(file: TFile | null): boolean {
}
export async function getAllNotesContent(vault: Vault): Promise<string> {
let allContent = "";
const vaultNotes: string[] = [];
const markdownFiles = vault.getMarkdownFiles();
for (const file of markdownFiles) {
const fileContent = await vault.cachedRead(file);
allContent += fileContent + " ";
// Import is not available at the top level due to circular dependency
const { VAULT_NOTE_TAG } = await import("@/constants");
vaultNotes.push(
`<${VAULT_NOTE_TAG}>\n<path>${file.path}</path>\n<content>\n${fileContent}\n</content>\n</${VAULT_NOTE_TAG}>`
);
}
return allContent;
return vaultNotes.join("\n\n");
}
export function areEmbeddingModelsSame(
@ -372,6 +376,17 @@ export interface ChatHistoryEntry {
content: string;
}
/**
* Extract text-only chat history from memory variables.
* This function pairs messages by index (i, i+1) and returns only string content.
*
* Note: For multimodal chains (CopilotPlus, AutonomousAgent), use
* chatHistoryUtils.processRawChatHistory instead to preserve image content.
*
* @param memoryVariables Memory variables from LangChain memory
* @returns Array of text-only chat history entries
*/
// TODO: Deprecated, use chatHistoryUtils.processRawChatHistory instead
export function extractChatHistory(memoryVariables: MemoryVariables): ChatHistoryEntry[] {
const chatHistory: ChatHistoryEntry[] = [];
const { history } = memoryVariables;
@ -655,6 +670,41 @@ export function getProviderKeyManagementURL(provider: string): string {
return ProviderInfo[provider as Provider]?.keyManagementURL || "";
}
/**
* Cleans a message by removing Think blocks, Action blocks (writeToFile), and tool call markers
* for copying to clipboard. This is more comprehensive than removeThinkTags which is used for RAG.
*/
export function cleanMessageForCopy(message: string): string {
let cleanedMessage = message;
// First use the existing removeThinkTags function
cleanedMessage = removeThinkTags(cleanedMessage);
// Remove writeToFile blocks wrapped in XML codeblocks
cleanedMessage = cleanedMessage.replace(
/```xml\s*[\s\S]*?<writeToFile>[\s\S]*?<\/writeToFile>[\s\S]*?```/g,
""
);
// Remove standalone writeToFile blocks
cleanedMessage = cleanedMessage.replace(/<writeToFile>[\s\S]*?<\/writeToFile>/g, "");
// Remove tool call markers
// Format: <!--TOOL_CALL_START:id:toolName:displayName:emoji:confirmationMessage:isExecuting-->content<!--TOOL_CALL_END:id:result-->
cleanedMessage = cleanedMessage.replace(
/<!--TOOL_CALL_START:[^:]+:[^:]+:[^:]+:[^:]+:[^:]*:[^:]+-->[\s\S]*?<!--TOOL_CALL_END:[^:]+:[\s\S]*?-->/g,
""
);
// Clean up any resulting multiple consecutive newlines (more than 2)
cleanedMessage = cleanedMessage.replace(/\n{3,}/g, "\n\n");
// Trim leading and trailing whitespace
cleanedMessage = cleanedMessage.trim();
return cleanedMessage;
}
export async function insertIntoEditor(message: string, replace: boolean = false) {
let leaf = app.workspace.getMostRecentLeaf();
if (!leaf) {
@ -676,8 +726,8 @@ export async function insertIntoEditor(message: string, replace: boolean = false
const cursorFrom = editor.getCursor("from");
const cursorTo = editor.getCursor("to");
// Remove think tags before inserting
const cleanedMessage = removeThinkTags(message);
// Clean the message before inserting (removes think tags, writeToFile blocks, tool calls)
const cleanedMessage = cleanMessageForCopy(message);
if (replace) {
editor.replaceRange(cleanedMessage, cursorFrom, cursorTo);
@ -864,3 +914,10 @@ export async function withSuppressedTokenWarnings<T>(fn: () => Promise<T>): Prom
console.warn = originalWarn;
}
}
/**
* Check if the current Obsidian editor setting is in live preview mode
*/
export function isLivePreviewModeOn(): boolean {
return !!(app.vault as any).config?.livePreview;
}

View file

@ -0,0 +1,67 @@
import {
truncateToolResult,
formatToolResultForMemory,
processToolResults,
} from "./toolResultUtils";
// No need to mock settings since we're not using them
describe("toolResultUtils", () => {
describe("truncateToolResult", () => {
it("should not truncate results shorter than max length", () => {
const result = "Short result";
expect(truncateToolResult(result)).toBe(result);
});
it("should truncate results longer than max length", () => {
// Default max length is 10000, so we need a string longer than that
const result = "a".repeat(10050);
const truncated = truncateToolResult(result);
expect(truncated).toContain("a".repeat(10000));
expect(truncated).toContain("... (truncated 50 characters)");
});
it("should use custom max length when provided", () => {
const result = "a".repeat(50);
const truncated = truncateToolResult(result, 20);
expect(truncated).toContain("a".repeat(20));
expect(truncated).toContain("... (truncated 30 characters)");
});
it("should handle empty or null results", () => {
expect(truncateToolResult("")).toBe("");
expect(truncateToolResult(null as any)).toBe(null);
});
});
describe("formatToolResultForMemory", () => {
it("should format and truncate tool results", () => {
const result = "a".repeat(10050);
const formatted = formatToolResultForMemory("testTool", result);
expect(formatted).toMatch(/^Tool 'testTool' result: /);
expect(formatted).toContain("... (truncated 50 characters)");
});
});
describe("processToolResults", () => {
const toolResults = [
{ toolName: "tool1", result: "a".repeat(10050) },
{ toolName: "tool2", result: "b".repeat(10100) },
];
it("should process results without truncation when truncate=false", () => {
const processed = processToolResults(toolResults, false);
expect(processed).toContain("Tool 'tool1' result: " + "a".repeat(10050));
expect(processed).toContain("Tool 'tool2' result: " + "b".repeat(10100));
expect(processed).not.toContain("truncated");
});
it("should process results with truncation when truncate=true", () => {
const processed = processToolResults(toolResults, true);
expect(processed).toContain("Tool 'tool1' result: " + "a".repeat(10000));
expect(processed).toContain("Tool 'tool2' result: " + "b".repeat(10000));
expect(processed).toContain("... (truncated 50 characters)");
expect(processed).toContain("... (truncated 100 characters)");
});
});
});

View file

@ -0,0 +1,68 @@
/**
* Utility functions for handling tool results
*/
export const DEFAULT_TOOL_RESULT_MAX_LENGTH = 10000; // Default max length for tool results in chat memory
/**
* Get the configured max length for tool results
*/
function getToolResultMaxLength(): number {
// For now, just return the default since toolResultMaxLength doesn't exist in settings
return DEFAULT_TOOL_RESULT_MAX_LENGTH;
}
/**
* Truncate tool result if it exceeds the maximum length
* @param result The tool result string
* @param maxLength Maximum allowed length (default: from settings)
* @returns Truncated result with ellipsis if needed
*/
export function truncateToolResult(result: string, maxLength?: number): string {
const actualMaxLength = maxLength ?? getToolResultMaxLength();
if (!result || result.length <= actualMaxLength) {
return result;
}
// Truncate and add ellipsis with info about truncation
const truncated = result.substring(0, actualMaxLength);
const remainingChars = result.length - actualMaxLength;
return `${truncated}\n\n... (truncated ${remainingChars.toLocaleString()} characters)`;
}
/**
* Format tool result for memory storage
* @param toolName The name of the tool
* @param result The raw tool result
* @param maxLength Maximum allowed length
* @returns Formatted and truncated tool result
*/
export function formatToolResultForMemory(
toolName: string,
result: string,
maxLength?: number
): string {
const truncatedResult = truncateToolResult(result, maxLength);
return `Tool '${toolName}' result: ${truncatedResult}`;
}
/**
* Process tool results for different contexts
* @param toolResults Array of tool execution results
* @param truncate Whether to truncate the results
* @returns Formatted tool results string
*/
export function processToolResults(
toolResults: Array<{ toolName: string; result: string }>,
truncate: boolean = false
): string {
return toolResults
.map((result) => {
const formattedResult = truncate
? formatToolResultForMemory(result.toolName, result.result)
: `Tool '${result.toolName}' result: ${result.result}`;
return formattedResult;
})
.join("\n\n");
}