No description
Find a file
Wenzheng Jiang 4884d9513d
Setup Prompt test for Agent (#1688)
* Implement autonomous agent mode and new message architecture

- Implement autonomous agent mode with sequential thinking capabilities
- Migrate from sharedState to new clean message management architecture
- Add ChatManager, MessageRepository, and ContextManager for better separation of concerns
- Implement project-based chat isolation with separate message repositories per project
- Add ChatPersistenceManager for saving/loading chat history
- Refactor chain runner architecture with specialized runners (Autonomous, Project, VaultQA)
- Add XML tool call handling for better compatibility across LLM providers
- Introduce vault and web search toggles in settings
- Enhance model adapters with explicit tool usage instructions
- Add comprehensive test coverage for new architecture components
- Update UI components to use new ChatUIState instead of SharedState

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Enhance context processing and documentation

- Added detailed XML structure for context notes, including metadata (creation and modification times) for notes and selected text.
- Introduced new tests for message lifecycle and XML tag formatting to ensure proper context handling.
- Created TODO.md to track technical debt and future improvements related to context processing and file handling.
- Updated CLAUDE.md to reference the new TODO.md for technical debt awareness.

This commit improves the clarity and functionality of context processing in messages, ensuring rich context is maintained throughout the message lifecycle.

* Fix composer instruction in new message implementation

* Add writeToFile tool to agent (#1632)

* Add writeToFile tool call to agent

* Update src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/tools/ComposerTools.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove unuseful comment

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix tests

* Enhance context processing with new XML tags for notes and selected text (#1633)

* Enhance context processing with new XML tags for notes and selected text

- Introduced new constants for XML tags: SELECTED_TEXT_TAG, VARIABLE_TAG, VARIABLE_NOTE_TAG, EMBEDDED_PDF_TAG, VAULT_NOTE_TAG, and RETRIEVED_DOCUMENT_TAG.
- Updated context processing to format embedded PDFs and notes with XML structure.
- Modified prompt processing to include XML tags for selected text and variables, improving clarity and structure in generated outputs.
- Enhanced tests to validate the new XML formatting in processed prompts.

* Implement XML escaping utility functions and integrate into context processing

- Added `escapeXml` and `escapeXmlAttribute` functions to handle XML special character escaping, enhancing security against XML injection.
- Updated context processing to utilize these functions for escaping note titles, paths, and content, ensuring proper XML formatting.
- Enhanced tests to validate XML escaping functionality across various scenarios, including selected text and variable names.
- Introduced new test files for XML utility functions to ensure comprehensive coverage and reliability.

* Refactor XML utility functions and update imports

- Moved `escapeXml` and `escapeXmlAttribute` functions from `utils/xmlUtils` to `LLMProviders/chainRunner/utils/xmlParsing`, enhancing modularity.
- Updated all relevant imports across the codebase to reflect the new location of XML utility functions.
- Removed obsolete `xmlUtils` file and its associated tests, consolidating XML handling in a single module for better maintainability.

* Add message creation callback in ChatManager for immediate UI updates (#1634)

- Introduced a new method `setOnMessageCreatedCallback` in ChatManager to allow setting a callback that triggers when a message is created.
- Updated ChatUIState to utilize this callback for notifying listeners, ensuring the UI updates immediately upon message creation.
- This enhancement improves the responsiveness of the chat interface by synchronizing message creation events with UI state updates.

* Enhance TimeTools tests for invalid expressions

- Added setup to mock console.warn in tests for invalid time expressions.
- Verified that the appropriate warning message is logged when parsing fails.
- Improved test coverage for handling various invalid time inputs.

* Refactor local search handling and restore sources display

- Unified tool handling in CopilotPlusChainRunner to fix websearch results being ignored when localSearch has results
- All tool outputs now go through the same prepareEnhancedUserMessage method
- Preserved QA format and instruction when only localSearch is used
- Added proper validation for localSearch results (non-empty documents array)
- Maintained time expression support for temporal queries
- Fixed source tracking and display in ChatUIState
- Enhanced error handling for JSON parsing of search results

* Enhance AutonomousAgentChainRunner to track tool calls and results (#1639)

- Introduced a new property to store LLM-formatted messages for memory management.
- Reset LLM messages at the start of each run to ensure fresh context.
- Updated response handling to include LLM-formatted outputs, enhancing memory context for future iterations.
- Modified the base class to accept an optional LLM-formatted output parameter for improved context saving.
- Enhanced the writeToFile tool's response message for clarity on user actions.

* Quick Command (cmd+k) (#1640)

* Quick Command (cmd+k)

* Update custom command modal style

* Improve prompt

* Use submenu for commands

* Fix cursor comment

* Align selected_text variable name

* Fix embedded image wikilinks (#1641)

- Updated the `extractEmbeddedImages` method to accept an optional `sourcePath` parameter for resolving wikilinks.
- Implemented logic to determine the source path from context notes or fallback to the active file.
- Enhanced logging for unresolved images to improve debugging and user feedback.
- Ensured that embedded images are processed correctly based on the new source path handling.

* Fix salient term language issue

* Dedupe source notes (#1642)

* Fix salient term language issue

* Refactor source handling to include path in addition to title and score

- Updated source structures across multiple classes to include a `path` property alongside `title` and `score`.
- Modified methods in `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to accommodate the new source format.
- Enhanced the `deduplicateSources` function to use `path` as the unique key, falling back to `title` if necessary.
- Adjusted the `SourcesModal` to display paths correctly and ensure links open using the appropriate source path.
- Updated type definitions in `message.ts` to reflect the new source structure.

* Update path handling in source mapping for AutonomousAgentChainRunner

- Modified the source mapping logic to ensure the `path` property defaults to an empty string if not provided, enhancing robustness in source data handling.
- This change improves the consistency of source objects by ensuring that all properties are defined, which aids in downstream processing.

* Bump manifest version to v3

* Fix image (#1643)

* Fix salient term language issue

* Bump manifest version to v3

* Fix image passing

- Updated Chat, ChatManager, MessageRepository, and ChatUIState to include an optional `content` parameter in message handling.
- Adjusted method signatures and message creation logic to accommodate the new parameter, improving flexibility in message processing.

* Fix regenerate (#1644)

* Fix image in agent chain (#1646)

* Add YouTube transcription processing to AutonomousAgentChainRunner (#1647)

- Implemented a new method to extract and process YouTube URLs from user messages.
- Fetch transcriptions using the simpleYoutubeTranscriptionTool and handle errors gracefully.
- Updated conversation messages to include fetched transcriptions for improved context in responses.

* Disable quick command when live-mode is not on (#1648)

* Do not call writeToFileTool again for accepted changes as well (#1652)

* Trigger youtube tool only for urls in prompt not context (#1653)

* Stop calling youtube tool in context notes in agent mode

- Added a new instruction for handling YouTube URLs in the DEFAULT_SYSTEM_PROMPT.
- Removed the direct usage of simpleYoutubeTranscriptionTool in the AutonomousAgentChainRunner, as YouTube transcriptions are now automatically processed.
- Updated parameter naming conventions in modelAdapter for clarity and consistency.
- Cleaned up the simpleYoutubeTranscriptionTool definition by removing unnecessary parameters.

* Add agent loop limit message (#1655)

* Add composer toggle and implement auto-preview (#1651)

* Update composer output format to XML and refactor related components

- Changed the output format for composer instructions from JSON to XML in `constants.ts`.
- Updated `CopilotPlusChainRunner` to utilize the new XML format and adjusted streaming logic accordingly.
- Added a toggle for the composer feature in `ChatInput.tsx` and updated UI elements to reflect this change.
- Removed the `ComposerCodeBlock` component as it is no longer needed with the new implementation.
- Cleaned up unused code and comments in `ChatSingleMessage.tsx` related to the previous composer handling.

* Update src/constants.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/constants.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove redundant tests

* Update doc

* Refactor composer handling and implement ActionBlockStreamer

- Replaced `ComposerBlockStreamer` with `ActionBlockStreamer` to handle `writeToFile` blocks more efficiently.
- Updated `constants.ts` to reflect changes in output format for composer instructions.
- Enhanced `ChatSingleMessage.tsx` to process `writeToFile` sections and integrate collapsible UI elements.
- Removed the deprecated `ComposerBlockStreamer` and its associated tests.
- Improved handling of XML codeblocks and unclosed tags during streaming.

* Fix reported bugs

* Refactor ActionBlockStreamer tests and implementation

- Updated `processChunks` to always push content, including null and empty strings, to the output.
- Adjusted test expectations to reflect changes in output handling for chunks with null content.
- Simplified regex in `findCompleteBlock` method by removing XML format handling.

* Fix reported issue

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix tool description generation (#1659)

* Access tool description from zod.shape when zod.properties is missing.

* Remove the access to schema.properties

* Remove unused print

* Also support schema.propertes

* Implement SimpleTool and remove Langchain tool calling (#1662)

* Make web search tool not overly eager

* Use SimpleTool interface instead of langchain tool call

- Replaced LangChain tool definitions with a new SimpleTool interface for better type safety and validation.
- Updated existing tools (e.g., writeToFile, localSearch, webSearch, and YouTube transcription) to utilize the new createTool function.
- Enhanced parameter extraction from Zod schemas for improved tool description generation.
- Cleaned up related code and documentation to reflect the new tool structure and ensure consistency across the codebase.
- Add tests

* Update ChatManager tests to handle undefined context cases

- Modified test cases in ChatManager to include handling of undefined context parameters.
- Ensured that the tests accurately reflect scenarios where no active file exists and when context is undefined.
- Improved overall test coverage for message processing in various contexts.

* Add validation tests for tool schemas and improve schema definitions

- Introduced comprehensive validation tests for tool schemas in `allTools.validation.test.ts` and `SearchTools.schema.test.ts` to ensure adherence to best practices and proper typing.
- Enhanced `SearchTools.ts` by replacing `z.any()` with specific object schemas for `chatHistory` to improve type safety.
- Added tests for various tool patterns, including metadata validation and schema best practices, ensuring tools are correctly defined and validated against expected interfaces.

* Add back example content for canvas file in ComposerTools schema

* Make web search tool not overly eager

* Add license key in agent mode (#1663)

* Add tool execution tests and implement Plus subscription checks (#1664)

- Introduced unit tests for the `executeSequentialToolCall` function to validate tool execution behavior, including handling of Plus-only tools.
- Enhanced `executeSequentialToolCall` to check for Plus subscription requirements before executing tools.
- Updated `createTool` and `SimpleTool` interfaces to include an `isPlusOnly` flag, indicating tools that require a Copilot Plus subscription.
- Marked existing tools (e.g., webSearch and YouTube transcription) as Plus-only where applicable.

* Add timezone conversion tools and enhance current time functionality (#1665)

* Add timezone conversion tools and enhance current time functionality

- Introduced `convertTimeBetweenTimezonesTool` to allow conversion of time between different timezones.
- Updated `getCurrentTimeTool` to accept an optional timezone parameter, improving its functionality to return time in specified timezones.
- Enhanced documentation and added examples for both tools in the codebase.
- Added comprehensive unit tests for the new timezone conversion functionality to ensure accuracy and reliability.

* Enhance timezone functionality and add unit tests

- Updated `getCurrentTime` and `convertToTimeInfo` functions to utilize Luxon's offset for accurate timezone calculations.
- Adjusted timezone offset handling to reflect correct signs.
- Added unit tests for Tokyo and New York timezone offsets to ensure accuracy in timezone calculations.

* Add unit test for past time conversion in timezone tool

- Introduced a new test case to verify that converting a past time (6:00 AM PT) to Tokyo correctly reflects the same day (January 15) without erroneously adding a day.
- Updated the `convertTimeBetweenTimezones` function to ensure accurate handling of past times without unintended date shifts.

* Enhance timezone handling and update related tests

- Added `convertTimeBetweenTimezonesTool` to facilitate time conversion between different UTC offsets.
- Updated `getCurrentTimeTool` to accept timezone offsets instead of names, improving accuracy and flexibility.
- Enhanced error handling for invalid timezone offset formats and added comprehensive unit tests to validate new functionality.
- Updated existing tests to reflect changes in timezone offset handling and ensure consistent behavior across various scenarios.

* Refactor timezone handling in convertTimeBetweenTimezones function

- Updated the creation of DateTime objects to interpret parsed dates as already being in the source timezone, improving clarity and accuracy in timezone conversions.

* Fix image in chat history (#1666)

* Refactor chat history handling to preserve multimodal content

- Removed the deprecated `extractChatHistory` function and replaced it with direct access to raw history from memory, allowing for the preservation of multimodal content.
- Updated `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to utilize `BaseMessage` objects for chat history, ensuring that both user and AI messages maintain their original structure and content.
- Enhanced message processing to accommodate multimodal inputs, storing them appropriately for later use.

* Refactor chat history handling to improve message processing

- Introduced `addChatHistoryToMessages` utility to streamline the addition of chat history, ensuring safe handling of various message formats.
- Updated `AutonomousAgentChainRunner` and `CopilotPlusChainRunner` to utilize the new utility, enhancing clarity and maintainability of the code.
- Removed deprecated manual processing of chat history to improve code efficiency and readability.

* Refactor chat history processing to enhance multimodal support

- Removed the deprecated `extractChatHistory` function and replaced it with `processRawChatHistory` to ensure consistent handling of chat history.
- Introduced `processedMessagesToTextOnly` function to extract text-only content from multimodal messages, improving clarity for question condensing.
- Updated `CopilotPlusChainRunner` to utilize the new processing functions, ensuring that both LLM and question condensing use the same data format.
- Added comprehensive unit tests for `processedMessagesToTextOnly` to validate its functionality across various message formats.

* Refactor memory management in BaseChainRunner to enhance context saving

- Removed the creation of `HumanMessage` and `AIMessage` objects for chat history, simplifying the process.
- Implemented `saveContext` for atomic operations, ensuring proper memory management while preserving input and output data.
- Noted that LangChain's memory now expects text content, which limits the saving of multimodal content.

* Remove multimodal content storage from userMessage in AutonomousAgentChainRunner and CopilotPlusChainRunner to streamline message processing. This change aligns with recent refactors to enhance memory management and improve clarity in handling chat history.

* Enhance ChatInput component with autonomous agent toggle button (#1669)

- Added a toggle for the autonomous agent feature, allowing users to enable or disable it within the ChatInput component.
- Updated the state management to synchronize the autonomous agent toggle with user settings.
- Introduced a button for toggling the autonomous agent mode, which is only visible in Copilot Plus mode.
- Refactored tool call logic to accommodate the new autonomous agent toggle, ensuring proper handling of tool calls based on the toggle state.

* Implement custom topic in conversation history (#1670)

* Refactor LoadChatHistoryModal and ChatPersistenceManager for enhanced topic handling

- Updated LoadChatHistoryModal to prioritize custom topics from file frontmatter, falling back to filename extraction if not present.
- Enhanced ChatPersistenceManager to generate AI topics for new chat files and preserve existing topics when updating files.
- Introduced new methods for finding files by epoch and generating AI topics based on conversation content, improving chat file management and organization.

* Refactor LoadChatHistoryModal and ChatPersistenceManager for improved topic handling

- Updated LoadChatHistoryModal to ensure custom topics are trimmed and validated before use.
- Enhanced ChatPersistenceManager to utilize reduce for efficient conversation summary generation, introducing constants for message and character limits to improve readability and maintainability.

* Implement tool call UI banner (#1671)

* Only add tool call to the user message when agent is off

- Updated the tool call logic to only add tool calls when the autonomous agent toggle is off, ensuring that the autonomous agent manages all tools internally when enabled.
- Improved code clarity by adding comments to explain the new logic.

* Implement tool call collapsible UI element

- Introduced a new utility for managing tool call markers, allowing for structured display and tracking of tool calls during execution.
- Updated the AutonomousAgentChainRunner to utilize the new tool call markers, improving clarity in the streaming response and tool execution process.
- Added a ToolCallBanner component for better visualization of tool calls in the chat interface, including execution status and results.
- Refactored ChatSingleMessage to handle tool call updates dynamically, ensuring that the UI reflects the current state of tool calls.
- Enhanced CSS for tool call containers and added animations for improved user experience.

* Fix disappearing tool call in final response

* Implement tool call result formatting

- Updated ChatSingleMessage to use a stable ID for message roots, enhancing memory management and preventing leaks.
- Implemented cleanup logic for old message roots to optimize performance.
- Introduced ToolResultFormatter to format tool results for better display in the UI, ensuring user-friendly output for various tool types.
- Enhanced ToolCallBanner to utilize the new formatter, improving the presentation of tool call results in the chat interface.

* Add YouTube transcription tool in agent and avoid mistrigger from context

- Replaced the deprecated simpleYoutubeTranscriptionTool with a new youtubeTranscriptionTool that automatically extracts YouTube URLs from user messages.
- Removed the processYouTubeUrls method to streamline the handling of YouTube transcriptions.
- Updated tool execution logic to pass the original user message to tools that require it, enhancing flexibility in tool usage.
- Improved the ToolResultFormatter to support multi-URL responses and provide better error handling for transcription failures.
- Enhanced documentation for YouTube transcription usage in the codebase.

* Enhance ToolCallBanner and ToolResultFormatter for improved user experience

- Added animation constants to ToolCallBanner for better visual feedback.
- Updated ToolResultFormatter to robustly handle JSON parsing, improving error handling for both single objects and arrays.
- Implemented input validation in YouTube transcription tool to prevent excessive input lengths and ensure proper data types.

* Refactor tool execution messages and enhance JSON parsing in ToolResultFormatter

- Updated the confirmation message for the writeToFile tool to be more concise.
- Added a private method in ToolResultFormatter to robustly parse JSON strings, improving error handling and ensuring consistent results.
- Streamlined the handling of search results by separating JSON parsing and regex extraction logic in ToolResultFormatter.
- Enforced required user message content in the YouTube transcription tool to prevent potential errors.

* Truncate long tool results for compact memory (#1672)

* Truncate long tool results for compact memory

- Introduced `processToolResults` utility to format tool results for different contexts, allowing for both truncated and full results.
- Updated `AutonomousAgentChainRunner` to utilize the new utility for improved memory management and clarity in tool result presentation.
- Added unit tests for `toolResultUtils` to ensure proper functionality of truncation and formatting methods, enhancing reliability and maintainability of tool result processing.

* Enhance tool results handling in AutonomousAgentChainRunner

- Added a conditional check to ensure that only non-null tool results are pushed to the llmFormattedMessages array, improving the robustness of message formatting and preventing potential errors from undefined values.

* Add Agent settings section (#1676)

* Add autonomous agent settings and tool configurations

- Introduced settings for enabling the autonomous agent mode, including maximum iterations and available tools.
- Updated the CopilotPlusSettings component to allow users to toggle tools like local search, web search, and YouTube transcription.
- Enhanced the model adapter to utilize the new settings for tool management.

* Enhance tool management by adding always available tools and updating tool usage guidelines in autonomous agent mode

* Make filetree tool always available

* Refactor tool management system to use a centralized ToolRegistry, enabling dynamic tool registration and configuration. Update settings structure to support enabled tool IDs and enhance UI for tool selection in the Copilot settings.

* Initialize built-in tools with vault access and improve tool registration logic

* Enhance documentation for initializeBuiltinTools function to clarify tool registration process and dynamic filtering of user-enabled tools.

* fix: Fix the shouldIndexFile method to return all files when inclusions is empty in project mode. (#1667)

* feat: Display all category items by default in the context-manage-modal. (#1657)

* Remove escapingXML for context data (#1678)

* Add apply command (#1677)

* Implement ReplaceInFile tool (#1661)

* Relevant note improvements (#1681)

* Change message scrolling behavior (#1680)

* Fix timetools and tests (#1683)

* Refactor time tool descriptions for clarity and examples

* Update XML escaping tests to reflect correct behavior and remove unnecessary settings mock

* Strip partial tool call tag (#1682)

* Strip partial tool call tag

* Show a placeholder message

* Remove perserveToolIndicators option

* Add tool dynamic prompt (#1679)

* feat: Enhance tool execution error messages and update custom prompt instructions for built-in tools

* feat: Add unit tests for ModelAdapter and enhance tool instructions for time-based queries

* feat: Implement detailed formatting for replaceInFile tool results

* feat: Enhance GPT model instructions and refine replaceInFile tool usage guidelines

* feat: Expand documentation on schema descriptions and custom instructions for tool usage

* feat: Refine tool documentation by clarifying schema descriptions and custom prompt instructions

* feat: Update GitHub Actions workflow to run on all pull requests regardless of target branch

* Fix tests

* feat: Update integration tests to validate writeToFile blocks and enhance replaceInFile usage example

* Enhance image extraction logic to support both wiki-style and markdown image syntax (#1685)

* Turn off agent when in Projects mode (#1686)

* Remove think and action blocks from copy and insert (#1687)

* Remove think and action blocks from copy, and fix composer for thinking models

* Setup AgentTest with correct prompt and tools

* Clean up

* Add a few more tests and validate call order

---------

Co-authored-by: Logan Yang <logancyang@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Zero Liu <zero@lumos.com>
Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com>
2025-08-06 13:42:07 -07:00
.github Implement autonomous agent (#1689) 2025-08-04 09:51:18 -07:00
.husky Add code syntax highlighting and copy code (#520) 2024-08-21 15:04:42 -07:00
__mocks__ Setup Prompt test for Agent (#1688) 2025-08-06 13:42:07 -07:00
images Squash preview (#1502) 2025-05-30 20:06:25 -07:00
src Setup Prompt test for Agent (#1688) 2025-08-06 13:42:07 -07:00
typings Format code (#521) 2024-08-21 15:16:22 -07:00
.editorconfig Create vector store per vault (#348) 2024-03-08 21:50:45 -08:00
.eslintignore Initial commit 2023-03-30 17:15:32 -07:00
.eslintrc Add tw prefix to tailwind (#1497) 2025-05-31 22:30:54 -07:00
.gitignore Enhance abort signal in chains (#1566) 2025-06-23 17:03:53 -07:00
.npmrc Initial commit 2023-03-30 17:15:32 -07:00
.prettierrc Refactor Settings, add formatting (#518) 2024-08-20 21:53:09 -07:00
CLAUDE.md Implement autonomous agent (#1689) 2025-08-04 09:51:18 -07:00
components.json Add tw prefix to tailwind (#1497) 2025-05-31 22:30:54 -07:00
CONTRIBUTING.md Squash preview (#1502) 2025-05-30 20:06:25 -07:00
esbuild.config.mjs Reduce binary size (#1242) 2025-02-11 23:59:05 -08:00
jest.config.js Enhance inclusion/exclusion patterns settings (#1261) 2025-02-18 23:01:20 -08:00
jest.setup.js Avoid full vault scan on incremental indexing (#1148) 2025-02-02 16:24:15 -08:00
LICENSE Add license 2023-03-30 23:52:55 -07:00
local_copilot.md docs: update local copilot instructions for macOS (#1077) 2025-01-22 14:51:34 -08:00
manifest.json Update version to 2.9.4 in manifest.json (#1691) 2025-08-04 12:06:15 -07:00
MESSAGE_ARCHITECTURE.md Implement autonomous agent (#1689) 2025-08-04 09:51:18 -07:00
package-lock.json 2.9.4 (#1622) 2025-07-10 16:50:40 -07:00
package.json 2.9.4 (#1622) 2025-07-10 16:50:40 -07:00
README.md Rename @web to @websearch (#1547) 2025-06-14 17:52:51 -07:00
tailwind.config.js feat: Support project context preview. (#1551) 2025-06-24 12:35:48 -07:00
TODO.md Implement autonomous agent (#1689) 2025-08-04 09:51:18 -07:00
tsconfig.json Include context in user message (#1006) 2025-01-13 23:51:17 -08:00
version-bump.mjs Initial commit 2023-03-30 17:15:32 -07:00
versions.json 2.9.4 (#1622) 2025-07-10 16:50:40 -07:00
wasmPlugin.mjs Format code (#521) 2024-08-21 15:16:22 -07:00

Copilot for Obsidian

The Ultimate AI Assistant for Your Second Brain

GitHub release (latest SemVer) Obsidian Downloads

Documentation | Youtube | Report Bug | Request Feature

Reward Banner

Copilot for Obsidian is your best invault AI assistant, designed to listen, act at the speed of thought, and keep you creating in flow—all within Obsidians integrated, tabfree workspace.

  • 🔒 Your data is 100% yours: Local storage, no ads, and full control of your API keys.
  • 🧠 Elevate your second brain: Tap any OpenAI-compatible or local model to uncover insights, spark connections, and create powerful content.
  • 🌐 Instant multimedia understanding: Drop in webpages, YouTube videos, images, PDFs, or real-time web search for quick insights and summaries.
  • ✍️ Create at the speed of thought: Launch Prompt Palette or edit with AI in one click—your ideas, amplified effortlessly.

Your AI assistant in Obsidian—powerful yet intuitive, keeping you in the creative flow.

Product UI screenshot

Why People Love It ❤️

  • "Copilot is the missing link that turns Obsidian into a true second brain. I use it to draft investment memos with text, code, and visuals—all in one place. Its the first tool that truly unifies how I search, process, organize, and retrieve knowledge without ever leaving Obsidian. With AI-powered search, organization, and reasoning built into my notes, it unlocks insights Id otherwise miss. My workflow is faster, deeper, and more connected than ever—I cant imagine working without it." - @jasonzhangb, Investor & Research Analyst
  • "Since discovering Copilot, my writing process has been completely transformed. Conversing with my own articles and thoughts is the most refreshing experience Ive had in decades.” - Mat QV, Writer
  • "Copilot has transformed our family—not just as a productivity assistant, but as a therapist. I introduced it to my nontechnical wife, Mania, who was stressed about our daughters upcoming exam; within an hour, she gained clarity on her mindset and next steps, finding calm and confidence." - @screenfluent, A Loving Husband

Get Started in 5 Minutes

FREE Product Features

🔌 Install Copilot in Community Plugins in Obsidian

🔑 Set Up Your AI Model (API Key)

  • To start using Copilot AI features, you'll need access to an AI model of your choice.

AI Model API Key
Click the image to watch the video on YouTube

📖 Chat Mode: Summarize Specific Notes

  • 🧠 Use When: You want to reference specific notes or folders, generate content, or talk through ideas with Copilot like a knowledgeable thought partner.

  • 💭 In Chat mode, ask Copilot:

    "Summarize Meeting Notes March and create a follow-up task list based on notes in {projects}."

Chat Mode
Click the image to watch the video on YouTube

📖 Vault QA Mode: Chat With Your Entire Vault

  • 🧠 Use When: You want to search your vault for patterns, ideas, or facts without knowing exactly where the information is stored.

  • 💭 In Vault QA mode, ask Copilot:

    "What insights can I gather about the benefits of journaling from all of my notes?"

  • 💡 Tip: Replace the benefits of journaling with any topic mentioned in your notes to get more precise results.

Vault Mode
Click the image to watch the video on YouTube

📖 Edit and Apply with One Click

  • 🧠 Use When: You want to quickly fix grammar, spelling or wording directly in your notes—without switching tabs or manually rewriting.

  • 💭 Select the text and edit with one RIGHT click

  • 💡 Tip: Set up and customize your right-click menu with common actions you use often, like "Summarize", "Simplify Language", or "Translate to Formal Tone"—so you can apply them effortlessly while you write.

One-Click Commands
Click the image to watch the video on YouTube

📖 Automate your workflow with the Copilot Prompt Palette

  • 🧠 Use When: You want to speed up repetitive tasks like summarizing, rewriting, or translating without typing full prompts every time.

  • 💭 Type / to use Prompt Palette

  • 💡 Tip: Create shortcuts for your most-used actions—like "Translate to Spanish" or "Draft a blog post outline"—and trigger them instantly with typing / !

Prompt Palette
Click the image to watch the video on YouTube

📖 Stay in flow with the Relevant Notes

  • 🧠 Use When: You're working on a note and want to pull in context or insights from related notes—without breaking your focus.

  • 💭 Appears automatically when there's useful related content.

  • 💡 Tip: Use it to quickly reference past research, ideas, or decisions—no need to search or switch tabs.

Relevant Notes
Click the image to watch the video on YouTube

Level Up with Copilot Plus and Beyond

Copilot Plus brings powerful AI agentic capabilities, context-aware actions and seamless tool integration—built to elevate your knowledge work in Obsidian.

🆙 Upgrade to Copilot Plus

First, go to https://www.obsidiancopilot.com/en to subscribe to Copilot Plus. Then, set up Copilot Plus License Key in Obsidian.

Copilot Plus Setup
Click the image to watch the video on YouTube

Community is at the heart of everything we build. Join us on Discord for updates, priority support, and a voice in shaping the best AI products for your experience.

Discord support screenshot

📖 Get Precision Insights From a Specific Time Window

  • 🧠 Use When: You want to quickly review tasks, notes, or ideas from a specific time range without manually digging through files.

  • 💭 In Chat mode, ask Copilot:

    "Give me a recap of everything I captured last week."

  • 💡 Tip: Try variations like "Summarize my highlights from August 11 through August 22" for even more insights.

Time-Based Queries
Click the image to watch the video on YouTube

📖 One Prompt, Every Source—Instant Summaries from PDFs, Videos, and Web

  • 🧠 Use When: You want to combine information from multiple formats—documents, videos, web pages, and images—into one concise, actionable summary.

  • 💭 In PLUS mode, ask Copilot:

    "Please write a short intro of Kiwi birds based on the following information I collected about this animal.

    @youtube Summarize https://www.youtube.com/watch?v=tZ2jm_UPc6c&t=417s in a short paragraph.

    @websearch where can I find Kiwi birds?

    Summarize https://www.doc.govt.nz/nature/native-animals/birds/birds-a-z/kiwi/ in 300 words.“

  • 🛠️ Add PDFs and Images as Context to Enrich Your Learning

  • 💡 Tip: For large PDFs, reference specific sections to focus the AI's attention.

One Prompt, Every Source
Click the image to watch the video on YouTube

💡 Need Help?

  • Check the documentation for setup guides, how-tos, and advanced features.
  • Watch Youtube for walkthroughs.
  • If you're experiencing a bug or have a feature idea, please follow the steps below to help us help you faster:
    • 🐛 Bug Report Checklist
      • ☑️Use the bug report template when reporting an issue
      • ☑️Enable Debug Mode in Copilot Settings → Advanced for more detailed logs
      • ☑️Open the dev console to collect error messages:
        • Mac: Cmd + Option + I
        • Windows: Ctrl + Shift + I
      • ☑️Turn off all other plugins, keeping only Copilot enabled
      • ☑️Attach relevant console logs to your report
      • ☑️Submit your bug report here
    • 💡 Feature Request Checklist
      • ☑️Use the feature request template for requesting a new feature
      • ☑️Clearly describe the feature, why it matters, and how it would help
      • ☑️Submit your feature request here

🙋‍♂️ FAQ

Why isnt Vault search finding my notes?

If you're using the Vault QA mode (or the tool @vault in Plus), try the following:

  • Ensure you have a working embedding model from your AI model's provider (e.g. OpenAI). Watch this video: AI Model Setup (API Key)
  • Ensure your Copilot indexing is up-to-date. Watch this video: Vault Mode
  • If issues persist, run Force Re-Index or use List Indexed Files from the Command Palette to inspect what's included in the index.
  • ⚠️ Dont switch embedding models after indexing—it can break the results.
Why is my AI model returning error code429: Insufficient Quota?

Most likely this is happening because you havent configured billing with your chosen model provider—or youve hit your monthly quota. For example, OpenAI typically caps individual accounts at $120/month. To resolve:

  • ▶️ Watch the “AI Model Setup” video: AI Model Setup (API Key)
  • 🔍 Verify your billing settings in your OpenAI dashboard
  • 💳 Add a payment method if one isnt already on file
  • 📊 Check your usage dashboard for any quota or limit warnings

If youre using a different provider, please refer to their documentation and billing policies for the equivalent steps.

Why am I getting a token limit error?

Please refer to your model providers documentation for the context window size.

⚠️ If you set a large max token limit in your Copilot settings, you may encounter this error.

  • Max tokens refers to completion tokens, not input tokens.
  • A higher output token limit means less room for input!

🧠 Behind-the-scenes prompts for Copilot commands also consume tokens, so:

  • Keep your message length short
  • Set a reasonable max token value to avoid hitting the cap

💡 For QA with unlimited context, switch to the Vault QA mode in the dropdown (Copilot v2.1.0+ required).

💎 Choose the Copilot Plan Thats Right for You

Feature Free Plan Plus Plan 💎 Believer Plan 🛡️
No credit card or sign-up required
All open-source features
Bring your own API key
Best-in-class AI chat in Obsidian
Local data store for Vault QA
Support Essential Pro Elite
AI agent capabilities
Image and PDF support
Enhanced chat UI (context menu)
State-of-the-art embedding models included
Exclusive @AI tools (e.g., web, YouTube)
Exclusive chat model included in plan
Access to exclusive Discord channel
Lifetime access
Priority access to new features
Prioritized feature requests
Exclusive access to next-gen chat & embedding models (coming soon)

🙏 Thank You

If you share the vision of building the most powerful AI agent for our second brain, consider sponsoring this project or buying me a coffee. Help spread the word by sharing Copilot for Obsidian on Twitter/X, Reddit, or your favorite platform!

BuyMeACoffee

Acknowledgments

Special thanks to our top sponsors: @mikelaaron, @pedramamini, @Arlorean, @dashinja, @azagore, @MTGMAD, @gpythomas, @emaynard, @scmarinelli, @borthwick, @adamhill, @gluecode, @rusi, @timgrote, @JiaruiYu-Consilium, @ddocta, @AMOz1, @chchwy, @pborenstein, @GitTom, @kazukgw, @mjluser1, @joesfer, @rwaal, @turnoutnow-harpreet, @dreznicek, @xrise-informatik, @jeremygentles, @ZhengRui, @bfoujols, @jsmith0475, @pagiaddlemon, @sebbyyyywebbyyy, @royschwartz2, @vikram11, @amiable-dev, @khalidhalim, @DrJsPBs, @chishaku, @Andrea18500, @shayonpal, @rhm2k, @snorcup, @JohnBub, @obstinatelark, @jonashaefele, @vishnu2kmohan

Copilot Plus Disclosure

Copilot Plus is a premium product of Brevilabs LLC and it is not affiliated with Obsidian. It offers a powerful agentic AI integration into Obsidian. Please check out our website obsidiancopilot.com for more details!

  • An account and payment are required for full access.
  • Copilot Plus requires network use to faciliate the AI agent.
  • Copilot Plus does not access your files without your consent.
  • Copilot Plus collect server-side telemetry to improve the product. Please see the privacy policy on the website for more details.
  • The frontend code of Copilot plugin is fully open-source. However, the backend code facilitating the AI agents is close-sourced and proprietary.
  • We offer a full refund if you are not satisfied with the product within 14 days of your purchase, no questions asked.

Authors

Brevilabs Team | Email: logan@brevilabs.com | X/Twitter: @logancyang