* Prompt Improvement: Use getFileTree to explore ambiguous notes and folders.
* MInor
* fix: Clarify instructions for using getFileTree in note retrieval process
- Implemented `getUtf8ByteLength` to calculate the byte length of strings in UTF-8, accommodating various character types including ASCII, Cyrillic, CJK, and emojis.
- Introduced `truncateToByteLimit` to truncate strings while respecting UTF-8 character boundaries, ensuring no multibyte characters are cut in half.
- Added unit tests for both functions to validate their behavior across different scenarios, including edge cases.
- Updated `ChatPersistenceManager` to utilize these functions for managing filename lengths, ensuring compliance with filesystem byte limits.
* feat: Improve error handling architecture across chain runners.
# Conflicts:
# src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts
# src/chainFactory.ts
# src/chainUtils.ts
* fix: Preserve correct content ordering in streaming messages
Fix two issues that could confuse users during streaming:
- Error blocks now inserted at correct stream position instead of always appending to end, preserving visual context
- Reset lastDisplayedContent between runs to prevent stale content from appearing in new responses
* fix: fix branch conflicts.
* fix: correctly dispose error block roots by parameterizing disposal helpers with registry.
* Add Amazon Bedrock support
- Introduced new fields for Amazon Bedrock in `aiParams.ts` and `constants.ts`.
- Implemented API key and region validation in `utils.ts`.
- Created `BedrockChatModel` class for handling Bedrock-specific chat model interactions.
- Updated `ChatModelManager` to include Bedrock as a supported provider.
- Enhanced settings management to accommodate Bedrock API key and region.
- Added UI components for Bedrock model configuration in settings dialogs.
- Included unit tests for Bedrock chat model functionality.
This update expands the plugin's capabilities to integrate with Amazon Bedrock, allowing users to utilize its chat model features effectively.
* Support token count for bedrock
* Refactor Amazon Bedrock API key and region validation
- Simplified the API key validation logic in `utils.ts` to only require the API key, as the region defaults to "us-east-1" if not specified.
- Removed redundant checks for base URL and region in `chatModelManager.ts`, streamlining the endpoint construction process.
This update enhances the clarity and efficiency of the Bedrock integration, ensuring a smoother user experience.
* Building persistent memory for copilot (#1848)
* Implement chat input v3 (#1794)
* Fix chat crash React 409 (#1849)
* Fix mobile typeahead menu size (#1853)
* Enhance progress bar and bug fix (#1814)
* feat: Add edit context for progress card.
* fix: When adding a model, do not set the key in the single model setting.
* fix: fix the width of the popover on mobile.
* Refactor Brevilabs API integration to use models base URL (#1855)
- Updated constants to switch from BREVILABS_API_BASE_URL to BREVILABS_MODELS_BASE_URL for model-related API calls.
- Adjusted ChatModelManager and EmbeddingManager to utilize the new models base URL for configuration settings.
* feat: Add new chat history popover. (#1850)
* feat: Add new chat history popover.
* feat: Add open source file button to chat history popover
- Add ArrowUpRight button to each chat history item
- Implement openChatSourceFile method in main.ts
- Add onOpenSourceFile callback prop chain through components
- Open chat files in new Obsidian tabs when clicked
- Optimize error handling to prevent duplicate notices
* feat: Implement localSearch CiC prompting flow in CopilotPlusChainRunner (#1856)
* Properly extract response text in UserMemoryManager (#1857)
* feat: Temporarily disable autocomplete features (#1858)
* chore: Update version to 3.1.0-preview-250927 in manifest.json
* Support space in typeahead trigger and improve search (#1859)
* More chat input enhancement (#1864)
* Fix dropdown color
* Fix badge border in light theme
* Increase search result number
* support paste image
* fix folder context
* Enhance ChatPersistenceManager with filename sanitization tests (#1865)
- Added tests to ensure proper sanitization of wiki link brackets and illegal characters in filenames when saving chat messages.
- Updated filename generation logic to handle empty sanitized topics by defaulting to 'Untitled Chat'.
- Refactored imports in ChatPersistenceManager for better organization.
* Update ApplyView accept button styles for improved visibility and interaction (#1866)
- Adjusted the positioning of the action button to be further from the bottom of the viewport for better accessibility.
- Increased the z-index to ensure the button is always on top of other elements.
- Added a shadow effect to enhance the button's visibility against the background.
* Expose add selection to chat context to free users (#1867)
* New chat input improvements (#1868)
* Improve search logic in at mention search
* Subscribe to file changes
* Detect changes for folders and tags
* Make context badge tooltip always show
* Add full note path in preview
* Fix enter being blocked when no search result
* Rebuild "Active Note" and more chat input improvement (#1873)
* Performance improvement and extend "active note" to note typeahead menu (#1875)
* Add better agent prompt logging (#1882)
* Add read tool (#1883)
* Add readNote tool for reading notes in chunks
* Enhance readNote tool to support dynamic note path display and linked note extraction
* Refine readNote tool instructions for improved clarity and efficiency in note content retrieval
* Add support for readNote tool: integrate emoji, format results, and refine instructions
* Add inline citation reminder functionality to user questions
* Add an alert when user hits the maxToken limit (#1884)
* Refactor AutonomousAgentChainRunner (#1887)
* Refactor AutonomousAgentChainRunner to enhance agent workflow, streamline context preparation, and improve response handling
* Enhance type safety in addChatHistoryToMessages by specifying message structure
* Improve max iteration limit message for clarity and formatting
* Refactor tool display name handling for improved clarity and maintainability
* Refactor tool call ID generation and visibility handling for improved clarity and uniqueness
* Fix agent tool call ID (#1890)
* Enhance tool call ID generation and improve readNote handling in prompts
Update readNote description for clarity and improve prompt instructions
* Update instructions for registerFileTreeTool to clarify usage guidelines
* Clarify instructions for readNote tool to improve context inference and handling of partial note titles
* Add token counter (#1889)
* Fix index rebuild on semantic search toggle (#1891)
* Update token counter label (#1892)
* Implement tag search v3 (#1893)
* feat: Enhance TieredLexicalRetriever to support tag-based retrieval and improve search scoring
- Added support for returning all matching tags in TieredLexicalRetriever.
- Introduced new options for tag terms and returnAllTags in the retriever.
- Updated FullTextEngine to prioritize tag matches and improve scoring for documents with tags.
- Enhanced tokenization to handle hierarchical tags and prevent splitting hyphenated tags.
- Improved handling of frontmatter tags in documents.
- Updated search tools to accommodate new tag handling features.
- Added tests to verify the functionality of tag-based retrieval and scoring improvements.
* fix: Improve logging for query expansion in SearchCore
* feat: Add explanation for non-tag matches in FullTextEngine search results
* feat: Enhance QueryExpander to preserve standalone terms in tag handling and improve term validation
* feat: Enhance QueryExpander and TieredLexicalRetriever to improve tag handling and standalone term extraction
* Update version to 3.1.0-preview-251006 in manifest.json
* Normalize tag queries for case-insensitive matching and improve search functionality (#1894)
* Implement merge retriever (#1896)
* Fix note read (#1897)
* Enhance note resolution logic and add tests for wiki-linked notes and basename matching
* Implement note resolution outcome types and enhance readNoteTool tests for ambiguous matches
* Add deriveReadNoteDisplayName function and enhance readNoteTool tests for edge cases
* Fix note tool UI freeze (#1898)
* Update how tags work in context (#1895)
* Enhance file creation instructions in modelAdapter and update tool usage guidelines in builtinTools (#1901)
- Added instructions for confirming folder existence before creating new files in modelAdapter.
- Updated custom prompt instructions in builtinTools to clarify the use of getFileTree for folder lookups when creating new notes.
* Do not add url context for youtube url (#1899)
* Update YouTube Script command and modal title to indicate Plus feature (#1903)
* Enhance ChatPersistenceManager to handle file save conflicts (#1904)
* Enhance ChatPersistenceManager to handle file save conflicts and improve epoch handling
* Fix type checking for existing files in ChatPersistenceManager to prevent errors
* Fix verify add (#1905)
- Rename "Verify" button to "Test" in Add Model dialog
- Make verification not required for adding model in Set Keys
- Upgrade chatAnthropic client to fix the Top P -1 error for Claude Opus models
* Enhance XML parsing to handle tool calls missing closing tags
---------
Co-authored-by: Wenzheng Jiang <jwzh.hi@gmail.com>
Co-authored-by: Zero Liu <zero@lumos.com>
Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com>
* feat: add mobile-responsive components for settings.
* fix: Restrict HelpTooltip components to only affect mobile devices.
* fix: fix the args.diff case test.
* Refactor active note inclusion logic
* Refactor IndexEventHandler to manage event listeners based on semantic search settings
- Introduced `shouldHandleEvents` method to determine if indexing-related events should be processed.
- Replaced `initializeEventListeners` with `syncEventListeners` to conditionally register listeners based on settings.
- Added `teardownEventListeners` method to clean up listeners and reset state when not needed.
- Subscribed to settings changes to dynamically manage event listeners.
* Refactor constants and settings to enhance folder management
- Introduced `COPILOT_FOLDER_ROOT` constant to streamline folder path definitions.
- Updated `DEFAULT_CHAT_HISTORY_FOLDER` and `DEFAULT_CUSTOM_PROMPTS_FOLDER` to use the new root constant for improved organization.
- Added `DEFAULT_QA_EXCLUSIONS_SETTING` to ensure consistent exclusion of the Copilot folder in QA settings.
- Implemented `sanitizeQaExclusions` function to normalize QA exclusion patterns, ensuring the Copilot folder is properly excluded from settings.
* Update log file management to reduce in-memory buffer size
- Decreased the in-memory ring buffer from 1000 to 500 entries to optimize memory usage.
- Adjusted documentation to reflect the new buffer size for clarity.
- Updated log truncation logic in BaseChainRunner to use a constant for maximum log length, enhancing maintainability.
* Add unit tests for sanitizeQaExclusions function
- Introduced a new test file `model.test.ts` to validate the behavior of the `sanitizeQaExclusions` function.
- Added tests to ensure proper handling of non-string values, distinct slash-only patterns, and normalization of trailing slashes.
- Enhanced the `sanitizeQaExclusions` function in `model.ts` to improve key handling and normalization logic for QA exclusion patterns.
* Implement inline citation for vault search for both vault QA and Plus chains
refactor: enhance citation utilities and improve source handling in chain runners
- Introduced new utility functions for citation management, including sanitization and formatting of source catalogs.
- Updated CopilotPlusChainRunner and VaultQAChainRunner to utilize the new citation utilities for improved citation handling.
- Enhanced ChatSingleMessage component to consolidate duplicate sources and ensure accurate citation numbering.
- Added comprehensive tests for citation utilities to ensure functionality and edge case handling.
refactor: dedupe chunks for unique titles in search tool sources
refactor: enhance citation handling in CopilotPlus and VaultQA chain runners
- Introduced a sanitization function to remove pre-existing citation markers from document content, preventing number leakage.
- Updated the formatting of retrieved documents in VaultQA to utilize the new sanitization function for cleaner output.
- Improved footnote renumbering logic in ChatSingleMessage to prioritize first-mention order, ensuring accurate citation mapping.
refactor: streamline source citation handling in ChatSingleMessage component and enhance web search tool instructions
- Simplified the processing of source links in ChatSingleMessage to use a more efficient mapping approach.
- Updated the web search tool's instruction to clarify the use of footnote-style citations, ensuring compatibility with Markdown formatting.
* feat: add inline citation setting toggle and refactor
- Introduced a new setting to enable inline citations in AI-generated responses, allowing for footnote-style references within the text.
- Updated citation utilities to manage fallback sources and improve citation instructions based on user settings.
- Enhanced CopilotPlusChainRunner and VaultQAChainRunner to utilize the new inline citation feature, ensuring accurate source representation.
- Modified ChatSingleMessage component to process inline citations based on the new setting, improving the display of sources in chat messages.
- Added tests for new citation functionalities to ensure reliability and correctness.
* feat: enhance citation utilities and add tests for normalization
- Introduced the `normalizeCitations` function to improve citation formatting by removing periods after citations and handling footnote references.
- Updated `processInlineCitations` to ensure correct mapping of non-sequential citations and consolidate duplicate sources.
- Added comprehensive tests for the new citation functionalities, ensuring accurate citation processing and representation in AI-generated responses.
- Modified QASettings description for clarity regarding the inline citations feature.
* fix: address critical bug in citation processing for consecutive citations
- Enhanced the `normalizeCitations` function to correctly handle consecutive citations without spaces, ensuring accurate mapping and representation in output.
- Added a test case to replicate and validate the fix for the identified bug, confirming that both citations are processed and displayed correctly.
- Updated existing citation handling logic to improve robustness and prevent unconverted citations in the output.
* refactor: improve citation handling and enhance citation utilities
- Refactored `CopilotPlusChainRunner` to streamline citation source management, replacing `lastCitationLines` with `lastCitationSources` for better clarity and structure.
- Updated the `addFallbackSources` function to handle citation formatting more effectively, ensuring proper display of titles and paths.
- Enhanced the `normalizeCitations` function to support multiple citations in brackets, ensuring correct sorting and mapping of citation numbers.
- Added new test cases to validate the handling of alternate headings, HTML summaries, and multiple citations, improving overall robustness of citation processing.
- Introduced `getWebSearchCitationInstructions` for web-specific citation guidance, enhancing user experience with clear instructions for inline citations.
* fix: move documentations
* refactor: normalize footnote rendering in ChatSingleMessage component
- Introduced `normalizeFootnoteRendering` function to streamline footnote display by removing separators and backreferences, and fixing numbering artifacts.
- Updated `ChatSingleMessage` to utilize the new normalization function, enhancing the clarity and presentation of footnotes in chat messages.
- Added unit tests for `normalizeFootnoteRendering` to ensure proper functionality and handling of various footnote scenarios.
* test: enhance ChatSingleMessage tests and add deduplication tests for search tools
- Updated `ChatSingleMessage.test.tsx` to include `TooltipProvider` for improved rendering context.
- Refactored mock implementation for `obsidian` to streamline testing setup.
- Introduced new test file `SearchTools.dedupe.test.ts` to validate the deduplication logic in search tool sources, ensuring correct handling of duplicate entries while preserving the highest score.
- Added tests to verify deduplication behavior for various scenarios, enhancing overall test coverage and reliability.
- Add search explanation table in log
- Clean up log for log file
- Implement open and clear log file commands
- Exclude log file from search (both lexical and semantic)
- Replaced Buffer.from(...).toString("base64") with arrayBufferToBase64 in brevilabsClient, Chat, and ImageProcessor files for consistency and improved readability.
* Switch to the new file when creating files with composer tools.
* Handle file creation race condition with improved error handling
Co-authored-by: jwzh.hi <jwzh.hi@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
- Implemented `extractNoteContent` method to gather content from `<note_context>` and `<active_note>` blocks, excluding `<url_content>`.
- Updated image processing logic to utilize extracted note content, ensuring relevant images are included when the `passMarkdownImages` setting is enabled.
