mirror of
https://github.com/logancyang/obsidian-copilot.git
synced 2026-07-22 07:50:24 +00:00
16 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zero Liu
|
6ca2dc01ea
|
chore(eslint): enable no-explicit-any; fix ~395 violations (#2452)
* chore(eslint): enable no-explicit-any; fix ~395 violations Switches @typescript-eslint/no-explicit-any from "off" to "error" and replaces explicit `any` with proper types or `unknown` + narrowing across ~100 source files and 15 test files. Eleven eslint-disable comments remain: one for a BaseLanguageModel prototype patch and ten for Orama<any> API surfaces where typed alternatives poison Orama's internal inference to `never`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(scripts): cast chainContext to ChainManager in printPromptDebugEntry The earlier `as any` → `as unknown` rewrite left buildAgentPromptDebugReport's chainManager argument typed as `unknown`, which CI's `tsc -noEmit` (without `--skipLibCheck`) flagged. Restore the cast through the real ChainManager type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(eslint): fix remaining no-explicit-any and unnecessary-type-assertion errors Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(eslint): drop redundant no-explicit-any rule Already set to "error" by typescript-eslint/recommendedTypeChecked via obsidianmd's recommended config. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Zero Liu
|
08c14440dc
|
chore(lint): enable obsidianmd/rule-custom-message and fix violations (#2416)
* chore(lint): enable obsidianmd/rule-custom-message and fix violations Turn on the obsidianmd/rule-custom-message ESLint rule (which wraps no-console to enforce logInfo/logWarn/logError over console.log per AGENTS.md). Swap console.log → logInfo across LLMProviders/ and search/, and delete console.log noise from test files. src/logger.ts, src/chainFactory.ts (circular import with constants), and scripts/** are exempted via narrow file overrides. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): enable obsidianmd/no-unsupported-api (#2414) Removes the deferred "off" override so the rule runs at the recommended severity on .ts/.tsx. With manifest.minAppVersion=1.4.0 and obsidian@1.2.5 (no @since tags) the rule fires on nothing today — it acts as a guard for future obsidian type-stub bumps. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): enable obsidianmd/object-assign rule (#2415) Follow-up to #2410. The rule only flags `Object.assign(<ident-containing-default>, <non-object-literal>)` — the Obsidian anti-pattern of mutating DEFAULT_SETTINGS. No call sites in this repo match, so enabling produces zero new errors. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract ChainType into src/chainType.ts to break import cycle constants.ts → chainFactory.ts → logger.ts → settings/model.ts → constants.ts formed a runtime cycle because chainFactory.ts (a heavy LangChain module) re- exported the ChainType enum that constants.ts needed at module load. Move the enum into a tiny standalone file, update all 26 importers, and restore logInfo in chainFactory.ts. The eslint override for chainFactory.ts is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Zero Liu
|
7619fb5578
|
chore(promises): explicit handling of floating promises (W2/9) (#2407)
Third of nine workspaces splitting #2397. Surfaces previously swallowed promise rejections via .catch(logError) or `void` prefix. No logic changes - only error-handling explicitness. - ~67 floating promises now use void / .catch(logError) - Promise-returning callbacks where void was expected are wrapped - ConfirmModal accepts onConfirm/onCancel returning void | Promise<void>; rejections logged via logError - Custom command runners wrap `result instanceof Promise` paths - ChatUIState.replaceMessages is now async to match its delegate - ChainManager initialize / createChainWithNewModel are explicitly voided with .catch(logError) at call sites - dbOperations subscribeToSettingsChange uses void IIFE with try/catch Behavior change: errors previously hidden by floating promises now log via @/logger. No code paths changed. W0, W1, and #2402 already merged. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Logan Yang
|
e3d47adfa2
|
feat(lm-studio): use Responses API for LM Studio models (#2306)
Switch LM Studio from /v1/chat/completions to /v1/responses via a thin ChatLMStudio wrapper that patches LangChain compatibility issues (text.format requirement, strict:null in tool definitions). - New ChatLMStudio class with fetch wrapper for tool sanitization - Opt-out toggle in model settings (useResponsesApi) - Ping uses ChatLMStudio to test the correct endpoint - ThinkBlockStreamer: strip special tokens from text content Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Logan Yang
|
fff6b4efd0
|
chore: rename docs to designdocs and nest todo folder (#2252)
- Renamed docs/ to designdocs/ - Moved draft/todo docs into designdocs/todo/ subfolder - Added OBSIDIAN_CLI_INTEGRATION.md from master's todo/ folder - Updated all docs/ path references in CLAUDE.md, AGENTS.md, and designdocs Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
795ed5ff6f
|
fix: hard cap L1 project context + payload diagnostics (#2192)
* docs: add token budget enforcement analysis and fix plan Document root cause of context window overflow (2.7M tokens sent to 1M model): L4 chat history bypasses all compaction systems. Add fix plan with phased approach to enforce token budget at message assembly point. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: correct root cause analysis — L1 unbudgeted, not L4 Previous analysis incorrectly blamed L4 chat history as the primary culprit. Investigation shows L4 stores only bare L5 text + compacted responses. The real issue is systemic: no total payload enforcement, with L1 (project context) being the largest unbudgeted layer and PROJECT_COMPACT_THRESHOLD being blind to L1 size. Updated fix plan to be model-agnostic (use autoCompactThreshold as single budget, no model-specific lookup tables), added contextTurns deprecation, and history guarantee principle. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: hard cap L1 project context at 600k tokens + payload diagnostics - Truncate project context at 2.4M chars (~600k tokens) to prevent total payload from exceeding model context windows (temporary fix until full token budget enforcement is implemented) - Add per-layer token estimate logging when payload exceeds 2M chars to help diagnose context window overflow reports - Document Obsidian CLI dev tools in CLAUDE.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
8cea1a6f17
|
Audit context envelope, tag alignment, artifact dedup, and logging (#2164)
* fix: align context block tags and make parseContextIntoSegments registry-driven
The context envelope L2 layer was silently dropping URL/media context across
turns due to three compounding issues:
1. Tag mismatch: Mention.ts created <youtube_transcript> blocks but the
contextBlockRegistry expected <youtube_video_context>. Fixed the tag and
inner element (<transcript> → <content>) to match the registry.
2. Missing registration: twitter_content blocks had no registry entry, so
they were invisible to compaction and segment parsing. Added the entry.
3. Hardcoded parser: parseContextIntoSegments only matched <note_context>,
<active_note>, and <prior_context> via hardcoded regexes, silently
ignoring all URL/media block types. Rewrote it to dynamically build its
regex from CONTEXT_BLOCK_TYPES, ensuring any future block type added to
the registry is automatically handled.
Extracted parseContextIntoSegments into src/context/parseContextSegments.ts
as a standalone pure function for direct testability.
Added 19 new tests covering all registered block types, mixed blocks,
stable/unstable flags, and a registry completeness guard test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: context envelope improvements - regeneration, dedup, per-artifact IDs, XML escaping
Phase A: Fix regeneration from loaded chats by lazily reprocessing context
when contextEnvelope is missing (e.g., messages loaded from disk).
Phase B: Replace static segment IDs ("urls", "selected_text", "web_tabs")
with per-artifact IDs from parseContextIntoSegments, enabling accurate
smart referencing in LayerToMessagesConverter.
Phase C: Deduplicate L2 content by segment ID (last-write-wins) to prevent
linear growth when the same artifact appears across multiple turns.
Phase D: Escape note title/basename in XML context blocks for consistency
with URL/YouTube content. Keep path unescaped as it's used as identifier.
Additional fixes from Codex review:
- Unique IDs for blocks without source extractors (selected_text counter)
- Keep note.path raw to avoid dedup mismatch with TFile.path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent context artifact duplication in LLM payload and L4 memory
CopilotPlusChainRunner was re-injecting processedText (which includes
context XML artifacts) into the user message via ensureUserQueryLabel,
bypassing the envelope's clean L2/L3/L5 separation. This caused context
like YouTube transcripts to appear 3x in the LLM payload.
Fix: use L5_USER envelope text (expanded user query without context XML)
instead of processedText for cleanedUserMessage, trimmedQuestion fallback,
and messageForAnalysis. Also fix BaseChainRunner.handleResponse to save
L5 text to L4 memory instead of processedText.
Other changes:
- Remove dead updateMemoryWithLoadedMessages from chainManager
- Update CONTEXT_ENGINEERING.md with example walkthrough, L4 behavior
docs, and Phase 6 integration test suite roadmap
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add chain runner envelope usage section and fix Step 0 log
Document per-runner envelope behavior, CopilotPlus/Agent tool flows,
and token efficiency audit in CONTEXT_ENGINEERING.md. Fix Step 0 log
to show L5 user query instead of processedText with context XML.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: trim trailing whitespace from chat input display text
Trailing newlines from Lexical editor were preserved in displayText
and rendered as empty lines in chat bubbles due to whitespace-pre-wrap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Logan Yang
|
308c88332d
|
Address quick ask (#2146)
* chore: translate Chinese comments to English and apply formatting - Translate all Chinese comments in replaceGuard.ts to English for consistency - Apply Prettier formatting to 33 files Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add unit tests for Quick Ask core modules Add comprehensive tests for: - replaceGuard.ts: validation logic, error messages, MapPos and Highlight strategies - persistentHighlight.ts: CM6 state field, range mapping, show/hide behavior - quickCommandPrompts.ts: placeholder appending logic 64 new test cases covering edge cases like: - Leaf/editor/file change detection - Document bounds validation - Content change detection - Range mapping through document changes - Case-insensitive placeholder detection Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Logan Yang
|
a576ad7986
|
feat: implement auto-compact context with map-reduce summarization (#2106)
- Add ContextCompactor class for map-reduce style context summarization - Implement auto-compact when context exceeds configurable threshold - Track note paths for tags/folders in L3 segment metadata for L2 deduplication - Add loadAndAddChatHistory for streamlined chat history loading - Store compacted paths in envelope metadata for multi-turn deduplication - Fix: Only track L3 context paths in compactedPaths (excludes L5 user message files) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Emt-lin
|
bb2ebf1c9b
|
feat: Add comprehensive system prompt management system. (#1969)
* feat: Add comprehensive system prompt management system. * fix：fixed some bugs * fix: prevent false negatives in migration content verification Strip leading newlines before comparing saved vs original content. Obsidian may insert extra blank line after frontmatter block, causing stripFrontmatter to leave a leading newline and fail verification. |
||
|
Emt-lin
|
ff3636637f
|
feat: Add Web Viewer bridge for referencing open web tabs in chat (#2096)
* feat: Add Web Viewer bridge for referencing open web tabs in chat. * refactor: Simplify markdown image extraction parser。 * fix: Prevent web selection from auto-reappearing after removal or new chat. * fix: Suppress active web tab when web selection exists. * feat: Add YouTube transcript extraction for Web Viewer tabs. * feat: Built-in slash prompts for web clippers. * feat: Improve web selection tracking and simplify context settings - Merge context settings: combine note/web active content and selection settings - Add auto-clear for web selection badge when deselected on page - Make note and web selections mutually exclusive - Show website favicon in web selection badge - Add migration logic and tests for settings |
||
|
Logan Yang
|
ed884298ba
|
Integrate ProjectChainRunner and ChatManager with new layered context (#1973)
* Integrate ProjectChainRunner and ChatManager with new layered context - Introduced `ProjectChainRunner` to automatically include project context in L1 via `ChatManager.getSystemPromptForMessage()`, simplifying the class structure by inheriting behavior from `CopilotPlusChainRunner`. - Updated `ChatManager` to build system prompts that append project context when in project mode, ensuring seamless integration without special-case logic. - Implemented null guards for project context to prevent erroneous output when context is unavailable. - Refactored documentation to reflect changes in context handling and the new project chain runner functionality. * Enhance ChatManager tests with project context mocks - Added mocks for project context retrieval and system prompt settings in `ChatManager.test.ts` to facilitate testing without relying on actual project data. - Improved test coverage for context-related functionalities by simulating project context and system prompt retrieval, ensuring robust testing of the ChatManager's behavior in various scenarios. |
||
|
Logan Yang
|
05beb941eb
|
Context revamp (#1971)
* Implement context engine for a layered context * Wire in new context library to LLM chain - Updated the context engine to maintain a cumulative library of all context items (L2) without deduplication, enhancing cache stability. - Introduced smart referencing in the user message (L3) to reference existing items in L2 by ID and include full content for new items. - Improved the LayerToMessagesConverter to format and handle layered context envelopes for better debugging and clarity. - Fixed critical bugs related to context deduplication that previously broke cache stability. - Enhanced tests to validate the new context handling and ensure proper functionality across various scenarios. * Enhance VaultQAChainRunner with tag extraction and inline citation detection - Integrated tag extraction from user queries to improve context-aware retrieval. - Implemented `hasInlineCitations` function to detect inline citation markers in responses. - Updated message construction logic to utilize envelope-based context, enhancing the quality of AI interactions. - Added comprehensive tests for the new inline citation detection functionality, ensuring robustness and accuracy. - Improved logging for better debugging and traceability of the message processing flow. Refactor VaultQAChainRunner to enhance context handling and citation instructions - Integrated LayerToMessagesConverter for improved message construction, ensuring smart referencing and context preservation. - Updated logging to reflect the use of the new converter in envelope-based context construction. - Revised comments for clarity on context preparation and citation instructions. - Minor adjustment in citation utility to improve regex handling for citation updates. * Complete migration to envelope-based context in CopilotPlus and LLM chain runners - Removed all legacy fallback paths, ensuring that context envelopes are now mandatory for operations in CopilotPlusChainRunner and LLMChainRunner. - Enhanced image extraction to only pull from the active note, preventing unintended leaks from attached context notes. - Fixed critical issues with context-in-context (CiC) formatting and added early envelope guards in VaultQAChainRunner to prevent silent failures. - Updated message construction logic to utilize LayerToMessagesConverter for improved smart referencing and context preservation. - Comprehensive tests added to validate the new envelope-based context handling and ensure robustness across all chain runners. * Integrate Plus chain with context envelope * Enhance AutonomousAgentChainRunner with envelope-based context integration - Introduced `LayerToMessagesConverter` for improved message construction, ensuring system and user messages are derived from the context envelope. - Implemented envelope validation to ensure context is available before processing. - Updated message preparation logic to maintain consistency with the envelope-first architecture, preserving tool execution and multimodal support. - Added comprehensive logging for envelope-based context construction to aid debugging. - Documented changes in the context engineering documentation to reflect the new integration and design points. * Support loading saved chat context to L2 context library * Fix ChatPersistenceManager tests for context handling |
||
|
Logan Yang
|
5568b70e0f
|
Merge 3.1.0 preview (#1906)
* Building persistent memory for copilot (#1848) * Implement chat input v3 (#1794) * Fix chat crash React 409 (#1849) * Fix mobile typeahead menu size (#1853) * Enhance progress bar and bug fix (#1814) * feat: Add edit context for progress card. * fix: When adding a model, do not set the key in the single model setting. * fix: fix the width of the popover on mobile. * Refactor Brevilabs API integration to use models base URL (#1855) - Updated constants to switch from BREVILABS_API_BASE_URL to BREVILABS_MODELS_BASE_URL for model-related API calls. - Adjusted ChatModelManager and EmbeddingManager to utilize the new models base URL for configuration settings. * feat: Add new chat history popover. (#1850) * feat: Add new chat history popover. * feat: Add open source file button to chat history popover - Add ArrowUpRight button to each chat history item - Implement openChatSourceFile method in main.ts - Add onOpenSourceFile callback prop chain through components - Open chat files in new Obsidian tabs when clicked - Optimize error handling to prevent duplicate notices * feat: Implement localSearch CiC prompting flow in CopilotPlusChainRunner (#1856) * Properly extract response text in UserMemoryManager (#1857) * feat: Temporarily disable autocomplete features (#1858) * chore: Update version to 3.1.0-preview-250927 in manifest.json * Support space in typeahead trigger and improve search (#1859) * More chat input enhancement (#1864) * Fix dropdown color * Fix badge border in light theme * Increase search result number * support paste image * fix folder context * Enhance ChatPersistenceManager with filename sanitization tests (#1865) - Added tests to ensure proper sanitization of wiki link brackets and illegal characters in filenames when saving chat messages. - Updated filename generation logic to handle empty sanitized topics by defaulting to 'Untitled Chat'. - Refactored imports in ChatPersistenceManager for better organization. * Update ApplyView accept button styles for improved visibility and interaction (#1866) - Adjusted the positioning of the action button to be further from the bottom of the viewport for better accessibility. - Increased the z-index to ensure the button is always on top of other elements. - Added a shadow effect to enhance the button's visibility against the background. * Expose add selection to chat context to free users (#1867) * New chat input improvements (#1868) * Improve search logic in at mention search * Subscribe to file changes * Detect changes for folders and tags * Make context badge tooltip always show * Add full note path in preview * Fix enter being blocked when no search result * Rebuild "Active Note" and more chat input improvement (#1873) * Performance improvement and extend "active note" to note typeahead menu (#1875) * Add better agent prompt logging (#1882) * Add read tool (#1883) * Add readNote tool for reading notes in chunks * Enhance readNote tool to support dynamic note path display and linked note extraction * Refine readNote tool instructions for improved clarity and efficiency in note content retrieval * Add support for readNote tool: integrate emoji, format results, and refine instructions * Add inline citation reminder functionality to user questions * Add an alert when user hits the maxToken limit (#1884) * Refactor AutonomousAgentChainRunner (#1887) * Refactor AutonomousAgentChainRunner to enhance agent workflow, streamline context preparation, and improve response handling * Enhance type safety in addChatHistoryToMessages by specifying message structure * Improve max iteration limit message for clarity and formatting * Refactor tool display name handling for improved clarity and maintainability * Refactor tool call ID generation and visibility handling for improved clarity and uniqueness * Fix agent tool call ID (#1890) * Enhance tool call ID generation and improve readNote handling in prompts Update readNote description for clarity and improve prompt instructions * Update instructions for registerFileTreeTool to clarify usage guidelines * Clarify instructions for readNote tool to improve context inference and handling of partial note titles * Add token counter (#1889) * Fix index rebuild on semantic search toggle (#1891) * Update token counter label (#1892) * Implement tag search v3 (#1893) * feat: Enhance TieredLexicalRetriever to support tag-based retrieval and improve search scoring - Added support for returning all matching tags in TieredLexicalRetriever. - Introduced new options for tag terms and returnAllTags in the retriever. - Updated FullTextEngine to prioritize tag matches and improve scoring for documents with tags. - Enhanced tokenization to handle hierarchical tags and prevent splitting hyphenated tags. - Improved handling of frontmatter tags in documents. - Updated search tools to accommodate new tag handling features. - Added tests to verify the functionality of tag-based retrieval and scoring improvements. * fix: Improve logging for query expansion in SearchCore * feat: Add explanation for non-tag matches in FullTextEngine search results * feat: Enhance QueryExpander to preserve standalone terms in tag handling and improve term validation * feat: Enhance QueryExpander and TieredLexicalRetriever to improve tag handling and standalone term extraction * Update version to 3.1.0-preview-251006 in manifest.json * Normalize tag queries for case-insensitive matching and improve search functionality (#1894) * Implement merge retriever (#1896) * Fix note read (#1897) * Enhance note resolution logic and add tests for wiki-linked notes and basename matching * Implement note resolution outcome types and enhance readNoteTool tests for ambiguous matches * Add deriveReadNoteDisplayName function and enhance readNoteTool tests for edge cases * Fix note tool UI freeze (#1898) * Update how tags work in context (#1895) * Enhance file creation instructions in modelAdapter and update tool usage guidelines in builtinTools (#1901) - Added instructions for confirming folder existence before creating new files in modelAdapter. - Updated custom prompt instructions in builtinTools to clarify the use of getFileTree for folder lookups when creating new notes. * Do not add url context for youtube url (#1899) * Update YouTube Script command and modal title to indicate Plus feature (#1903) * Enhance ChatPersistenceManager to handle file save conflicts (#1904) * Enhance ChatPersistenceManager to handle file save conflicts and improve epoch handling * Fix type checking for existing files in ChatPersistenceManager to prevent errors * Fix verify add (#1905) - Rename "Verify" button to "Test" in Add Model dialog - Make verification not required for adding model in Set Keys - Upgrade chatAnthropic client to fix the Top P -1 error for Claude Opus models * Enhance XML parsing to handle tool calls missing closing tags --------- Co-authored-by: Wenzheng Jiang <jwzh.hi@gmail.com> Co-authored-by: Zero Liu <zero@lumos.com> Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com> |
||
|
Logan Yang
|
bc2d7f660a
|
Implement autonomous agent (#1689)
* Implement autonomous agent mode and new message architecture - Implement autonomous agent mode with sequential thinking capabilities - Migrate from sharedState to new clean message management architecture - Add ChatManager, MessageRepository, and ContextManager for better separation of concerns - Implement project-based chat isolation with separate message repositories per project - Add ChatPersistenceManager for saving/loading chat history - Refactor chain runner architecture with specialized runners (Autonomous, Project, VaultQA) - Add XML tool call handling for better compatibility across LLM providers - Introduce vault and web search toggles in settings - Enhance model adapters with explicit tool usage instructions - Add comprehensive test coverage for new architecture components - Update UI components to use new ChatUIState instead of SharedState 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Enhance context processing and documentation - Added detailed XML structure for context notes, including metadata (creation and modification times) for notes and selected text. - Introduced new tests for message lifecycle and XML tag formatting to ensure proper context handling. - Created TODO.md to track technical debt and future improvements related to context processing and file handling. - Updated CLAUDE.md to reference the new TODO.md for technical debt awareness. This commit improves the clarity and functionality of context processing in messages, ensuring rich context is maintained throughout the message lifecycle. * Fix composer instruction in new message implementation * Add writeToFile tool to agent (#1632) * Add writeToFile tool call to agent * Update src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/tools/ComposerTools.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove unuseful comment --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix tests * Enhance context processing with new XML tags for notes and selected text (#1633) * Enhance context processing with new XML tags for notes and selected text - Introduced new constants for XML tags: SELECTED_TEXT_TAG, VARIABLE_TAG, VARIABLE_NOTE_TAG, EMBEDDED_PDF_TAG, VAULT_NOTE_TAG, and RETRIEVED_DOCUMENT_TAG. - Updated context processing to format embedded PDFs and notes with XML structure. - Modified prompt processing to include XML tags for selected text and variables, improving clarity and structure in generated outputs. - Enhanced tests to validate the new XML formatting in processed prompts. * Implement XML escaping utility functions and integrate into context processing - Added `escapeXml` and `escapeXmlAttribute` functions to handle XML special character escaping, enhancing security against XML injection. - Updated context processing to utilize these functions for escaping note titles, paths, and content, ensuring proper XML formatting. - Enhanced tests to validate XML escaping functionality across various scenarios, including selected text and variable names. - Introduced new test files for XML utility functions to ensure comprehensive coverage and reliability. * Refactor XML utility functions and update imports - Moved `escapeXml` and `escapeXmlAttribute` functions from `utils/xmlUtils` to `LLMProviders/chainRunner/utils/xmlParsing`, enhancing modularity. - Updated all relevant imports across the codebase to reflect the new location of XML utility functions. - Removed obsolete `xmlUtils` file and its associated tests, consolidating XML handling in a single module for better maintainability. * Add message creation callback in ChatManager for immediate UI updates (#1634) - Introduced a new method `setOnMessageCreatedCallback` in ChatManager to allow setting a callback that triggers when a message is created. - Updated ChatUIState to utilize this callback for notifying listeners, ensuring the UI updates immediately upon message creation. - This enhancement improves the responsiveness of the chat interface by synchronizing message creation events with UI state updates. * Enhance TimeTools tests for invalid expressions - Added setup to mock console.warn in tests for invalid time expressions. - Verified that the appropriate warning message is logged when parsing fails. - Improved test coverage for handling various invalid time inputs. * Refactor local search handling and restore sources display - Unified tool handling in CopilotPlusChainRunner to fix websearch results being ignored when localSearch has results - All tool outputs now go through the same prepareEnhancedUserMessage method - Preserved QA format and instruction when only localSearch is used - Added proper validation for localSearch results (non-empty documents array) - Maintained time expression support for temporal queries - Fixed source tracking and display in ChatUIState - Enhanced error handling for JSON parsing of search results * Enhance AutonomousAgentChainRunner to track tool calls and results (#1639) - Introduced a new property to store LLM-formatted messages for memory management. - Reset LLM messages at the start of each run to ensure fresh context. - Updated response handling to include LLM-formatted outputs, enhancing memory context for future iterations. - Modified the base class to accept an optional LLM-formatted output parameter for improved context saving. - Enhanced the writeToFile tool's response message for clarity on user actions. * Quick Command (cmd+k) (#1640) * Quick Command (cmd+k) * Update custom command modal style * Improve prompt * Use submenu for commands * Fix cursor comment * Align selected_text variable name * Fix embedded image wikilinks (#1641) - Updated the `extractEmbeddedImages` method to accept an optional `sourcePath` parameter for resolving wikilinks. - Implemented logic to determine the source path from context notes or fallback to the active file. - Enhanced logging for unresolved images to improve debugging and user feedback. - Ensured that embedded images are processed correctly based on the new source path handling. * Fix salient term language issue * Dedupe source notes (#1642) * Fix salient term language issue * Refactor source handling to include path in addition to title and score - Updated source structures across multiple classes to include a `path` property alongside `title` and `score`. - Modified methods in `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to accommodate the new source format. - Enhanced the `deduplicateSources` function to use `path` as the unique key, falling back to `title` if necessary. - Adjusted the `SourcesModal` to display paths correctly and ensure links open using the appropriate source path. - Updated type definitions in `message.ts` to reflect the new source structure. * Update path handling in source mapping for AutonomousAgentChainRunner - Modified the source mapping logic to ensure the `path` property defaults to an empty string if not provided, enhancing robustness in source data handling. - This change improves the consistency of source objects by ensuring that all properties are defined, which aids in downstream processing. * Bump manifest version to v3 * Fix image (#1643) * Fix salient term language issue * Bump manifest version to v3 * Fix image passing - Updated Chat, ChatManager, MessageRepository, and ChatUIState to include an optional `content` parameter in message handling. - Adjusted method signatures and message creation logic to accommodate the new parameter, improving flexibility in message processing. * Fix regenerate (#1644) * Fix image in agent chain (#1646) * Add YouTube transcription processing to AutonomousAgentChainRunner (#1647) - Implemented a new method to extract and process YouTube URLs from user messages. - Fetch transcriptions using the simpleYoutubeTranscriptionTool and handle errors gracefully. - Updated conversation messages to include fetched transcriptions for improved context in responses. * Disable quick command when live-mode is not on (#1648) * Do not call writeToFileTool again for accepted changes as well (#1652) * Trigger youtube tool only for urls in prompt not context (#1653) * Stop calling youtube tool in context notes in agent mode - Added a new instruction for handling YouTube URLs in the DEFAULT_SYSTEM_PROMPT. - Removed the direct usage of simpleYoutubeTranscriptionTool in the AutonomousAgentChainRunner, as YouTube transcriptions are now automatically processed. - Updated parameter naming conventions in modelAdapter for clarity and consistency. - Cleaned up the simpleYoutubeTranscriptionTool definition by removing unnecessary parameters. * Add agent loop limit message (#1655) * Add composer toggle and implement auto-preview (#1651) * Update composer output format to XML and refactor related components - Changed the output format for composer instructions from JSON to XML in `constants.ts`. - Updated `CopilotPlusChainRunner` to utilize the new XML format and adjusted streaming logic accordingly. - Added a toggle for the composer feature in `ChatInput.tsx` and updated UI elements to reflect this change. - Removed the `ComposerCodeBlock` component as it is no longer needed with the new implementation. - Cleaned up unused code and comments in `ChatSingleMessage.tsx` related to the previous composer handling. * Update src/constants.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/constants.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove redundant tests * Update doc * Refactor composer handling and implement ActionBlockStreamer - Replaced `ComposerBlockStreamer` with `ActionBlockStreamer` to handle `writeToFile` blocks more efficiently. - Updated `constants.ts` to reflect changes in output format for composer instructions. - Enhanced `ChatSingleMessage.tsx` to process `writeToFile` sections and integrate collapsible UI elements. - Removed the deprecated `ComposerBlockStreamer` and its associated tests. - Improved handling of XML codeblocks and unclosed tags during streaming. * Fix reported bugs * Refactor ActionBlockStreamer tests and implementation - Updated `processChunks` to always push content, including null and empty strings, to the output. - Adjusted test expectations to reflect changes in output handling for chunks with null content. - Simplified regex in `findCompleteBlock` method by removing XML format handling. * Fix reported issue --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix tool description generation (#1659) * Access tool description from zod.shape when zod.properties is missing. * Remove the access to schema.properties * Remove unused print * Also support schema.propertes * Implement SimpleTool and remove Langchain tool calling (#1662) * Make web search tool not overly eager * Use SimpleTool interface instead of langchain tool call - Replaced LangChain tool definitions with a new SimpleTool interface for better type safety and validation. - Updated existing tools (e.g., writeToFile, localSearch, webSearch, and YouTube transcription) to utilize the new createTool function. - Enhanced parameter extraction from Zod schemas for improved tool description generation. - Cleaned up related code and documentation to reflect the new tool structure and ensure consistency across the codebase. - Add tests * Update ChatManager tests to handle undefined context cases - Modified test cases in ChatManager to include handling of undefined context parameters. - Ensured that the tests accurately reflect scenarios where no active file exists and when context is undefined. - Improved overall test coverage for message processing in various contexts. * Add validation tests for tool schemas and improve schema definitions - Introduced comprehensive validation tests for tool schemas in `allTools.validation.test.ts` and `SearchTools.schema.test.ts` to ensure adherence to best practices and proper typing. - Enhanced `SearchTools.ts` by replacing `z.any()` with specific object schemas for `chatHistory` to improve type safety. - Added tests for various tool patterns, including metadata validation and schema best practices, ensuring tools are correctly defined and validated against expected interfaces. * Add back example content for canvas file in ComposerTools schema * Make web search tool not overly eager * Add license key in agent mode (#1663) * Add tool execution tests and implement Plus subscription checks (#1664) - Introduced unit tests for the `executeSequentialToolCall` function to validate tool execution behavior, including handling of Plus-only tools. - Enhanced `executeSequentialToolCall` to check for Plus subscription requirements before executing tools. - Updated `createTool` and `SimpleTool` interfaces to include an `isPlusOnly` flag, indicating tools that require a Copilot Plus subscription. - Marked existing tools (e.g., webSearch and YouTube transcription) as Plus-only where applicable. * Add timezone conversion tools and enhance current time functionality (#1665) * Add timezone conversion tools and enhance current time functionality - Introduced `convertTimeBetweenTimezonesTool` to allow conversion of time between different timezones. - Updated `getCurrentTimeTool` to accept an optional timezone parameter, improving its functionality to return time in specified timezones. - Enhanced documentation and added examples for both tools in the codebase. - Added comprehensive unit tests for the new timezone conversion functionality to ensure accuracy and reliability. * Enhance timezone functionality and add unit tests - Updated `getCurrentTime` and `convertToTimeInfo` functions to utilize Luxon's offset for accurate timezone calculations. - Adjusted timezone offset handling to reflect correct signs. - Added unit tests for Tokyo and New York timezone offsets to ensure accuracy in timezone calculations. * Add unit test for past time conversion in timezone tool - Introduced a new test case to verify that converting a past time (6:00 AM PT) to Tokyo correctly reflects the same day (January 15) without erroneously adding a day. - Updated the `convertTimeBetweenTimezones` function to ensure accurate handling of past times without unintended date shifts. * Enhance timezone handling and update related tests - Added `convertTimeBetweenTimezonesTool` to facilitate time conversion between different UTC offsets. - Updated `getCurrentTimeTool` to accept timezone offsets instead of names, improving accuracy and flexibility. - Enhanced error handling for invalid timezone offset formats and added comprehensive unit tests to validate new functionality. - Updated existing tests to reflect changes in timezone offset handling and ensure consistent behavior across various scenarios. * Refactor timezone handling in convertTimeBetweenTimezones function - Updated the creation of DateTime objects to interpret parsed dates as already being in the source timezone, improving clarity and accuracy in timezone conversions. * Fix image in chat history (#1666) * Refactor chat history handling to preserve multimodal content - Removed the deprecated `extractChatHistory` function and replaced it with direct access to raw history from memory, allowing for the preservation of multimodal content. - Updated `AutonomousAgentChainRunner`, `BaseChainRunner`, and `CopilotPlusChainRunner` to utilize `BaseMessage` objects for chat history, ensuring that both user and AI messages maintain their original structure and content. - Enhanced message processing to accommodate multimodal inputs, storing them appropriately for later use. * Refactor chat history handling to improve message processing - Introduced `addChatHistoryToMessages` utility to streamline the addition of chat history, ensuring safe handling of various message formats. - Updated `AutonomousAgentChainRunner` and `CopilotPlusChainRunner` to utilize the new utility, enhancing clarity and maintainability of the code. - Removed deprecated manual processing of chat history to improve code efficiency and readability. * Refactor chat history processing to enhance multimodal support - Removed the deprecated `extractChatHistory` function and replaced it with `processRawChatHistory` to ensure consistent handling of chat history. - Introduced `processedMessagesToTextOnly` function to extract text-only content from multimodal messages, improving clarity for question condensing. - Updated `CopilotPlusChainRunner` to utilize the new processing functions, ensuring that both LLM and question condensing use the same data format. - Added comprehensive unit tests for `processedMessagesToTextOnly` to validate its functionality across various message formats. * Refactor memory management in BaseChainRunner to enhance context saving - Removed the creation of `HumanMessage` and `AIMessage` objects for chat history, simplifying the process. - Implemented `saveContext` for atomic operations, ensuring proper memory management while preserving input and output data. - Noted that LangChain's memory now expects text content, which limits the saving of multimodal content. * Remove multimodal content storage from userMessage in AutonomousAgentChainRunner and CopilotPlusChainRunner to streamline message processing. This change aligns with recent refactors to enhance memory management and improve clarity in handling chat history. * Enhance ChatInput component with autonomous agent toggle button (#1669) - Added a toggle for the autonomous agent feature, allowing users to enable or disable it within the ChatInput component. - Updated the state management to synchronize the autonomous agent toggle with user settings. - Introduced a button for toggling the autonomous agent mode, which is only visible in Copilot Plus mode. - Refactored tool call logic to accommodate the new autonomous agent toggle, ensuring proper handling of tool calls based on the toggle state. * Implement custom topic in conversation history (#1670) * Refactor LoadChatHistoryModal and ChatPersistenceManager for enhanced topic handling - Updated LoadChatHistoryModal to prioritize custom topics from file frontmatter, falling back to filename extraction if not present. - Enhanced ChatPersistenceManager to generate AI topics for new chat files and preserve existing topics when updating files. - Introduced new methods for finding files by epoch and generating AI topics based on conversation content, improving chat file management and organization. * Refactor LoadChatHistoryModal and ChatPersistenceManager for improved topic handling - Updated LoadChatHistoryModal to ensure custom topics are trimmed and validated before use. - Enhanced ChatPersistenceManager to utilize reduce for efficient conversation summary generation, introducing constants for message and character limits to improve readability and maintainability. * Implement tool call UI banner (#1671) * Only add tool call to the user message when agent is off - Updated the tool call logic to only add tool calls when the autonomous agent toggle is off, ensuring that the autonomous agent manages all tools internally when enabled. - Improved code clarity by adding comments to explain the new logic. * Implement tool call collapsible UI element - Introduced a new utility for managing tool call markers, allowing for structured display and tracking of tool calls during execution. - Updated the AutonomousAgentChainRunner to utilize the new tool call markers, improving clarity in the streaming response and tool execution process. - Added a ToolCallBanner component for better visualization of tool calls in the chat interface, including execution status and results. - Refactored ChatSingleMessage to handle tool call updates dynamically, ensuring that the UI reflects the current state of tool calls. - Enhanced CSS for tool call containers and added animations for improved user experience. * Fix disappearing tool call in final response * Implement tool call result formatting - Updated ChatSingleMessage to use a stable ID for message roots, enhancing memory management and preventing leaks. - Implemented cleanup logic for old message roots to optimize performance. - Introduced ToolResultFormatter to format tool results for better display in the UI, ensuring user-friendly output for various tool types. - Enhanced ToolCallBanner to utilize the new formatter, improving the presentation of tool call results in the chat interface. * Add YouTube transcription tool in agent and avoid mistrigger from context - Replaced the deprecated simpleYoutubeTranscriptionTool with a new youtubeTranscriptionTool that automatically extracts YouTube URLs from user messages. - Removed the processYouTubeUrls method to streamline the handling of YouTube transcriptions. - Updated tool execution logic to pass the original user message to tools that require it, enhancing flexibility in tool usage. - Improved the ToolResultFormatter to support multi-URL responses and provide better error handling for transcription failures. - Enhanced documentation for YouTube transcription usage in the codebase. * Enhance ToolCallBanner and ToolResultFormatter for improved user experience - Added animation constants to ToolCallBanner for better visual feedback. - Updated ToolResultFormatter to robustly handle JSON parsing, improving error handling for both single objects and arrays. - Implemented input validation in YouTube transcription tool to prevent excessive input lengths and ensure proper data types. * Refactor tool execution messages and enhance JSON parsing in ToolResultFormatter - Updated the confirmation message for the writeToFile tool to be more concise. - Added a private method in ToolResultFormatter to robustly parse JSON strings, improving error handling and ensuring consistent results. - Streamlined the handling of search results by separating JSON parsing and regex extraction logic in ToolResultFormatter. - Enforced required user message content in the YouTube transcription tool to prevent potential errors. * Truncate long tool results for compact memory (#1672) * Truncate long tool results for compact memory - Introduced `processToolResults` utility to format tool results for different contexts, allowing for both truncated and full results. - Updated `AutonomousAgentChainRunner` to utilize the new utility for improved memory management and clarity in tool result presentation. - Added unit tests for `toolResultUtils` to ensure proper functionality of truncation and formatting methods, enhancing reliability and maintainability of tool result processing. * Enhance tool results handling in AutonomousAgentChainRunner - Added a conditional check to ensure that only non-null tool results are pushed to the llmFormattedMessages array, improving the robustness of message formatting and preventing potential errors from undefined values. * Add Agent settings section (#1676) * Add autonomous agent settings and tool configurations - Introduced settings for enabling the autonomous agent mode, including maximum iterations and available tools. - Updated the CopilotPlusSettings component to allow users to toggle tools like local search, web search, and YouTube transcription. - Enhanced the model adapter to utilize the new settings for tool management. * Enhance tool management by adding always available tools and updating tool usage guidelines in autonomous agent mode * Make filetree tool always available * Refactor tool management system to use a centralized ToolRegistry, enabling dynamic tool registration and configuration. Update settings structure to support enabled tool IDs and enhance UI for tool selection in the Copilot settings. * Initialize built-in tools with vault access and improve tool registration logic * Enhance documentation for initializeBuiltinTools function to clarify tool registration process and dynamic filtering of user-enabled tools. * fix: Fix the shouldIndexFile method to return all files when inclusions is empty in project mode. (#1667) * feat: Display all category items by default in the context-manage-modal. (#1657) * Remove escapingXML for context data (#1678) * Add apply command (#1677) * Implement ReplaceInFile tool (#1661) * Relevant note improvements (#1681) * Change message scrolling behavior (#1680) * Fix timetools and tests (#1683) * Refactor time tool descriptions for clarity and examples * Update XML escaping tests to reflect correct behavior and remove unnecessary settings mock * Strip partial tool call tag (#1682) * Strip partial tool call tag * Show a placeholder message * Remove perserveToolIndicators option * Add tool dynamic prompt (#1679) * feat: Enhance tool execution error messages and update custom prompt instructions for built-in tools * feat: Add unit tests for ModelAdapter and enhance tool instructions for time-based queries * feat: Implement detailed formatting for replaceInFile tool results * feat: Enhance GPT model instructions and refine replaceInFile tool usage guidelines * feat: Expand documentation on schema descriptions and custom instructions for tool usage * feat: Refine tool documentation by clarifying schema descriptions and custom prompt instructions * feat: Update GitHub Actions workflow to run on all pull requests regardless of target branch * Fix tests * feat: Update integration tests to validate writeToFile blocks and enhance replaceInFile usage example * Enhance image extraction logic to support both wiki-style and markdown image syntax (#1685) * Turn off agent when in Projects mode (#1686) * Remove think and action blocks from copy and insert (#1687) * Remove think and action blocks from copy, and fix composer for thinking models * Refactor ToolSettingsSection layout for improved readability and consistency --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: wenzhengjiang <jwzh.hi@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Zero Liu <zero@lumos.com> Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com> |
||
|
Logan Yang
|
b79b8d0a38
|
Migrate chat messages (#1629)
* Migrate from sharedState to new chat message system - Replaced SharedState with a new ChatManager and ChatUIState architecture to streamline message processing and UI integration. - Updated message imports to utilize the new types from "@/types/message". - Introduced unique message IDs for better tracking and management. - Enhanced the Chat component to support pending messages and improved message deletion logic. - Removed deprecated shared state references and cleaned up related code for clarity and maintainability. Enhance message management architecture and context processing - Introduced a new MessageRepository for single source of truth in message storage, eliminating dual-array synchronization. - Updated ChatManager to coordinate message operations and context processing, ensuring fresh context during edits. - Refactored ChatUIState for clean UI state management, delegating business logic to ChatManager. - Added comprehensive testing for message handling and context processing to prevent bugs and ensure reliability. - Removed legacy SharedState references and improved overall architecture for clarity and maintainability. Update claude.md Remove conditional project tag from chat content rendering in Chat component Ensure project chat isolation - Introduced a method to retrieve the current project ID in ProjectManager. - Enhanced ChatManager to manage multiple project-specific message repositories, allowing for dynamic switching based on the current project. - Implemented logic to load existing messages from the ProjectManager's cache when creating new message repositories. - Updated message handling methods in ChatManager to utilize the current project's message repository, ensuring context consistency across project switches. * Fix message isolation between projects and non-project - Introduced ChatManager and ChatUIState to streamline message processing and UI updates. - Removed legacy message caching from ProjectManager, delegating message persistence to ChatManager. - Updated CopilotView and Chat components to utilize the new architecture, enhancing message loading and project switching. - Cleaned up deprecated code related to pending messages and shared state for improved maintainability. * Implement a new chat persistence module - Introduced ChatPersistenceManager to handle saving and loading chat history in markdown format, ensuring project-aware file naming. - Updated ChatManager to utilize the new persistence functionality, streamlining message loading and saving processes. - Refactored ChatUIState to support asynchronous loading and saving of messages, improving UI responsiveness. - Added comprehensive tests for ChatPersistenceManager to ensure reliability in chat content formatting and parsing. - Updated architecture documentation to reflect the new message management structure and flow. * Update message management architecture doc - Removed detailed architecture diagrams from CLAUDE.md and linked to MESSAGE_ARCHITECTURE.md for clarity. - Updated MESSAGE_ARCHITECTURE.md to include project isolation features, enhancing the message management system. - Improved documentation on message flow and project-specific repositories, ensuring seamless context switching. - Streamlined the explanation of core classes and their interactions within the new architecture. * Fix test * Use llm message for commands to ensure context access * Update docs |