mirror of
https://github.com/logancyang/obsidian-copilot.git
synced 2026-07-22 07:50:24 +00:00
21 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Logan Yang
|
d3ee3f999b
|
fix: agent loop improvements, clickable citations, and UI fixes (#2287)
* fix: prevent agent ReAct loop from getting stuck with small local models Small local models (e.g. qwen3.5-4b-mlx) get stuck calling localSearch repeatedly with near-identical queries and never synthesizing an answer. - Remove QueryExpander from agent loop to avoid model contention on single-connection local models (retriever still expands internally) - Add query deduplication that pre-filters tool call batches using Jaccard word-overlap similarity before execution - Force synthesis via raw model (without tools) after 2 consecutive iterations where all tool calls are duplicates - Add agent-specific system prompt guidance for search limits - Strip leaked chat template role tokens from intermediate content - Add intermediate model output logging for debugging Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: make inline citation numbers clickable and restore Max Sources setting Inline citation numbers like [1], [2] in chat responses are now clickable links that open the corresponding source note. Also restores the Max Sources slider in QA settings, reverting the hardcoded DEFAULT_MAX_SOURCE_CHUNKS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove excessive 32px bottom padding from chat view on desktop The padding-bottom on .view-content used max(safe-area + 4px, 32px), forcing a 32px minimum even on desktop where safe-area-inset-bottom is 0. Simplified to safe-area + 4px so desktop gets minimal padding while devices with safe areas still get proper spacing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(agent): track dedup queries post-execution to keep failed searches retryable Move previousSearchQueries tracking from before to after each localSearch tool call executes. Previously, queries were marked as seen before any tool ran, so if a call was aborted or the loop threw early, those queries were still blocked from retrying on the next iteration. Now each query is added to previousSearchQueries immediately after its tool call returns, keeping the dedup effective for completed calls while leaving transient failures retryable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(citations): preserve Obsidian internal-link attributes on inline citation anchors When building clickable inline citation links (e.g. [1], [2]), the previous code only copied the href from the source section anchor. Obsidian vault-note links have additional attributes such as data-href and class="internal-link" that are required for in-app note navigation. Without them, clicking an inline citation resolves the link as a plain relative URL instead of opening the corresponding note. Fix: store the source anchor element rather than just the href, then copy all attributes onto the generated citation anchor before setting the citation-specific class and aria-label overrides. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(agent): only mark localSearch queries as seen after successful execution Change the dedup tracking condition from always tracking queries post-execution to only tracking when the tool call succeeds (result.success is true). This ensures transient failures (e.g. temporary index unavailability) leave the query retryable: if a localSearch fails, the model can issue the same query again on the next iteration. Previously, even a failed execution would mark the query as seen, blocking recovery. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: deduplicate inline citations after source consolidation When multiple chunks come from the same note, the LLM cites them separately. After consolidation maps them to the same source number, adjacent duplicates like [1][1] or [1] and [1] now collapse to [1]. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update GPT-5.2 to GPT-5.4 in builtin models Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add dynamic status bar clearance for desktop chat view The chat input was hidden behind Obsidian's status bar on desktop. Adds ChatViewLayout which measures the geometric overlap between the view-content and the status bar, setting a CSS variable to provide the exact padding needed. Adapts to any theme: no clearance for auto-hide themes (opacity: 0) or themes that already account for the status bar in their layout; correct clearance for themes where the bar is visible and overlapping. Re-checks on theme switches via debounced css-change listener. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: suppress thinking tokens during streaming when reasoning is disabled When a local model (e.g. Qwen 3.5 via LM Studio) emits text-level <think> tags but the user has not enabled the reasoning capability, the thinking content was visibly rendered during streaming and only stripped once the </think> tag arrived. Now tracks the position of excluded think blocks across chunks and truncates fullResponse to the safe prefix on each chunk, so thinking content never appears. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
d7d54d2a34
|
docs: add user-facing documentation (#2254)
* docs: add user-facing documentation (closes #2253) - 13 docs covering all major features: getting started, chat interface, LLM providers, models/parameters, context/mentions, custom commands, vault search/indexing, agent mode/tools, projects, system prompts, Copilot Plus/self-host, troubleshooting/FAQ, and index - Written for non-technical Obsidian users - Each doc is standalone with cross-references to related docs - All values verified against source code (defaults, model names, etc.) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: improve user docs with website reference material - getting-started: add glossary (LLM, API, Token, Context Window, Embeddings, RAG, Vector Store) and API key billing note - chat-interface: add user/AI message buttons (edit, copy, delete, insert/replace at cursor, regenerate), [[Note Title]] inline reference syntax, Relevant Notes feature, manual Save Chat button, and auto-compact user experience note - context-and-mentions: add PDF context (+ Add context button) and image context (drag/drop or image button) methods - custom-commands: fix placeholder syntax ({} not {selected text}), add {FolderPath} variable, note tags must be in note properties, add richer example prompts - llm-providers: add LM Studio CORS requirement, 3rd-party CORS warning - vault-search-and-indexing: add cost estimation tip (Count total tokens command), RangeError/partitioning note, tag property note - agent-mode-and-tools: add Revert option, note @composer works in both Plus and Projects modes - projects: add 50+ file type support detail - copilot-plus-and-self-host: add dashboard URL - troubleshooting-and-faq: add first-steps section, RangeError fix, response cut-off fix, notes-not-found checklist, note referencing FAQ, English response FAQ, image/PDF FAQ, privacy note Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: remove UI_RENDERING_PERFORMANCE.md from docs (already in designdocs) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add doc maintenance rule to CLAUDE.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: fix custom command variable syntax and project deletion claim Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: clarify 16 built-in providers + unlimited OpenAI-compatible models Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: fix semantic search default and partitioning claims Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: fix self-host status and remaining partitioning claim Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: fix self-host setup steps, tool count, and memory tool availability Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
fff6b4efd0
|
chore: rename docs to designdocs and nest todo folder (#2252)
- Renamed docs/ to designdocs/ - Moved draft/todo docs into designdocs/todo/ subfolder - Added OBSIDIAN_CLI_INTEGRATION.md from master's todo/ folder - Updated all docs/ path references in CLAUDE.md, AGENTS.md, and designdocs Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
97763aaf93
|
feat(chat-history): add infinite scroll pagination to ChatHistoryPopover (#2251)
* feat(chat-history): add infinite scroll pagination to ChatHistoryPopover Progressively loads chat history in batches of 50 via IntersectionObserver on a sentinel div, preventing UI lag for users with 1000+ conversations. - Observer is created once on mount with empty deps; reads live pagination state via paginationStateRef — no disconnect/reconnect as pages load or search filters change - Reset only fires when popover opens or search changes while open (not close) - Status indicator is plain text (data is local/synchronous, not async) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(chat-history): use useLayoutEffect to prevent pagination reset render spike Replace useEffect with useLayoutEffect for the displayCount reset so it runs synchronously before the browser paints, eliminating the one-frame render spike that occurred when reopening the popover with a large displayCount from a previous scroll session. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add UI rendering performance audit report Comprehensive audit of React rendering inefficiencies across the chat UI, state management, and streaming paths. 14 findings ranked by severity (Critical/High/Medium/Low) with recommended fix priorities. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
0b6bb5a7bf
|
fix: support hidden directory chat save and history (#2188) (#2204)
When defaultSaveFolder is a hidden directory (e.g. .copilot/conversations), Obsidian's metadata cache doesn't index it, causing saves to fail and chat history to show empty. Add adapter-level fallback utilities in src/utils/vaultAdapterUtils.ts: - resolveFileByPath: vault lookup with synthetic TFile fallback - listMarkdownFiles: folder listing with adapter fallback - patchFrontmatter: processFrontMatter with adapter YAML patching fallback - readFrontmatterViaAdapter: parse frontmatter from raw file content Apply fallbacks across ChatPersistenceManager (save, load, list, find) and main.ts (history picker, load, delete, title update, lastAccessedAt). Also removes obsolete design docs from docs/. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Logan Yang
|
5ea26b1b7b
|
Add acp design doc (#2195) | ||
|
Logan Yang
|
795ed5ff6f
|
fix: hard cap L1 project context + payload diagnostics (#2192)
* docs: add token budget enforcement analysis and fix plan Document root cause of context window overflow (2.7M tokens sent to 1M model): L4 chat history bypasses all compaction systems. Add fix plan with phased approach to enforce token budget at message assembly point. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: correct root cause analysis — L1 unbudgeted, not L4 Previous analysis incorrectly blamed L4 chat history as the primary culprit. Investigation shows L4 stores only bare L5 text + compacted responses. The real issue is systemic: no total payload enforcement, with L1 (project context) being the largest unbudgeted layer and PROJECT_COMPACT_THRESHOLD being blind to L1 size. Updated fix plan to be model-agnostic (use autoCompactThreshold as single budget, no model-specific lookup tables), added contextTurns deprecation, and history guarantee principle. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: hard cap L1 project context at 600k tokens + payload diagnostics - Truncate project context at 2.4M chars (~600k tokens) to prevent total payload from exceeding model context windows (temporary fix until full token budget enforcement is implemented) - Add per-layer token estimate logging when payload exceeds 2M chars to help diagnose context window overflow reports - Document Obsidian CLI dev tools in CLAUDE.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Wenzheng Jiang
|
dc729fdba4
|
Miyo integration: backend + retriever (#2156)
* feat: integrate miyo search backend * fix: avoid node builtins in miyo discovery * chore: omit model name for miyo embeddings * Revert "chore: omit model name for miyo embeddings" This reverts commit 9d1622b7d8bd8243b5b2e301e356a42fd0c934de. * Add Miyo debug logging for search and upsert * Include vault path hash in Miyo source_id * Lock Miyo embedding model in settings UI * Update Miyo collection naming and search payloads * Implement Miyo garbage collection * Add Miyo time-range filters * Remove Miyo embeddings and move Miyo search toggle * Revert "Remove Miyo embeddings and move Miyo search toggle" This reverts commit b0b787103561459dbd9128f6a96dad3d2914e6c9. * refactor miyo indexing integration * fix: resolve build type import and remove Miyo API requirement doc * Add Miyo related-notes endpoint support for Relevant Notes * Refine Miyo search toggle availability and indexing flow * Update Miyo API fields and streamline relevant-notes Miyo flow |
||
|
Logan Yang
|
9535bf5a74
|
Replace indexing Notice with in-chat progress card & fix phantom re-indexing (#2173)
* Replace indexing Notice with in-chat progress card & fix phantom re-indexing Replace the Obsidian Notice popup for indexing progress with a persistent in-chat progress card using the same Jotai atom pattern as project mode. Key changes: - New IndexingProgressCard component with progress bar, pause/resume/stop, error display, and auto-close on completion - Jotai atom (indexingProgressAtom) driven by IndexOperations, read by React - Card only appears when files actually need indexing (no flash on mode switch) - User-initiated actions (refresh/reindex buttons) show "Index Up to Date" feedback with green check icon - Fix phantom re-indexing: checkIndexIntegrity() is now diagnostic-only - Fix progress bar accuracy: totalFiles counted after chunk preparation - Remove duplicate completion Notices from ChatControls and RelevantNotes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix insert/replace at cursor including agent reasoning blocks (#2174) Strip <!--AGENT_REASONING:...--> markers in cleanMessageForCopy() so that "Insert / Replace at cursor" only inserts the actual AI response, not the internal agent reasoning metadata. Uses greedy .* (single-line) so the regex matches to the real closing --> even if the JSON payload contains that sequence. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Restore integrity check marking files for re-indexing checkIndexIntegrity() was made diagnostic-only to fix phantom re-indexing, but this left files with missing embeddings permanently skipped. The real safeguard is clearFilesMissingEmbeddings() at the start of each indexing run, which prevents the repeat cycle. Restore the mark so the next indexing trigger picks up integrity failures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Wenzheng Jiang
|
bee0a66958
|
Miyo Integration Phase 1: abstract semantic index backend (#2155)
* refactor: abstract semantic index backend * refactor: abstract get documents by path |
||
|
Logan Yang
|
8cea1a6f17
|
Audit context envelope, tag alignment, artifact dedup, and logging (#2164)
* fix: align context block tags and make parseContextIntoSegments registry-driven
The context envelope L2 layer was silently dropping URL/media context across
turns due to three compounding issues:
1. Tag mismatch: Mention.ts created <youtube_transcript> blocks but the
contextBlockRegistry expected <youtube_video_context>. Fixed the tag and
inner element (<transcript> → <content>) to match the registry.
2. Missing registration: twitter_content blocks had no registry entry, so
they were invisible to compaction and segment parsing. Added the entry.
3. Hardcoded parser: parseContextIntoSegments only matched <note_context>,
<active_note>, and <prior_context> via hardcoded regexes, silently
ignoring all URL/media block types. Rewrote it to dynamically build its
regex from CONTEXT_BLOCK_TYPES, ensuring any future block type added to
the registry is automatically handled.
Extracted parseContextIntoSegments into src/context/parseContextSegments.ts
as a standalone pure function for direct testability.
Added 19 new tests covering all registered block types, mixed blocks,
stable/unstable flags, and a registry completeness guard test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: context envelope improvements - regeneration, dedup, per-artifact IDs, XML escaping
Phase A: Fix regeneration from loaded chats by lazily reprocessing context
when contextEnvelope is missing (e.g., messages loaded from disk).
Phase B: Replace static segment IDs ("urls", "selected_text", "web_tabs")
with per-artifact IDs from parseContextIntoSegments, enabling accurate
smart referencing in LayerToMessagesConverter.
Phase C: Deduplicate L2 content by segment ID (last-write-wins) to prevent
linear growth when the same artifact appears across multiple turns.
Phase D: Escape note title/basename in XML context blocks for consistency
with URL/YouTube content. Keep path unescaped as it's used as identifier.
Additional fixes from Codex review:
- Unique IDs for blocks without source extractors (selected_text counter)
- Keep note.path raw to avoid dedup mismatch with TFile.path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent context artifact duplication in LLM payload and L4 memory
CopilotPlusChainRunner was re-injecting processedText (which includes
context XML artifacts) into the user message via ensureUserQueryLabel,
bypassing the envelope's clean L2/L3/L5 separation. This caused context
like YouTube transcripts to appear 3x in the LLM payload.
Fix: use L5_USER envelope text (expanded user query without context XML)
instead of processedText for cleanedUserMessage, trimmedQuestion fallback,
and messageForAnalysis. Also fix BaseChainRunner.handleResponse to save
L5 text to L4 memory instead of processedText.
Other changes:
- Remove dead updateMemoryWithLoadedMessages from chainManager
- Update CONTEXT_ENGINEERING.md with example walkthrough, L4 behavior
docs, and Phase 6 integration test suite roadmap
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add chain runner envelope usage section and fix Step 0 log
Document per-runner envelope behavior, CopilotPlus/Agent tool flows,
and token efficiency audit in CONTEXT_ENGINEERING.md. Fix Step 0 log
to show L5 user query instead of processedText with context XML.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: trim trailing whitespace from chat input display text
Trailing newlines from Lexical editor were preserved in displayText
and rendered as empty lines in chat bubbles due to whitespace-pre-wrap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Logan Yang
|
b128b25558
|
Agent UIUX Improvements (#2149)
* Use openrouter openai embedding small as default embedding model * Increase agent max iterations to 16 and add 5-minute loop timeout - Raise max tool call iterations limit from 8 to 16 - Add AGENT_LOOP_TIMEOUT_MS constant (5 minutes) to prevent runaway loops - Update slider max in settings UI to match new limit - Show appropriate message when loop exits due to timeout vs max iterations Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Update agent reasoning spinner to sigma shape with smooth animation - Change from 9-dot diamond to 7-dot sigma (Σ) pattern - Animation traces sigma shape: top-right → top-left → center → bottom - Use ease-in-out timing and gradual opacity transitions for smooth motion - Adjust size to 13.5px for better alignment with text Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Improve agent reasoning step summaries and fix composer tool UX - Add meaningful summaries for writeToFile/replaceInFile tools (e.g., "Writing to filename" instead of "Calling writeToFile") - Show "Edit rejected" when user rejects changes in diff view - Fix "Changes applied successfully" notice showing on reject - Extract model's intermediate reasoning as finding summaries - Add TODO doc for composer tool redesign to address 30s feedback gap Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add self-host mode eligibility gating and permanent validation - Increase tool timeout from 60s to 120s - Show self-host mode section only to Believer/Supporter plan users - Add permanent validation after 3 consecutive successful checks - Add useIsSelfHostEligible hook for offline-friendly eligibility check - Preserve permanent validation status when re-enabling self-host mode Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix self-host validation to require 14-day intervals between count increments - Only increment validation count when 14+ days have passed since last check - Document need for structured ComposerTools results in TODO doc - Add TODO for no-op case handling in AgentReasoningState Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix self-host validation timer and offline re-enable issues - Don't reset timestamp when < 14 days (preserves interval countdown) - Allow permanently validated users to see section even if toggle is off - Supports offline re-enable for users with 3+ successful validations Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Wire refreshSelfHostModeValidation into plugin startup Enables the 14-day periodic validation to actually increment the count toward permanent self-host mode (3 successful checks). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Improve self-host mode validation with offline support and documentation - Add grace period check in validateSelfHostMode for offline re-enable - Change grace period from 14 to 15 days - Add comprehensive documentation explaining validation flow - Fix offline re-enable for users within grace period Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Logan Yang
|
1e3e0786ef
|
Migrate to native tool call in plus and agent (#2123)
* Implement self-host mode * Migrate from XML tool calls to native tool calls and reimplement agent mode * Use ChatOpenRouter for copilot-plus-flash for native tool call sse support * Fix QA exclusion for search v3 * Update plan and remove debug messages in agent * Migrate chains off of xml and clean up related logic * Implement agent reasoning block * Update css for responsiveness * Implement agent query pre-expansion and proper reasoning block display for search * Update agent docs * Refine agent reason block * Refine agent reasoning UX * Fallback to plus non-agent if native tool call is not supported by the model * Fix time filter query * Update self-host description * Skip agent reasoning block and old tool call banners during chat save and load * Fix projects mode switch to plus and chat auto saves as non-project bug |
||
|
Logan Yang
|
1d668d3367
|
Deprecate IntentAnalyzer (#2069)
* Deprecate IntentAnalyzer and replace with model-based tool planning in CopilotPlusChainRunner * Refine planning prompt to enforce XML-only responses and remove note extraction logic |
||
|
Logan Yang
|
1cd87d75f3
|
Fix guidance (#1997)
* refactor: Simplify local search guidance handling in AutonomousAgentChainRunner and CopilotPlusChainRunner - Removed the `appendInlineCitationReminder` function and adjusted the logic in `AutonomousAgentChainRunner` to directly manage local search guidance. - Updated `CopilotPlusChainRunner` to retain and utilize the most recent local search guidance, ensuring it is positioned correctly adjacent to user queries. - Modified `buildLocalSearchInnerContent` to accept introductory text instead of guidance, enhancing clarity in the payload structure. - Removed related tests for the deprecated `appendInlineCitationReminder` function, streamlining the codebase. * refactor: Enhance localSearch payload structure and guidance handling - Made localSearch payload self-contained by embedding RAG instructions, documents, and citation guidance directly within the `<localSearch>` block. - Updated AutonomousAgentChainRunner and CopilotPlusChainRunner to ensure each localSearch call carries its own guidance block, improving citation accuracy in multi-search scenarios. - Introduced a new utility function to insert guidance before the user query label, ensuring clarity in the payload structure. - Adjusted related tests to validate the new guidance handling logic. * docs: Add deprecation guide for IntentAnalyzer and Broca - Created a comprehensive document outlining the responsibilities of the `IntentAnalyzer` and Broca API within the Copilot Plus intent analysis flow. - Detailed the current functionalities, known consumers, migration constraints, and a phased migration plan to ensure feature parity and smooth transition. - Highlighted risks and mitigation strategies associated with the deprecation process, along with open questions for further consideration. * fix: Prevent infinite rolling animation in ToolCallBanner when result is present Add defensive check that stops animation when tool result exists, even if isExecuting flag wasn't updated. This fixes race conditions where marker updates fail or are delayed during streaming. 🤖 Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Happy <yesreply@happy.engineering> |
||
|
Logan Yang
|
c4027fa7a4
|
Update Corpus-in-Context and web search tool guide (#1988)
* Make inline citation permanent and add image guide for local search - Removed the `enableInlineCitations` setting from the configuration and related components to simplify citation management. - Updated citation functions to operate without the setting, ensuring consistent behavior across the codebase. - Adjusted tests and documentation to reflect the removal of inline citation handling logic. - Enhanced the `renderCiCMessage` and `processInlineCitations` functions for improved clarity and functionality. * fix: Update web search tool description for clarity - Revised the description of the `webSearch` tool to specify that it searches the internet for information only when explicitly requested by the user. - Enhanced the instructions for using the `webSearch` tool to clarify when it should be utilized, ensuring users understand the intent required for web searches. - Reintroduced the `createGetTagListTool` import to maintain functionality in the `builtinTools` module. * fix: Enhance response handling in AgentPrompt integration tests - Updated the response processing logic in the AgentPrompt integration tests to ensure that `fullResponse` is always a string, accommodating multimodal and array content. - Implemented a function to extract text parts from arrays and handle various content types, improving the robustness of the tests. * Remove websearch related prompt from modelAdapter * fix: Update time query instructions in builtinTools - Added detailed instructions for handling time queries, including the conversion of city or timezone names to UTC offsets. - Clarified when to use the timezoneOffset parameter and emphasized the need for user clarification if the offset cannot be determined. |
||
|
Logan Yang
|
ed884298ba
|
Integrate ProjectChainRunner and ChatManager with new layered context (#1973)
* Integrate ProjectChainRunner and ChatManager with new layered context - Introduced `ProjectChainRunner` to automatically include project context in L1 via `ChatManager.getSystemPromptForMessage()`, simplifying the class structure by inheriting behavior from `CopilotPlusChainRunner`. - Updated `ChatManager` to build system prompts that append project context when in project mode, ensuring seamless integration without special-case logic. - Implemented null guards for project context to prevent erroneous output when context is unavailable. - Refactored documentation to reflect changes in context handling and the new project chain runner functionality. * Enhance ChatManager tests with project context mocks - Added mocks for project context retrieval and system prompt settings in `ChatManager.test.ts` to facilitate testing without relying on actual project data. - Improved test coverage for context-related functionalities by simulating project context and system prompt retrieval, ensuring robust testing of the ChatManager's behavior in various scenarios. |
||
|
Logan Yang
|
05beb941eb
|
Context revamp (#1971)
* Implement context engine for a layered context * Wire in new context library to LLM chain - Updated the context engine to maintain a cumulative library of all context items (L2) without deduplication, enhancing cache stability. - Introduced smart referencing in the user message (L3) to reference existing items in L2 by ID and include full content for new items. - Improved the LayerToMessagesConverter to format and handle layered context envelopes for better debugging and clarity. - Fixed critical bugs related to context deduplication that previously broke cache stability. - Enhanced tests to validate the new context handling and ensure proper functionality across various scenarios. * Enhance VaultQAChainRunner with tag extraction and inline citation detection - Integrated tag extraction from user queries to improve context-aware retrieval. - Implemented `hasInlineCitations` function to detect inline citation markers in responses. - Updated message construction logic to utilize envelope-based context, enhancing the quality of AI interactions. - Added comprehensive tests for the new inline citation detection functionality, ensuring robustness and accuracy. - Improved logging for better debugging and traceability of the message processing flow. Refactor VaultQAChainRunner to enhance context handling and citation instructions - Integrated LayerToMessagesConverter for improved message construction, ensuring smart referencing and context preservation. - Updated logging to reflect the use of the new converter in envelope-based context construction. - Revised comments for clarity on context preparation and citation instructions. - Minor adjustment in citation utility to improve regex handling for citation updates. * Complete migration to envelope-based context in CopilotPlus and LLM chain runners - Removed all legacy fallback paths, ensuring that context envelopes are now mandatory for operations in CopilotPlusChainRunner and LLMChainRunner. - Enhanced image extraction to only pull from the active note, preventing unintended leaks from attached context notes. - Fixed critical issues with context-in-context (CiC) formatting and added early envelope guards in VaultQAChainRunner to prevent silent failures. - Updated message construction logic to utilize LayerToMessagesConverter for improved smart referencing and context preservation. - Comprehensive tests added to validate the new envelope-based context handling and ensure robustness across all chain runners. * Integrate Plus chain with context envelope * Enhance AutonomousAgentChainRunner with envelope-based context integration - Introduced `LayerToMessagesConverter` for improved message construction, ensuring system and user messages are derived from the context envelope. - Implemented envelope validation to ensure context is available before processing. - Updated message preparation logic to maintain consistency with the envelope-first architecture, preserving tool execution and multimodal support. - Added comprehensive logging for envelope-based context construction to aid debugging. - Documented changes in the context engineering documentation to reflect the new integration and design points. * Support loading saved chat context to L2 context library * Fix ChatPersistenceManager tests for context handling |
||
|
Logan Yang
|
cea72d61b1
|
Support dataview result in active note (#1918) | ||
|
Logan Yang
|
5568b70e0f
|
Merge 3.1.0 preview (#1906)
* Building persistent memory for copilot (#1848) * Implement chat input v3 (#1794) * Fix chat crash React 409 (#1849) * Fix mobile typeahead menu size (#1853) * Enhance progress bar and bug fix (#1814) * feat: Add edit context for progress card. * fix: When adding a model, do not set the key in the single model setting. * fix: fix the width of the popover on mobile. * Refactor Brevilabs API integration to use models base URL (#1855) - Updated constants to switch from BREVILABS_API_BASE_URL to BREVILABS_MODELS_BASE_URL for model-related API calls. - Adjusted ChatModelManager and EmbeddingManager to utilize the new models base URL for configuration settings. * feat: Add new chat history popover. (#1850) * feat: Add new chat history popover. * feat: Add open source file button to chat history popover - Add ArrowUpRight button to each chat history item - Implement openChatSourceFile method in main.ts - Add onOpenSourceFile callback prop chain through components - Open chat files in new Obsidian tabs when clicked - Optimize error handling to prevent duplicate notices * feat: Implement localSearch CiC prompting flow in CopilotPlusChainRunner (#1856) * Properly extract response text in UserMemoryManager (#1857) * feat: Temporarily disable autocomplete features (#1858) * chore: Update version to 3.1.0-preview-250927 in manifest.json * Support space in typeahead trigger and improve search (#1859) * More chat input enhancement (#1864) * Fix dropdown color * Fix badge border in light theme * Increase search result number * support paste image * fix folder context * Enhance ChatPersistenceManager with filename sanitization tests (#1865) - Added tests to ensure proper sanitization of wiki link brackets and illegal characters in filenames when saving chat messages. - Updated filename generation logic to handle empty sanitized topics by defaulting to 'Untitled Chat'. - Refactored imports in ChatPersistenceManager for better organization. * Update ApplyView accept button styles for improved visibility and interaction (#1866) - Adjusted the positioning of the action button to be further from the bottom of the viewport for better accessibility. - Increased the z-index to ensure the button is always on top of other elements. - Added a shadow effect to enhance the button's visibility against the background. * Expose add selection to chat context to free users (#1867) * New chat input improvements (#1868) * Improve search logic in at mention search * Subscribe to file changes * Detect changes for folders and tags * Make context badge tooltip always show * Add full note path in preview * Fix enter being blocked when no search result * Rebuild "Active Note" and more chat input improvement (#1873) * Performance improvement and extend "active note" to note typeahead menu (#1875) * Add better agent prompt logging (#1882) * Add read tool (#1883) * Add readNote tool for reading notes in chunks * Enhance readNote tool to support dynamic note path display and linked note extraction * Refine readNote tool instructions for improved clarity and efficiency in note content retrieval * Add support for readNote tool: integrate emoji, format results, and refine instructions * Add inline citation reminder functionality to user questions * Add an alert when user hits the maxToken limit (#1884) * Refactor AutonomousAgentChainRunner (#1887) * Refactor AutonomousAgentChainRunner to enhance agent workflow, streamline context preparation, and improve response handling * Enhance type safety in addChatHistoryToMessages by specifying message structure * Improve max iteration limit message for clarity and formatting * Refactor tool display name handling for improved clarity and maintainability * Refactor tool call ID generation and visibility handling for improved clarity and uniqueness * Fix agent tool call ID (#1890) * Enhance tool call ID generation and improve readNote handling in prompts Update readNote description for clarity and improve prompt instructions * Update instructions for registerFileTreeTool to clarify usage guidelines * Clarify instructions for readNote tool to improve context inference and handling of partial note titles * Add token counter (#1889) * Fix index rebuild on semantic search toggle (#1891) * Update token counter label (#1892) * Implement tag search v3 (#1893) * feat: Enhance TieredLexicalRetriever to support tag-based retrieval and improve search scoring - Added support for returning all matching tags in TieredLexicalRetriever. - Introduced new options for tag terms and returnAllTags in the retriever. - Updated FullTextEngine to prioritize tag matches and improve scoring for documents with tags. - Enhanced tokenization to handle hierarchical tags and prevent splitting hyphenated tags. - Improved handling of frontmatter tags in documents. - Updated search tools to accommodate new tag handling features. - Added tests to verify the functionality of tag-based retrieval and scoring improvements. * fix: Improve logging for query expansion in SearchCore * feat: Add explanation for non-tag matches in FullTextEngine search results * feat: Enhance QueryExpander to preserve standalone terms in tag handling and improve term validation * feat: Enhance QueryExpander and TieredLexicalRetriever to improve tag handling and standalone term extraction * Update version to 3.1.0-preview-251006 in manifest.json * Normalize tag queries for case-insensitive matching and improve search functionality (#1894) * Implement merge retriever (#1896) * Fix note read (#1897) * Enhance note resolution logic and add tests for wiki-linked notes and basename matching * Implement note resolution outcome types and enhance readNoteTool tests for ambiguous matches * Add deriveReadNoteDisplayName function and enhance readNoteTool tests for edge cases * Fix note tool UI freeze (#1898) * Update how tags work in context (#1895) * Enhance file creation instructions in modelAdapter and update tool usage guidelines in builtinTools (#1901) - Added instructions for confirming folder existence before creating new files in modelAdapter. - Updated custom prompt instructions in builtinTools to clarify the use of getFileTree for folder lookups when creating new notes. * Do not add url context for youtube url (#1899) * Update YouTube Script command and modal title to indicate Plus feature (#1903) * Enhance ChatPersistenceManager to handle file save conflicts (#1904) * Enhance ChatPersistenceManager to handle file save conflicts and improve epoch handling * Fix type checking for existing files in ChatPersistenceManager to prevent errors * Fix verify add (#1905) - Rename "Verify" button to "Test" in Add Model dialog - Make verification not required for adding model in Set Keys - Upgrade chatAnthropic client to fix the Top P -1 error for Claude Opus models * Enhance XML parsing to handle tool calls missing closing tags --------- Co-authored-by: Wenzheng Jiang <jwzh.hi@gmail.com> Co-authored-by: Zero Liu <zero@lumos.com> Co-authored-by: Emt-lin <41323133+Emt-lin@users.noreply.github.com> |
||
|
Logan Yang
|
92d6f98af2
|
Implement inline citation (#1821)
* Implement inline citation for vault search for both vault QA and Plus chains refactor: enhance citation utilities and improve source handling in chain runners - Introduced new utility functions for citation management, including sanitization and formatting of source catalogs. - Updated CopilotPlusChainRunner and VaultQAChainRunner to utilize the new citation utilities for improved citation handling. - Enhanced ChatSingleMessage component to consolidate duplicate sources and ensure accurate citation numbering. - Added comprehensive tests for citation utilities to ensure functionality and edge case handling. refactor: dedupe chunks for unique titles in search tool sources refactor: enhance citation handling in CopilotPlus and VaultQA chain runners - Introduced a sanitization function to remove pre-existing citation markers from document content, preventing number leakage. - Updated the formatting of retrieved documents in VaultQA to utilize the new sanitization function for cleaner output. - Improved footnote renumbering logic in ChatSingleMessage to prioritize first-mention order, ensuring accurate citation mapping. refactor: streamline source citation handling in ChatSingleMessage component and enhance web search tool instructions - Simplified the processing of source links in ChatSingleMessage to use a more efficient mapping approach. - Updated the web search tool's instruction to clarify the use of footnote-style citations, ensuring compatibility with Markdown formatting. * feat: add inline citation setting toggle and refactor - Introduced a new setting to enable inline citations in AI-generated responses, allowing for footnote-style references within the text. - Updated citation utilities to manage fallback sources and improve citation instructions based on user settings. - Enhanced CopilotPlusChainRunner and VaultQAChainRunner to utilize the new inline citation feature, ensuring accurate source representation. - Modified ChatSingleMessage component to process inline citations based on the new setting, improving the display of sources in chat messages. - Added tests for new citation functionalities to ensure reliability and correctness. * feat: enhance citation utilities and add tests for normalization - Introduced the `normalizeCitations` function to improve citation formatting by removing periods after citations and handling footnote references. - Updated `processInlineCitations` to ensure correct mapping of non-sequential citations and consolidate duplicate sources. - Added comprehensive tests for the new citation functionalities, ensuring accurate citation processing and representation in AI-generated responses. - Modified QASettings description for clarity regarding the inline citations feature. * fix: address critical bug in citation processing for consecutive citations - Enhanced the `normalizeCitations` function to correctly handle consecutive citations without spaces, ensuring accurate mapping and representation in output. - Added a test case to replicate and validate the fix for the identified bug, confirming that both citations are processed and displayed correctly. - Updated existing citation handling logic to improve robustness and prevent unconverted citations in the output. * refactor: improve citation handling and enhance citation utilities - Refactored `CopilotPlusChainRunner` to streamline citation source management, replacing `lastCitationLines` with `lastCitationSources` for better clarity and structure. - Updated the `addFallbackSources` function to handle citation formatting more effectively, ensuring proper display of titles and paths. - Enhanced the `normalizeCitations` function to support multiple citations in brackets, ensuring correct sorting and mapping of citation numbers. - Added new test cases to validate the handling of alternate headings, HTML summaries, and multiple citations, improving overall robustness of citation processing. - Introduced `getWebSearchCitationInstructions` for web-specific citation guidance, enhancing user experience with clear instructions for inline citations. * fix: move documentations * refactor: normalize footnote rendering in ChatSingleMessage component - Introduced `normalizeFootnoteRendering` function to streamline footnote display by removing separators and backreferences, and fixing numbering artifacts. - Updated `ChatSingleMessage` to utilize the new normalization function, enhancing the clarity and presentation of footnotes in chat messages. - Added unit tests for `normalizeFootnoteRendering` to ensure proper functionality and handling of various footnote scenarios. * test: enhance ChatSingleMessage tests and add deduplication tests for search tools - Updated `ChatSingleMessage.test.tsx` to include `TooltipProvider` for improved rendering context. - Refactored mock implementation for `obsidian` to streamline testing setup. - Introduced new test file `SearchTools.dedupe.test.ts` to validate the deduplication logic in search tool sources, ensuring correct handling of duplicate entries while preserving the highest score. - Added tests to verify deduplication behavior for various scenarios, enhancing overall test coverage and reliability. |