logancyang_obsidian-copilot/docs
Logan Yang 8cea1a6f17
Audit context envelope, tag alignment, artifact dedup, and logging (#2164)
* fix: align context block tags and make parseContextIntoSegments registry-driven

The context envelope L2 layer was silently dropping URL/media context across
turns due to three compounding issues:

1. Tag mismatch: Mention.ts created <youtube_transcript> blocks but the
   contextBlockRegistry expected <youtube_video_context>. Fixed the tag and
   inner element (<transcript> → <content>) to match the registry.

2. Missing registration: twitter_content blocks had no registry entry, so
   they were invisible to compaction and segment parsing. Added the entry.

3. Hardcoded parser: parseContextIntoSegments only matched <note_context>,
   <active_note>, and <prior_context> via hardcoded regexes, silently
   ignoring all URL/media block types. Rewrote it to dynamically build its
   regex from CONTEXT_BLOCK_TYPES, ensuring any future block type added to
   the registry is automatically handled.

Extracted parseContextIntoSegments into src/context/parseContextSegments.ts
as a standalone pure function for direct testability.

Added 19 new tests covering all registered block types, mixed blocks,
stable/unstable flags, and a registry completeness guard test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: context envelope improvements - regeneration, dedup, per-artifact IDs, XML escaping

Phase A: Fix regeneration from loaded chats by lazily reprocessing context
when contextEnvelope is missing (e.g., messages loaded from disk).

Phase B: Replace static segment IDs ("urls", "selected_text", "web_tabs")
with per-artifact IDs from parseContextIntoSegments, enabling accurate
smart referencing in LayerToMessagesConverter.

Phase C: Deduplicate L2 content by segment ID (last-write-wins) to prevent
linear growth when the same artifact appears across multiple turns.

Phase D: Escape note title/basename in XML context blocks for consistency
with URL/YouTube content. Keep path unescaped as it's used as identifier.

Additional fixes from Codex review:
- Unique IDs for blocks without source extractors (selected_text counter)
- Keep note.path raw to avoid dedup mismatch with TFile.path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent context artifact duplication in LLM payload and L4 memory

CopilotPlusChainRunner was re-injecting processedText (which includes
context XML artifacts) into the user message via ensureUserQueryLabel,
bypassing the envelope's clean L2/L3/L5 separation. This caused context
like YouTube transcripts to appear 3x in the LLM payload.

Fix: use L5_USER envelope text (expanded user query without context XML)
instead of processedText for cleanedUserMessage, trimmedQuestion fallback,
and messageForAnalysis. Also fix BaseChainRunner.handleResponse to save
L5 text to L4 memory instead of processedText.

Other changes:
- Remove dead updateMemoryWithLoadedMessages from chainManager
- Update CONTEXT_ENGINEERING.md with example walkthrough, L4 behavior
  docs, and Phase 6 integration test suite roadmap

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add chain runner envelope usage section and fix Step 0 log

Document per-runner envelope behavior, CopilotPlus/Agent tool flows,
and token efficiency audit in CONTEXT_ENGINEERING.md. Fix Step 0 log
to show L5 user query instead of processedText with context XML.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: trim trailing whitespace from chat input display text

Trailing newlines from Lexical editor were preserved in displayText
and rendered as empty lines in chat bubbles due to whitespace-pre-wrap.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 21:18:25 -08:00
..
AGENT_REASONING_BLOCK.md Migrate to native tool call in plus and agent (#2123) 2026-01-26 18:35:27 -08:00
BEDROCK_TOOL_CALLING.md Migrate to native tool call in plus and agent (#2123) 2026-01-26 18:35:27 -08:00
CITATION_IMPLEMENTATION.md Implement inline citation (#1821) 2025-09-21 20:03:09 -07:00
CONTEXT_ENGINEERING.md Audit context envelope, tag alignment, artifact dedup, and logging (#2164) 2026-02-07 21:18:25 -08:00
deprecating-intent-analyzer.md Deprecate IntentAnalyzer (#2069) 2025-12-15 19:15:44 -08:00
MESSAGE_ARCHITECTURE.md Context revamp (#1971) 2025-10-27 23:28:36 -07:00
NATIVE_TOOL_CALLING_MIGRATION.md Migrate to native tool call in plus and agent (#2123) 2026-01-26 18:35:27 -08:00
SELF_HOSTED_SEARCH.md Migrate to native tool call in plus and agent (#2123) 2026-01-26 18:35:27 -08:00
TECHDEBT.md Implement inline citation (#1821) 2025-09-21 20:03:09 -07:00
TODO-composer-tool-redesign.md Agent UIUX Improvements (#2149) 2026-02-04 17:10:28 -08:00
TOOLS.md Migrate to native tool call in plus and agent (#2123) 2026-01-26 18:35:27 -08:00