# Tool System Documentation

## Overview

The Copilot tool system uses a centralized registry pattern that makes it easy to add new tools, including future MCP (Model Context Protocol) tools. All tools are managed through a singleton `ToolRegistry` that provides a unified interface for tool discovery, configuration, and execution.

## Tool Prompt Architecture

### How Tool Instructions Flow to the LLM

The system uses **native tool calling** via LangChain's `bindTools()` for tool invocation. Tool results are formatted as context in a layered approach:

1. **Tool Schema Descriptions** (Zod schemas in tool implementations)

   - Defines parameter formats, rules, and validation
   - Provided to LLM via `bindTools()` for native tool calling

2. **Custom Prompt Instructions** (in `builtinTools.ts`)

   - Behavioral guidance for when and how to use tools
   - Special requirements (e.g., "always provide salientTerms")

3. **Model-Specific Adaptations** (in `modelAdapter.ts`)
   - Last resort for model-specific quirks

### Layered Prompt Integration

- `ContextManager` promotes user-attached artifacts from earlier turns into **L2 (Context Library)**, so every chain runner starts with the same cacheable system prefix.
- When a tool executes during the current turn, its result payload is prepended to the **user message** (L3 + L5) using `renderCiCMessage(...)`. Nothing is injected into the system message, keeping L1/L2 stable.
- `LayerToMessagesConverter.convert(envelope, { includeSystemMessage: true, mergeUserContent: true })` materializes the base messages; runners then append tool results before sending to the model.
- `promptPayloadRecorder` inspects the final payload and highlights tool blocks in its layered view, making it easy to debug the L1-L5 structure.

### Why Two Layers: Schema vs Custom Instructions

**Key Difference**: Schema descriptions document parameters via Zod, while custom instructions provide behavioral guidance.

1. **Clear Separation**

   - **Schema**: Parameter documentation (types, formats, rules) via Zod - used by `bindTools()`
   - **Custom Instructions**: Behavioral guidance on when and how to use the tool

2. **MCP Compatibility**

   - External tools provide immutable schemas
   - We add custom instructions without modifying their code

3. **Example**

   ```typescript
   // Schema (provided to LLM via bindTools())
   const searchSchema = z.object({
     query: z.string().min(1).describe("The search query"),
     salientTerms: z.array(z.string()).describe("Keywords to find in notes"),
   });

   // Custom Instructions (behavioral guidance)
   customPromptInstructions: `
   When searching notes:
   - Always provide salientTerms extracted from the user's query
   - Use getTimeRangeMs first for time-based queries
   - Examine relevance scores before using results
   `;
   ```

### Best Practices

1. **Schema Descriptions: Parameter Documentation Only**

   - Document parameter types, formats, and validation rules via Zod
   - Schemas are automatically provided to LLM via `bindTools()`
   - Focus on the data contract

2. **Custom Instructions: Behavioral Guidance**

   - Explain when to use this tool vs alternatives
   - Show common usage patterns and requirements
   - Include tips for better results (e.g., "use getTimeRangeMs before localSearch")

3. **Model Adapters: Model-Specific Fixes**
   - Only for persistent model-specific failures
   - Keep minimal and targeted

### localSearch CiC Prompting Flow

- CiC: Corpus in Context https://arxiv.org/pdf/2406.13121
- **Instruction First**: `CopilotPlusChainRunner` now assembles the localSearch payload via `buildLocalSearchInnerContent`, ensuring citation guidance (e.g., `<guidance>` rules) tops the XML block before any documents.
- **Documents Next**: Search hits are serialized once through `formatSearchResultsForLLM`; the helper simply appends them after guidance, keeping the documents section untouched but clearly separated.
- **Question Last**: `renderCiCMessage` formats the final prompt so any context precedes the user's original query; this matches the CiC recommendation for instruction → context → query ordering.
  - **CopilotPlus**: Uses `LayerToMessagesConverter` which adds `[User query]:` label when merging L3+L5 content from envelope
  - **AutonomousAgent**: Uses `ensureCiCOrderingWithQuestion` which adds `[User query]:` label to clearly separate tool results from original query in iterative loop
  - **Consistency**: Both chains use the same `[User query]:` label format for uniform prompting across the codebase
- **Reusable Wrapping**: `wrapLocalSearchPayload` centralizes the `<localSearch>` tag creation (including optional `timeRange`), making the layout reusable for future chains without copying string glue.

## Current Implementation

### Core Files

- `src/tools/ToolRegistry.ts` - Central registry for all tools
- `src/tools/builtinTools.ts` - Built-in tool definitions and initialization
- `src/LLMProviders/chainRunner/AutonomousAgentChainRunner.ts` - Tool execution in the agent
- `src/settings/v2/components/ToolSettingsSection.tsx` - Settings UI for tool configuration

### Tool Registry Pattern

The `ToolRegistry` is a singleton that manages all tools:

```typescript
class ToolRegistry {
  static getInstance(): ToolRegistry;
  register(definition: ToolDefinition): void;
  registerAll(definitions: ToolDefinition[]): void;
  getAllTools(): ToolDefinition[];
  getEnabledTools(enabledToolIds: Set<string>, vaultAvailable: boolean): SimpleTool<any, any>[];
  getToolsByCategory(): Map<string, ToolDefinition[]>;
  getConfigurableTools(): ToolDefinition[];
  getToolMetadata(id: string): ToolMetadata | undefined;
  clear(): void;
}
```

### Tool Definition Structure

```typescript
interface ToolDefinition {
  tool: SimpleTool<any, any>; // The actual tool implementation
  metadata: ToolMetadata; // UI and configuration metadata
}

interface ToolMetadata {
  id: string; // Unique identifier
  displayName: string; // Shown in UI
  description: string; // Help text
  category: "search" | "time" | "file" | "media" | "mcp" | "custom";
  isAlwaysEnabled?: boolean; // If true, not configurable (e.g., time tools)
  requiresVault?: boolean; // Needs vault access
  customPromptInstructions?: string; // Tool-specific prompts
}
```

## Adding a New Built-in Tool

### 1. Implement the Tool

Create your tool following the `SimpleTool` interface:

```typescript
// Example: New built-in tool
import { z } from "zod";
import { SimpleTool } from "./SimpleTool";

export const myNewTool: SimpleTool<{ input: string }, { result: string }> = {
  name: "myNewTool",
  description: "Description for the LLM to understand when to use this tool",
  schema: z.object({
    input: z.string().describe("The input parameter description"),
  }),
  func: async (params) => {
    // Tool implementation
    const result = await performOperation(params.input);
    return { result };
  },
};
```

### 2. Add to Built-in Tools

Update `src/tools/builtinTools.ts`:

```typescript
export const BUILTIN_TOOLS: ToolDefinition[] = [
  // ... existing tools ...
  {
    tool: myNewTool,
    metadata: {
      id: "myNewTool",
      displayName: "My New Tool",
      description: "User-friendly description for settings UI",
      category: "custom", // Choose appropriate category
      // Optional flags:
      isAlwaysEnabled: false, // Set true if tool should always be available
      requiresVault: true, // Set true if tool needs vault access
      customPromptInstructions: "Special instructions for the AI when using this tool",
    },
  },
];
```

### 3. Update Default Settings (if configurable)

If the tool is configurable (not always-enabled), add its ID to the default enabled tools in `src/constants.ts`:

```typescript
autonomousAgentEnabledToolIds: [
  "localSearch",
  "webSearch",
  "pomodoro",
  "youtubeTranscription",
  "writeToFile",
  "myNewTool"  // Add your tool ID here
],
```

## Adding MCP Tools (Future Implementation)

### 1. MCP Tool Wrapper

Create a wrapper to convert MCP tools to the SimpleTool interface:

```typescript
function createMcpToolWrapper(serverName: string, mcpTool: McpTool): SimpleTool<any, any> {
  return {
    name: `${serverName}_${mcpTool.name}`,
    description: mcpTool.description || `MCP tool from ${serverName}`,
    schema: convertMcpSchemaToZod(mcpTool.inputSchema),
    func: async (params) => {
      // Call the MCP server
      const result = await mcpHub.callTool(serverName, mcpTool.name, params);

      // Convert MCP response to expected format
      return {
        result: formatMcpResponse(result),
      };
    },
  };
}
```

### 2. Dynamic MCP Tool Registration

Register MCP tools when servers connect:

```typescript
// In your MCP initialization code
export async function registerMcpServerTools(serverName: string, mcpTools: McpTool[]) {
  const registry = ToolRegistry.getInstance();

  for (const mcpTool of mcpTools) {
    registry.register({
      tool: createMcpToolWrapper(serverName, mcpTool),
      metadata: {
        id: `mcp_${serverName}_${mcpTool.name}`,
        displayName: mcpTool.displayName || mcpTool.name,
        description: mcpTool.description || `MCP tool from ${serverName}`,
        category: "mcp",
        // MCP tools are user-configurable by default
        isAlwaysEnabled: false,
        // Add any MCP-specific prompt instructions
        customPromptInstructions: mcpTool.systemPrompt,
      },
    });
  }
}

// When MCP server disconnects
export function unregisterMcpServerTools(serverName: string) {
  const registry = ToolRegistry.getInstance();
  const allTools = registry.getAllTools();

  // Remove tools from this server
  const toolsToKeep = allTools.filter((t) => !t.metadata.id.startsWith(`mcp_${serverName}_`));

  registry.clear();
  registry.registerAll(toolsToKeep);

  // Re-initialize built-in tools
  initializeBuiltinTools(app.vault);
}
```

### 3. Schema Conversion Helper

Convert MCP JSON Schema to Zod schema:

```typescript
function convertMcpSchemaToZod(jsonSchema: any): z.ZodSchema {
  // Basic implementation - extend as needed
  if (jsonSchema.type === "object") {
    const shape: any = {};

    for (const [key, prop] of Object.entries(jsonSchema.properties || {})) {
      const propSchema = prop as any;

      if (propSchema.type === "string") {
        shape[key] = z.string();
        if (propSchema.description) {
          shape[key] = shape[key].describe(propSchema.description);
        }
      } else if (propSchema.type === "number") {
        shape[key] = z.number();
      } else if (propSchema.type === "boolean") {
        shape[key] = z.boolean();
      } else if (propSchema.type === "array") {
        shape[key] = z.array(z.any()); // Simplification
      } else if (propSchema.type === "object") {
        shape[key] = z.object({});
      }

      // Handle optional properties
      if (!jsonSchema.required?.includes(key)) {
        shape[key] = shape[key].optional();
      }
    }

    return z.object(shape);
  }

  // Fallback for other types
  return z.any();
}
```

### 4. Settings Storage for MCP Tools

MCP tool preferences are stored in the same array as built-in tools:

```typescript
// When enabling/disabling MCP tools:
function updateMcpToolSetting(toolId: string, enabled: boolean) {
  const settings = getSettings();
  const enabledIds = new Set(settings.autonomousAgentEnabledToolIds || []);

  if (enabled) {
    enabledIds.add(toolId);
  } else {
    enabledIds.delete(toolId);
  }

  updateSetting("autonomousAgentEnabledToolIds", Array.from(enabledIds));
}
```

## How the System Works

### Tool Discovery Flow

1. **Initialization**: `initializeBuiltinTools()` registers all built-in tools
2. **MCP Connection**: When MCP servers connect, their tools are dynamically registered
3. **Settings UI**: `ToolSettingsSection` component reads from the registry to generate UI
4. **Tool Execution**: `AutonomousAgentChainRunner.getAvailableTools()` filters tools based on settings

## Tool Call Rendering Roots

React invariant #409 surfaced when tool-call banners attempted to render into React roots that had already been unmounted. To prevent this regression, the plugin routes all banner rendering through the shared manager in `src/components/chat-components/toolCallRootManager.tsx`.

### Manager Responsibilities

- Tracks `{ root, isUnmounting }` per message/tool call via `window.__copilotToolCallRoots`.
- `ensureToolCallRoot` finalises pending disposals and creates a new `createRoot` when needed.
- `renderToolCallBanner` renders `<ToolCallBanner />` into the managed root; components never call `root.render` directly.
- `removeToolCallRoot` and `cleanupMessageToolCallRoots` schedule unmounts on the next tick and drop entries only after disposal completes.
- `cleanupStaleToolCallRoots` purges message IDs older than one hour to avoid leaking historical roots.

### Integration Notes

`ChatSingleMessage` keeps `const rootsRef = useRef(getMessageToolCallRoots(messageId))`, which provides a stable registry for each message. The component delegates all lifecycle calls to the manager and snapshots `rootsRef.current` inside effect cleanup to satisfy `react-hooks/exhaustive-deps`.

### Verification

Run the focused test to cover the streaming behaviour and tool-call integration:

```
npm test -- src/components/chat-components/ChatSingleMessage.test.tsx
```