📋 Table of Contents

• Overview
• Requirements
• Quick Start
• Model Compatibility
• Configuration
• Smart Features
• How It Works
• Troubleshooting

Overview

This extension enables VS Code Copilot to use:

• Local Models running on llama.cpp or Ollama
• Cloud Providers including OpenAI, Anthropic, Gemini, and DeepSeek

With full support for:

• ✅ Tool Calling - Full support for function calling (read_file, grep_search, run_in_terminal, etc.)
• ✅ XML Tool Fallback - Automatic parsing of XML-style tool calls from models that don't support JSON format
• ✅ Vision - Image analysis capabilities (if model supports it)
• ✅ Thinking/Reasoning - Support for reasoning models with adaptive token budgeting
• ✅ Multi-turn Conversations - Proper handling of tool results and context
• ✅ Context Management - Smart overflow recovery and compression
• ✅ Token-aware Processing - Smart context management and output budgeting

Requirements

• VS Code 1.120.0 or higher
• API Key for cloud providers OR local model server (llama.cpp/Ollama)
• Local Models benefit from --jinja flag for Qwen models (recommended but not required)

Quick Start

1. Start Your Local Model Server (Optional)

Using llama.cpp with Qwen 3.6:

# Windows PowerShell
Start-Process -NoNewWindow -FilePath "C:\path\to\llama-server.exe" -ArgumentList @(
    "--model", "C:\models\qwen3.6.gguf",
    "--mmproj", "C:\models\mmproj-f16.gguf",
    "--host", "0.0.0.0",
    "--port", "8080",
    "--n-gpu-layers", "999",
    "-fa", "on",
    "--ctx-size", "131072",
    "--jinja",
    "--temp", "0.5",
    "--top-p", "0.85",
    "--top-k", "64",
    "--api-key", "sk-xxx"
)

# Linux/macOS
nohup ~/llama.cpp/build/bin/llama-server \
  --model /home/kelvin/llama.cpp/models/qwen3.6/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf \
  --mmproj /home/kelvin/llama.cpp/models/qwen3.6/mmproj-Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-f16.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --n-gpu-layers 999 \
  -fa on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --batch-size 2048 \
  --ubatch-size 1024 \
  --threads 40 \
  --ctx-size 131072 \
  --jinja \
  --temp 0.5 \
  --top-p 0.85 \
  --top-k 64 \
  --alias qwen3.6 \
  --api-key sk-xxx \
  > ~/llama-server.log 2>&1 &

2. Configure VS Code

Add to .vscode/settings.json or ~/.config/Code/User/settings.json:

{
  "laicopilot.models": [
    {
      "id": "qwen3.6",
      "owned_by": "llamacpp",
      "baseUrl": "http://localhost:8080",
      "context_length": 131072,
      "temperature": 0.5,
      "top_p": 0.85,
      "top_k": 64,
      "vision": true,
      "apiMode": "llamacpp"
    }
  ],
  "laicopilot.logLevel": "info",
  "chat.byokUtilityModelDefault": "mainAgent",
}

3. Use in Copilot Chat

1. Select the model in Copilot Chat dropdown
2. Start asking questions - the model will automatically use tools when needed
3. Watch as it reads files, searches code, and analyzes your project

Model Compatibility

✅ Recommended Models (Full Tool Calling Support)

🤖 Cloud Providers

💡 About XML Tool Call Fallback

Previously: Models like Gemma that output XML-style tool calls (<tool_calls>...</tool_calls>) would cause infinite loops because VS Code didn't recognize them.

Now: The extension automatically parses XML-style tool calls and converts them to proper LanguageModelToolCallPart objects. This means:

• ✅ Models that output XML format now work correctly
• ✅ No more infinite loops from unrecognized tool formats
• ✅ Graceful degradation for models without native JSON tool support

Example: A model outputting <tool call="read_file"><arg name="filePath" value="src/main.rs"/></tool> will be automatically converted to a proper tool call.

Configuration

Core Parameters

Tool Calling & Output

Message & History Management

Context Optimization (Advanced)

Extra (Provider-Specific)

Complete Example Configuration

{
  "laicopilot.models": [
    {
      "id": "qwen3.6",
      "owned_by": "llamacpp",
      "baseUrl": "http://localhost:8080",
      "displayName": "Qwen 3.6 35B (Full Features)",
      
      // Context & Output Budgeting
      "context_length": 131072,
      "max_tokens": 32768,
      "thinking_budget": 256,
      
      // Capabilities
      "vision": true,
      
      // Sampling Parameters
      "temperature": 0.8,
      "top_p": 0.95,
      "top_k": 64,
      "frequency_penalty": 0,
      "presence_penalty": 0,
      
      // History Management
      "max_history_messages": null,        // Keep all
      "max_system_message_length": null,   // Keep full
      
      // Performance Optimization
      "compress_context": false,
      "extra": {
        "cache_prompt": true,
        "n_keep": 512
      }
    }
  ],
  "laicopilot.logLevel": "info"
}

How It Works

Tool Calling Workflow

1. User: "Read src/main.rs and explain it"
   ↓
2. [Extension sends to model]
   - Message: user query
   - Tools: list of 20+ available tools
   ↓
3. [Model processes request]
   - Understands tool format (JSON or XML)
   - Decides to call read_file
   ↓
4. [Model returns]
   JSON: {"tool_calls": [{"name": "read_file", "arguments": {...}}]}
   XML:  <tool call="read_file"><arg name="filePath" value="src/main.rs"/></tool>
   ↓
5. [Extension executes tool, gets file content]
   ↓
6. [Tool result added to conversation history]
   - Role: "tool"
   - Content: file contents
   ↓
7. [Model continues]
   - Processes file content
   - Generates explanation
   ↓
8. User sees: "Here's what main.rs does..."

XML Tool Call Fallback

Previously: Models outputting XML-style tool calls (<tool_calls>...</tool_calls>) would cause infinite loops.

Now: The extension automatically parses XML-style tool calls in two formats:

• Format 1: <tool_call><function=read_file><parameter=filePath>...</parameter></function></tool_call>
• Format 2: <tool call="read_file"><arg name="filePath" value="src/main.rs"/></tool>

These are converted to proper LanguageModelToolCallPart objects, so models that don't support native JSON tool calling now work correctly.

Troubleshooting

❌ "Model produced no visible output"

Cause: Model consumed all tokens on thinking/reasoning, none left for output

Solutions:

1. Reduce thinking_budget:

   "thinking_budget": 128   // or 0 to disable
   

2. Increase max_tokens:

   "max_tokens": 65536
   

3. Check model compatibility:

   • ✅ Use: Qwen 3.6, Qwen 2.5, Hermes, Functionary
   • ❌ Don't use: Gemma, base Llama, generic models

❌ XML Tags Appearing in Response

Example: <tool_calls><tool call="read_file">...</tool></tool_calls>

Cause: Model doesn't understand OpenAI tool format

Solution: Use a tool-aware model

# ✅ These work:
qwen3.6-35B
qwen2.5-32B  
hermes-3-llama-3.1-70B

# ❌ These don't work:
gemma-2-27b
llama-2-7b
mistral-7b

❌ Tool Calling Loops Infinitely

Cause: Model doesn't understand tool results

Verify:

1. Model is tool-aware (use Qwen/Hermes)
2. --jinja flag is enabled in llama-server
3. Check llama-server logs for Jinja template errors

# Check if llama-server is running correctly
curl http://localhost:8080/v1/models

✅ Debugging Tool Calling Issues

Check logs on Windows:

$logPath = Get-ChildItem "C:\Users\$env:USERNAME\.copilot\laicopilotdev\logs" -File | 
  Sort-Object LastWriteTime -Descending | Select-Object -First 1

Get-Content $logPath.FullName | Select-String 'tools.configured|tool_result|isQwenModel'

Expected log output:

tools.configured.openai_compat: 
  {"modelId": "qwen3.6-uncensored", "toolCount": 20, "isQwenModel": true}

llamacpp.conversion.tool_result:
  {"callId": "call_abc123", "contentLength": 1234, "contentPreview": "..."}

llamacpp.conversion.tool_message_added:
  {"tool_call_id": "call_abc123", "contentLength": 1234}

If you see these logs, tool calling is working correctly.

Smart Defaults

The extension automatically calculates optimal values when not configured:

Architecture

• Entry: src/extension.ts - Registers language model provider
• Core: src/provider.ts - Routes requests to API providers
• llama.cpp: src/llamacpp/llamacppApi.ts - OpenAI-compatible endpoint handler
• Tools: src/commonApi.ts - Unified tool processing for all providers
• Utilities: src/utils.ts - Token counting, data URL generation, etc.

Known Limitations

1. Gemma models don't support tool calling - Use Qwen instead
2. Generic models need --jinja for tools - This is a llama.cpp + Qwen requirement
3. Vision requires matching model support - Not all models support images

Acknowledgments

• VS Code Chat Provider API - Official VS Code extension API
• llama.cpp - Local LLM inference engine
• Qwen - Native tool calling support
• Contributor

License

MIT License - Copyright (c) 2026