Extension Deprecated - Please use VS Code's Custom Endpoint Provider

VS Code recently added the ability to add models via any endpoint as of version 1.122 meaning that this extension is no longer needed. You can read more at https://code.visualstudio.com/updates/v1_122#_custom-endpoint-provider-in-stable. This extension is now deprecated and will not receive further updates.

I am thankful to everyone who contributed to this project and hope it served as a useful tool.

Llama.cpp Provider for GitHub Copilot

This extension integrates Llama.cpp models into GitHub Copilot in VS Code. It allows users to connect to a local Llama.cpp server and use its models for chat interactions, code generation, and more.

Features

• Integrates Llama.cpp server into VS Code's language model chat.
• Supports streaming responses.
• Handles tool calling for function invocations.
• Manages multiple models from the Llama.cpp server.

Requirements

• VS Code version 1.104.0 or higher.
• A running Llama.cpp server with OpenAI-compatible API.

Stack

• TypeScript: A typed superset of JavaScript.
• VS Code API: APIs for building extensions.

Design

The extension uses a base provider class for OpenAI-compatible chat APIs. The Llama.cpp provider extends this base to connect to a local Llama.cpp server. It handles model fetching, message conversion, and streaming responses. Tool calling is supported through OpenAI-compatible formats.

Setting Up Project Locally

1. Clone the repository.

git clone https://github.com/mbeps/llama-vscode-chat.git

2. Install dependencies.

npm install

3. Compile the extension.

npm run compile

4. Open in VS Code and run the extension.

Configuration

1. Open the command palette.
2. Select "Llama.cpp: Set Server URL"
3. Configure the URL of your Llama.cpp server (e.g., http://localhost:8000).

References

• Llama.cpp Documentation
• VS Code Extension API