AI Toolkit is a powerful extension for Visual Studio Code that streamlines agent development. With AI Toolkit, you can:
- 🔍 Explore and evaluate models from a wide range of providers—including Anthropic, OpenAI, GitHub—or run models locally using ONNX and Ollama.
- ⚡ Build and test agents in minutes with prompt generation, quick starters, and seamless MCP tool integrations.
Complete features include:
| Feature |
Description |
Screenshot |
| Model Catalog |
Browse and access AI models from various sources. Simplified discovery of GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google models. |
 |
| Playground |
Interactive environment for testing AI models. Quick experimentation with model capabilities including multi-modal support. |
 |
| Prompt (Agent) Builder |
Tools for creating and optimizing prompts. Iterative improvement of prompt engineering techniques. |
 |
| Bulk Run |
Execute multiple prompts across selected models. Efficient testing at scale with various inputs. |
 |
| Evaluate an AI model with a dataset |
Test AI models against datasets using standard metrics. Measure performance with using built-in evaluators such as F1 score, relevance, similarity, and coherence or create your own evaluators. |
 |
| Fine-tune |
Customize models for specific use cases. Adapt models to specialized domains and requirements. |
 |
Getting started
We recommend starting with models hosted by GitHub.
- Follow the installation guide to set up AI Toolkit for your device.
- From the extension tree view, select CATALOG > Models to explore models available. We recommend to getting started with models hosted by GitHub.
- From the model card, select Try in Playground to start experimenting the capability of an AI Model.
Build AI agents
The key feature of AI Toolkit is to build AI agents. The agent builder provides a set of tools to help you create and optimize your AI agents. You can use the agent builder to:
- 🚀 Generate starter prompts with natural language
- 🔁 Iterate and refine prompts based on model responses
- 🧩 Break down tasks with prompt chaining and structured outputs
- 🧪 Test integrations with real-time runs and tool use such as MCP servers
- 💻 Generate production-ready code for rapid app development
And a lot of features are coming soon, stay tuned for:
- 📝 Use variables in prompts
- 🧪 Run agent with test cases to test your agent easily
- 📊 Evaluate the accuracy and performance of your agent with built-in or your custom metrics
- ☁️ Deploy your agent to cloud
Agents can now connect to external tools through MCP (Model Control Protocol) servers, enabling them to perform real-world actions like querying a database, accessing APIs, or executing custom logic.
| Feature |
Description |
Screenshot |
| Connect to an Existing MCP Server |
Use tools from command(stdio) or HTTP (server-sent event) |
 |
| Build and Scaffold a New MCP Server |
Start creating your own MCP server from a simple scaffold and test in Agent Builder |
 |
Feedback and resources
Your feedback is crucial to shape the feature roadmap. Read our developer documentations for more features or reach us in GitHub to report an issue.
AI Toolkit ❤️ Developer Community.
Data and telemetry
The AI Toolkit for Visual Studio Code collects usage data and sends it to Microsoft to help improve our products and services. Read our privacy statement to learn more. This extension respects the telemetry.enableTelemetry setting which you can learn more about at disable telemetry reporting.
