AI Code Assistance — General Coding + Data + Data Science + AI Engineering

A Vibe Coding assistant for VS Code and Cursor that goes far beyond simple chat. AI Code Assistance turns your editor into a specialist across four full areas:

• General coding — explain, refactor, generate, fix code in any language
• Data analytics — inventory datasets, frame business questions, drill into root causes, generate decks
• Data Science — clean, EDA, build/train/tune ML models, explain them with SHAP/LIME
• AI Engineering — production RAG, prompt engineering, LoRA fine-tuning, LLM evaluation, agents

Powered by a local backend (a small FastAPI router you run on your machine) that talks directly to the model provider of your choice — bring your own keys for OpenRouter, OpenAI, Groq, Gemini, xAI or Ollama Cloud, or run fully local with Ollama, LM Studio or vLLM. No Colab, no tunnel, no third-party server in the middle.

The assistant responds in whatever language you write in — English, Portuguese, Italian, Spanish, French, or any other. Just type naturally.

Before you install: the extension needs the backend, and the backend lives in a repository that is currently private. Until it is opened up, installing from the Marketplace alone will not give you a working assistant — see Requirements.

Architecture

   ┌──────────────────────┐          ┌───────────────────────────┐        ┌────────────────────┐
   │ VS Code / Cursor     │  HTTP/   │ Local backend             │  HTTPS │ Provider           │
   │  ┌────────────────┐  │  SSE     │ (FastAPI router on        │ ─────▶ │ OpenRouter/OpenAI  │
   │  │ AI Code chat   │◄─┼──────────┼─ 127.0.0.1:8123)          │        │ Groq/Gemini/xAI    │
   │  └────────────────┘  │          │                           │        │ Ollama Cloud …or…  │
   │  - 16 slash skills   │          │  6 roles, one model each  │ ─────▶ │ Ollama / LM Studio │
   │  - 16 tools          │          │  task decomposition (DAG) │        │ vLLM (local, free) │
   │  - Voice + vision    │          │  MLflow tracing           │        └────────────────────┘
   │  - MCP servers       │          └───────────────────────────┘
   └──────────────────────┘

Each task routes to one of six roles — planner, coder, vision, general, agentic, datascience — and each role runs on whatever model you assigned it. Tools execute only on your machine; the backend just emits the intent and the extension asks you for permission.

Multi-agent orchestration

A simple request runs on a single agent. A request that spans several fronts is decomposed into a plan, and each step goes to the specialist it needs. Steps with no dependencies run at the same time.

  "analisa este projeto todo: a arquitetura, a qualidade do código
   e o dataset em data/vendas.csv"
                │
                ▼
        ┌───────────────┐
        │ orchestrator  │   decomposes → DAG, one model per step
        └───────┬───────┘
                │
    ┌───────────┴────────────┐
    │  WAVE 1 (in parallel)  │
    │                        │
    ▼                        ▼
┌────────────────┐   ┌──────────────────────┐
│ 01 planner     │   │ 03 datascience       │
│ architecture   │   │ profile vendas.csv   │
└───────┬────────┘   └──────────────────────┘
        │ depends_on: [1]
        ▼
┌────────────────┐        each step:  reads files · runs bash · calls tools
│ 02 coder       │                    can delegate() to another role
│ code quality   │
└────────────────┘

That plan is not an illustration — it is what the orchestrator returns for that request. Two things it does on purpose:

• A full data-science request stays ONE step. The phases share state (the loaded dataframe, the fitted pipeline); splitting them would make every step reload everything.
• A conceptual question goes to the specialist too. "What is the pipeline of an ML project?" is a datascience step, not a general one — the role that carries the field's literature answers it.

You can watch this run, step by step, at ai-code-assistance.netlify.app#agentes.

What you can do — the four areas

🧑‍💻 General coding (works everywhere)

• Open Chat: dedicated sidebar chat with an AI Code Assistance icon in the Activity Bar (same UX as Copilot Chat / Claude Code). Click and use, no setup ritual.
• Inline commands:
  • AI Code Assistance: Explain Selection (Ctrl+Alt+E)
  • AI Code Assistance: Refactor Selection (Ctrl+Alt+R)
  • AI Code Assistance: Generate at Cursor
• Tool calling — 16 tools (see the table below). Every call shows a confirmation modal with Accept / Always / Deny. Dangerous bash patterns (rm -rf /, dd, fork bombs) block the "Always" option as a safety net.
• Optional container sandbox — bash can run inside a container with only your project mounted, no network and all capabilities dropped. If Docker is missing it refuses; it never silently falls back to your host.
• MCP support: connect any MCP server via .mcp.json in your workspace root — both stdio and HTTP transports.
• Hooks and path scope — run your own command before/after any tool call, and decide what happens when the agent reaches outside the folders you opened.
• Voice input and image attachments for vision-capable models.
• Persistent chat history across sessions, with token-aware compaction.
• Multilingual: responds in the language you write in — no configuration needed.

📊 Data analytics — /explore, /question, /chart, /drill, /report

🔬 Data Science (ML) — /engineer, /eda, /model, /explain

🤖 AI Engineering — /rag, /prompt, /finetune, /eval, /agent

🧭 Craft — /karpathy, /workflow

Type /skills or /help in the chat to list all 16 with descriptions.

The 16 tools

Observability (optional)

Point the backend at an MLflow server and every turn becomes a trace: the role that ran, why it was routed there, time to first token, tokens per second, token usage, the tool calls, and 👍/👎 feedback. Secrets are redacted before anything is written. It is entirely optional — without MLflow the assistant works the same.

Choose which model runs each task

AI Code Assistance routes every request to one of six roles — planner, coder, vision, general, agentic, datascience — and you decide which model each role uses. Three interchangeable ways, all sharing one config file and hot-reloaded with no restart:

• ⚙️ Configure Models panel (AI Code Assistance: Configure Models) — a model dropdown per task, a token field per provider (saved to your local .env, never shared), and base-URL fields for Ollama / LM Studio / vLLM.
• Chat commands — /models to see assignments, /model coder openrouter qwen/qwen-2.5-coder-32b-instruct to pin one, /model now <provider> <model> to force a model for the current chat.
• Edit ai-code-assistance.models.json by hand. It holds only model names (no secrets), so you can commit and share it; teammates add their own keys.

Ten providers are accepted, and roles can mix freely: openrouter, openai, groq, gemini, xai, ollama_cloud, ollama, lmstudio, vllm, docker.

Requirements

• VS Code 1.85+ or Cursor.
• Python 3.10+ to run the backend.
• At least one provider key, or a local engine (Ollama / LM Studio / vLLM) for zero cost.
• Access to the backend repository. It is private today. The extension is a client — without the backend it installs and opens, but it cannot answer. If you do not have access, ask for it before installing.
• Docker only if you want the container sandbox for bash.

Quick start

1. Install — search AI Code Assistance in the Extensions panel (Ctrl+Shift+X), or install the .vsix (Cursor does not read the Microsoft Marketplace).

2. Run the backend locally (from the project repo):

   cp .env.example .env          # set AI_TOKEN + at least one provider key
   uvicorn backend.main:app --host 127.0.0.1 --port 8123
   

3. Connect the two — Command Palette (Ctrl+Shift+P) → AI Code Assistance: Quick Setup, which asks for the URL and the token. By hand, in Settings (Ctrl+,) → search AI Code Assistance:

   • aiCodeAssistance.backendUrl → http://127.0.0.1:8123 (must match the port uvicorn printed)
   • aiCodeAssistance.token → the AI_TOKEN from your .env

4. Click the AI Code Assistance icon in the Activity Bar (left rail) — the chat opens in the sidebar. Confirm with Command Palette → Check Backend Health.

5. Type /skills to see everything you can do, and click the ⚙️ to pick your models.

Settings

MCP Servers

Create a .mcp.json file at your workspace root to connect external MCP servers:

{
  "mcpServers": {
    "my-server": {
      "command": "npx",
      "args": ["-y", "@my-org/mcp-server"],
      "env": { "API_KEY": "your-key-here" }
    },
    "http-server": {
      "url": "http://localhost:3000/mcp",
      "headers": { "Authorization": "Bearer TOKEN" }
    }
  }
}

The ⚙️ Configure Models panel shows active MCP servers and has an "Edit .mcp.json" button.

Keybindings

Privacy & security

• Tools require explicit per-session approval. "Always" decisions live in RAM and reset when VS Code closes.
• The token is machine-scoped — it does not sync across devices via Settings Sync.
• The backend runs on your own machine and calls your own provider account with your own keys. No third-party server in the middle. Provider keys live in your local .env (gitignored) — never in the shareable model config.
• File system access and shell execution happen only on your machine — the backend just forwards the model's intent. With the sandbox enabled, bash additionally sees only your project, with no network.
• Traces sent to MLflow are redacted for secrets, and MLflow is optional.

Why this stack?

• Your models, your bill. Mix cloud (OpenRouter / OpenAI / Groq / Gemini / xAI / Ollama Cloud) and local (Ollama / LM Studio / vLLM) per task.
• Right model per task. A cheap fast model for general chat, a strong coder for code, a data specialist for datasets, a vision model for screenshots — configured independently.
• Shareable, secret-free config. The model config file carries model choices; teammates plug in their own keys.
• Editor stays local. All file I/O and shell execution happen on your machine, behind a permission prompt.
• Multilingual. The assistant responds in the language you write in, automatically.

Limitations

• You run the backend yourself (a single uvicorn command) and supply at least one provider key, or a local engine. The extension does not include hosted inference.
• The backend repository is private today — see Requirements.
• The AI Engineering skills are prompts, not pipelines: they tell you what to write and why. The agent can then run what it wrote — bash executes it and run_notebook executes notebooks — but no skill trains a model or builds an index on its own.
• Nothing on the machine executes notebooks unless you install a runner (papermill, nbclient or nbconvert); run_notebook reports honestly when none is present.

Issues & contributing

Bug reports, feature requests and PRs: github.com/mendesalex89/AI_Assistance (private while the project is in development — ask for access).

License

MIT — see LICENSE.