CanMV for Visual Studio Code

Language: English | 中文

The CanMV for Visual Studio Code extension brings CanMV K230 board development into Visual Studio Code. It connects to the board through the bundled native backend, runs MicroPython scripts, streams camera frames, manages files on the device, and provides an integrated board terminal.

Features

• Connect and disconnect CanMV K230 boards from the CanMV activity bar or Command Palette.
• Auto-detect supported boards by USB VID/PID 1209:abd1, with a manual serial path override when needed.
• Use legacy and v2 board protocol support through backend capability negotiation.
• Run the active Python file on the board, stop a running script, or run a Python file directly from the device tree.
• Preview live IDE framebuffer images with fit/original-size modes, rotation, PNG capture, pixel RGB picking, ROI histogram sampling, video recording, FPS display, and RGB/grayscale/LAB/YUV histograms with hover readouts.
• Tune grayscale and LAB thresholds with the Threshold Editor, including image file loading, Frame Buffer capture, tuple copy, and selected tuple apply.
• Send virtual touch clicks from the preview when the connected firmware reports virtual touch support.
• Browse mounted device storage, including /sdcard, /data, and /udisk.
• Create, rename, delete, upload, download, open, edit, and auto-sync remote files.
• Save the active editor directly as /sdcard/main.py or /sdcard/boot.py.
• Use the CanMV Terminal panel for board output, REPL input, Ctrl-C script interrupt, log clearing, and log export.
• Browse downloaded official CanMV examples, open examples as editable unsaved buffers, reveal them on disk, and run Python examples on the board.
• Expose a CanMV MCP server for compatible VS Code AI clients, with board detection, connection, script, terminal, remote filesystem, examples, and stubs tools.
• Resolve firmware resources through full commit-hash manifests, then configure matching K230 MicroPython stubs for Pylance and download matching examples.
• Inspect extension, backend, stubs, preview, and transfer logs in the CanMV Output channel.

Requirements

• Visual Studio Code 1.90.0 or newer.
• A CanMV K230 board connected over USB.
• Pylance is installed automatically as an extension dependency for Python analysis.
• For source development, install Node.js/npm and Go.

Quick Start

1. Connect a CanMV K230 board over USB.
2. Open the Command Palette and run CanMV: Connect Board.
3. Open a Python file.
4. Run CanMV: Run Active Python Script from the Command Palette, the editor run button, or the editor context menu.
5. Use the CanMV activity bar views for board status, tools, and device files.
6. Open the Examples view to browse downloaded examples, or open the CanMV Terminal panel to view output and enter REPL commands when no script is running.

Main Views

Common Workflows

Connect to a Board

Run CanMV: Connect Board. The extension starts the backend, detects the board, performs the board handshake, updates the status bar, refreshes the Device tree, and configures firmware-matched resources when possible.

On activation, the extension resolves firmware/latest and uses that firmware manifest to select default stubs and examples. After a board connects, it resolves firmware/<full-commit-hash>/manifest.json and switches to resources matching the connected firmware. If the exact manifest is unavailable, the extension falls back to the latest manifest and then to usable local caches.

If auto-detection does not find the board, set canmv.serialPath to a serial device path such as /dev/ttyACM0. When canmv.serialPath is set, canmv.baudRate is used for the connection.

Run a Script

Open a Python file and run CanMV: Run Active Python Script. While a script is running, the terminal mirrors script output and the run button changes to stop. Use CanMV: Stop Script or press Ctrl-C in the CanMV Terminal to interrupt the script.

The editor context menu also includes CanMV: Run Active File on K230. For remote or mirrored files, this saves/syncs the file and runs the remote path. For ordinary local files, it sends the current editor contents directly to the board.

Preview Camera Frames

Open Preview from the Toolbox or run CanMV: Enable Preview. When the running script publishes IDE framebuffer data, the preview streams JPEG frames into a tool tab.

The Preview tool supports:

• Fit-to-window and original-size viewing.
• 90-degree rotation.
• Saving the current frame as PNG.
• Picking RGB pixel values from the current frame.
• FPS and frame count display.
• RGB, grayscale, LAB, and YUV histograms.
• Selecting an ROI rectangle so histograms are calculated from only that image region.
• Histogram hover readouts for inspecting bin values.
• Recording preview frames to video. The recorder saves MP4 when the VS Code webview runtime supports MP4 encoding and falls back to WebM otherwise.
• Virtual touch click forwarding when supported by the board firmware.

Preview starts automatically after a script begins when the Preview tool is open and preview is not manually disabled.

Tune Thresholds

Open Threshold Editor from the Toolbox or run CanMV: Threshold Editor. The editor can load a local image file or grab the current Frame Buffer/preview canvas image, then preview grayscale or LAB threshold results.

You can copy the generated tuple, or select an existing grayscale/LAB tuple in the active editor and apply the new value directly.

Manage Device Files

Open the Device view after connecting. Directories can be expanded, and files can be opened from the tree. Right-click a directory or mounted root to create files/folders, upload files, or upload a folder. Right-click files or folders to download, rename, or delete them.

Python files opened from the Device tree are mirrored into an extension-managed temp folder outside the current workspace. Saving a mirrored file syncs it back to the board automatically. The extension also updates python.analysis.extraPaths so Pylance can resolve imports from the mirror.

Browse Examples

Open the Examples view in the CanMV activity bar. The extension downloads examples from the same firmware manifest used for stubs and stores them locally. The view shows the active cached examples package, including examples/ and models/ content when present.

Opening an example creates an editable unsaved editor. This lets you experiment freely while keeping the downloaded cache unchanged; use Save As to write a copy into your workspace. Python example files can also be run directly on the connected board from the Examples tree.

Examples are cached under:

~/.kendryte/k230_canmv_examples/<examples-id>

Use CanMV: Refresh Examples to resolve the latest firmware resources and download examples when needed. Use CanMV: Reveal Examples to open the active examples cache in the OS file manager.

Save Startup Files

Use the editor context menu while connected:

• CanMV: Save as main.py writes the active editor to /sdcard/main.py.
• CanMV: Save as boot.py writes the active editor to /sdcard/boot.py.

Use the Terminal

The CanMV Terminal panel keeps recent scrollback, mirrors board/script output, and accepts REPL input when the board is connected and no script is running. Terminal input is disabled while a script runs, except Ctrl-C, which requests a script stop. The terminal webview also supports clearing output and saving the log.

Use MCP Tools

The extension contributes a CanMV MCP Server definition to VS Code. Compatible MCP clients can discover tools for capability analysis, board detection/connection, script execution, preview frames, virtual touch, terminal input/output, remote filesystem operations, host-side artifact saving, and read-only access to cached CanMV examples and MicroPython stubs.

The MCP server runs as a standalone stdio Node process and uses the same bundled backend as the extension. It honors canmv.backendPath, canmv.serialPath, and canmv.baudRate through environment passed by the extension. Example and stub tools read the local caches under ~/.kendryte/k230_canmv_examples and ~/.kendryte/k230_canmv_stubs, so refresh/connect once if those caches are empty.

Board-facing MCP tools auto-connect when hardware access is needed and keep the board session alive for related follow-up calls, such as running a script, starting preview, and reading a frame. The server disconnects on canmv_disconnect_board, when the MCP client exits, or after an idle timeout. Set CANMV_MCP_IDLE_DISCONNECT_MS to adjust the timeout; the default is 120000 milliseconds.

When an AI workflow needs to save an image or downloaded artifact on the host, prefer the host-save tools instead of asking the MCP client to decode base64 text. canmv_save_latest_frame_to_host saves the current preview JPEG directly, canmv_download_file_to_host copies a remote board file to the host, and canmv_save_base64_to_host decodes base64 data returned by another tool. Relative output paths are written under CANMV_MCP_OUTPUT_DIR when set, or a temporary canmv-mcp output directory by default.

MCP capabilities include:

For best script generation, the MCP server instructs AI clients to call canmv_resource_summary, search/read relevant examples, and search/read relevant stubs before writing, saving, or running generated MicroPython code. This grounding step helps avoid scripts that use incorrect imports, outdated APIs, or signatures that do not match the connected CanMV firmware.

Commands

Settings

The backend path can also be overridden with the CANMV_BACKEND_PATH environment variable.

Firmware Resources, Examples, and Python Stubs

The extension resolves stubs and examples through firmware manifests published under:

https://download.kendryte.com/developer/tools/canmv_vscode_extension/

Resource routing starts from a firmware manifest:

firmware/latest
firmware/<full-40-character-commit-hash>/manifest.json

The manifest points to the matching stubs zip and examples zip. Stubs use the full firmware commit hash so Pylance can match the connected firmware exactly. Examples use a content hash ID because examples do not change on every firmware build.

Firmware route metadata is cached under:

~/.kendryte/k230_canmv_resources/firmware/latest
~/.kendryte/k230_canmv_resources/firmware/<revision>/manifest.json

Stubs are cached under:

~/.kendryte/k230_canmv_stubs/<revision>

Examples are cached under:

~/.kendryte/k230_canmv_examples/<examples-id>

On activation, the extension resolves the latest firmware resources, reuses local caches when available, and downloads missing stubs/examples when canmv.stubsAutoDownload is enabled. After a board connects, it switches to the exact firmware route when the board reports a full firmware commit hash.

The extension configures Pylance with an extension-managed python.analysis.stubPath overlay so CanMV MicroPython stubs take precedence over host Python modules with the same names. Existing user stub roots are folded into that overlay when possible, and python.analysis.extraPaths is still updated for the active stubs cache and mirrored board files.

Backend

The extension talks to the board through a bundled native Go backend:

extension/bin/<platform>/canmv-backend
extension/bin/<platform>/canmv-backend.exe

Supported package targets are:

• linux-x64
• linux-arm64
• win32-x64
• win32-arm64
• darwin-x64
• darwin-arm64

For local troubleshooting, build a backend from native/go and set canmv.backendPath or CANMV_BACKEND_PATH to the resulting executable.

Development

Install dependencies and compile the extension:

cd extension
npm install
npm run compile

Build and stage the backend for the current platform:

./scripts/stage-current-backend.sh

Package a VSIX with all supported backend binaries:

./scripts/package.sh

You can also package from the extension directory after compiling and staging a backend:

cd extension
npm run package:vsix

Troubleshooting

• Board is not detected: check board power, USB data cable, permissions, and canmv.serialPath.
• Backend executable is missing: build/stage the backend or set canmv.backendPath.
• Script output is missing: open the CanMV Terminal panel and the CanMV Output channel.
• Preview is empty: make sure the running script publishes IDE framebuffer data.
• Preview stops updating: disable and re-enable Preview, or stop and restart the script.
• Remote file edits are not syncing: save the mirrored local file and check the CanMV Output channel for transfer errors.
• Examples are missing: run CanMV: Refresh Examples, connect once with auto-download enabled, and check the CanMV Output channel for resource download errors.
• Python completions are missing: confirm Pylance is installed, reload Visual Studio Code after stubs are configured, and check python.analysis.stubPath plus python.analysis.extraPaths.
• File operations fail: refresh the Device tree and inspect the CanMV Output channel.

Repository

• Issues: https://github.com/kendryte/canmv-vscode-extension/issues
• Source: https://github.com/kendryte/canmv-vscode-extension