Starlake for Visual Studio Code

Build data pipelines faster. The Starlake extension brings the full power of the Starlake data integration platform into VS Code: from schema inference and SQL transformations to database browsing, ER diagrams, lineage visualization, and workflow orchestration.

This extension also ships with Starlake Skills, a set of MCP-based skills that supercharge AI coding assistants like Claude Code, Google Gemini and GitHub Copilot with deep knowledge of the Starlake platform. With these skills, your AI assistant can help you build, debug, and optimize your data pipelines using Starlake best practices.

Works with BigQuery, Snowflake, Redshift, Databricks, DuckDB, and more.

Key Features

Data Ingestion

Load files, automatically infer schemas from your data sources, and generate YAML configurations. No manual schema writing required.

SQL Transformations

Write, preview, and execute SQL transformations with Jinja2 templating support. Dry-run jobs to validate before execution.

Visualization

Generate interactive ER diagrams of table relationships, explore data lineage across your pipeline, and review Access Control Lists, all rendered in a built-in viewer panel.

Database Browser

The Database view in the Starlake activity bar lists your project's connections, schemas, tables and columns (with their types). Preview the first rows of any table in the Starlake panel, or open the query in a new worksheet. The active connection is highlighted, and hovering a connection shows its details with credentials masked.

Worksheet Notebooks

Files under metadata/worksheets/ open as notebooks: mix SQL cells and Markdown cells, run a cell (or all of them) against the Starlake server, and read the results right under the cell. Worksheets stay plain .sql files on disk, so the text editor, the Starlake UI and the CLI keep working on them.

Query Performance

Every query you run (worksheet cells, Run Query, Run Transform, CodeLens Execute and Execute CTE, dry runs and table previews) shows how long it took next to its result. The status bar shows the last run, and the Query History view lists recent runs with their duration, SQL and status.

SQL Formatting

Right-click in a SQL file and choose Format Document or Format Selection (or press Shift+Alt+F), or right-click .sql files and folders in the Explorer and choose Format SQL. Formatting uses the SQL dialect of your active connection and works without the Starlake server.

Orchestration

Generate, dry-run, and deploy workflow DAGs for orchestrators like Airflow and Dagster directly from VS Code.

Project Management

Bootstrap new Starlake projects, validate configurations, switch environments and modes from the status bar, and manage server connectivity without leaving the editor. On startup, the extension detects the Starlake CLI and offers to install it or start the server for you.

Quick Start

1. Install Prerequisites

Java 17+ is required:

# macOS
brew install openjdk@17

# Linux
sudo apt install openjdk-17-jdk

Starlake CLI:

# macOS / Linux
bash <(curl -sL https://starlake.ai/setup.sh)

# Windows (PowerShell)
Invoke-Expression (Invoke-WebRequest -Uri "https://starlake.ai/setup.ps1").Content

If the CLI is not on your PATH, set starlake.installDir to its installation directory.

2. Open Your Project

Open your Starlake project folder in VS Code. The extension activates automatically and checks that the Starlake server is reachable:

• CLI not found: the extension offers to install it in a terminal, or to open the installation docs.
• Server not running: the extension offers to start it (starlake serve) in a terminal and waits until it is ready.

No project yet? Run Starlake: New Project from the Command Palette.

3. Start the Server Manually (optional)

starlake serve

Listens on port 9900 by default. Use --port PORT to change it, and update starlake.serverPort to match.

Commands

All commands are available via the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and context menus.

Orchestration (right-click context menu)

Database view

Database Browser

Open the Starlake icon in the activity bar to show the Database view. It lists the connections defined in application.sl.yml, then the schemas, tables and columns of each connection. Column types are shown with their precision, for example DECIMAL(10,2).

• The tree loads lazily and caches each node. Use Refresh on a node to reload it.
• Spark and filesystem connections cannot be browsed and are shown greyed out.
• The active connection (driven by the Env status bar item and Airplane Mode) is marked in the tree. Hover a connection to see its details; credentials are masked.
• Preview Data (inline button on a table) shows up to starlake.catalog.previewLimit rows in the Starlake panel, along with the active connection. If the preview fails, the panel shows the connection details above the error and lets you retry.
• Switching Env or Airplane Mode, or editing connectionRef in application.sl.yml or an env*.sl.yml file, refreshes both the tree and the panel.
• Requests time out after starlake.catalog.timeoutSeconds seconds.

The Database view requires a Starlake server with the catalog-* commands (1.8.11 or later). Older servers show a "doesn't support database browsing" message.

Environments and Airplane Mode

The Starlake Env status bar item controls where queries run. Clicking it opens a picker:

• Env: picks the environment whose variables (metadata/env.<name>.sl.yml) are applied. None uses the default env.sl.yml only.
• Airplane Mode (last entry of the picker): when on, everything runs locally on DuckDB; when off, queries run on the project's target data warehouse. The status bar shows ✈ Airplane while it is on. Switching the Env keeps the Airplane Mode setting.

Worksheet Notebooks

Every .sql file under metadata/worksheets/ opens as a notebook. Cells are delimited with comment markers, so the file remains valid SQL:

-- %% [markdown]
-- # Customer analysis
-- Some *markdown* documentation.
-- %% [sql]
SELECT * FROM sales.customers;

• A line matching -- %% (optionally followed by [sql] or [markdown]) starts a new cell; no tag means sql.
• Markdown cell lines are stored with a -- prefix; a worksheet that is a single SQL cell is saved without any marker.
• Run a cell with the play button or Ctrl+Enter / Shift+Enter; Run All runs cells top to bottom and stops at the first error.
• The notebook is saved before running. Results show up to starlake.notebookPageSize rows (default 100).
• Use Starlake: Open Worksheet as SQL Text (or right-click the file and choose Open With...) to edit the raw file.
• The notebook is wired through a workbench.editorAssociations default for **/worksheets/**/*.sql. Set that key to "default" in your settings if you prefer worksheets to open as plain text.

Query History

The Query History view in the Starlake activity bar lists the queries run in this workspace, newest first. Each entry shows the elapsed time, the first line of the SQL and when it ran; failed runs are shown in red. Hover an entry for the full SQL, the Env, the number of rows and the error message.

• Open in Worksheet opens the SQL in a new worksheet; Copy SQL copies it.
• Clear History (view title bar) empties the list.
• The status bar shows the duration of the last query; click it to open the view.
• Durations are measured in the extension, from sending the request to receiving the full response. They include network and result transfer time.
• The history keeps the last starlake.queryHistory.maxEntries runs (default 200). Set it to 0 to turn the history off; durations are still shown next to results.

SQL Formatting

The extension formats SQL with sql-formatter.

• In the editor: right-click and choose Format Document or Format Selection, press Shift+Alt+F, or enable editor.formatOnSave. In worksheet notebooks, use Format Cell. Indentation follows the editor's tab size settings.
• In the Explorer: right-click one or more .sql files or folders and choose Format SQL. Folders are searched recursively (node_modules, .git, target and dist are skipped), and you are asked to confirm before several files are changed. Worksheet notebooks are formatted cell by cell, leaving markdown cells untouched.
• Dialect: with starlake.format.dialect set to auto (the default), DuckDB is used in Airplane Mode; otherwise the dialect follows the engine of the active connection (BigQuery, Snowflake, Postgres, Redshift, ...). Set the option to force a dialect.
• Jinja: {{ ... }} expressions are kept as they are. Files that contain Jinja control blocks ({% ... %}) are not formatted.
• A file that cannot be parsed is left unchanged and the error is reported.

Extension Settings

Configure the extension in your VS Code settings.json:

BigQuery Setup

If you are working with Google BigQuery, authenticate and set your project:

gcloud auth application-default login
gcloud config set core/project MY_PROJECT_ID

Install gcloud via brew install google-cloud-sdk (macOS) or sudo apt install google-cloud-sdk (Linux).

Recommended Companion Extensions

• Jinja / SQL Highlighting: Syntax highlighting for SQL templates with Jinja2
• Log Output Colorizer: Colored log output for better readability

Resources

• Starlake Documentation
• GitHub Repository
• Report an Issue

License

MIT