Starlake for Visual Studio Code

Build data pipelines faster. The Starlake extension brings the full power of the Starlake data integration platform into VS Code — from schema inference and SQL transformations to ER diagrams, lineage visualization, and workflow orchestration.

This extension also ships with Starlake Skills — a set of MCP-based skills that supercharge AI coding assistants like Claude Code, Google Gemini and GitHub Copilot with deep knowledge of the Starlake platform. With these skills, your AI assistant can help you build, debug, and optimize your data pipelines using Starlake best practices.

Works with BigQuery, Snowflake, Redshift, Databricks, DuckDB, and more.

Key Features

Data Ingestion

Load files, automatically infer schemas from your data sources, and generate YAML configurations — no manual schema writing required.

SQL Transformations

Write, preview, and execute SQL transformations with Jinja2 templating support. Dry-run jobs to validate before execution.

Visualization

Generate interactive ER diagrams of table relationships, explore data lineage across your pipeline, and review Access Control Lists — all rendered in a built-in viewer panel.

Worksheet Notebooks

Files under metadata/worksheets/ open as notebooks: mix SQL cells and Markdown cells, run a cell (or all of them) against the Starlake server, and read the results right under the cell. Worksheets stay plain .sql files on disk, so the text editor, the Starlake UI and the CLI keep working on them.

Orchestration

Generate, dry-run, and deploy workflow DAGs for orchestrators like Airflow and Dagster directly from VS Code.

Project Management

Bootstrap new Starlake projects, validate configurations, and manage server connectivity without leaving the editor.

Quick Start

1. Install Prerequisites

Java 17+ is required:

# macOS
brew install openjdk@17

# Linux
sudo apt install openjdk-17-jdk

Starlake CLI:

bash <(curl https://raw.githubusercontent.com/starlake-ai/starlake/master/distrib/setup.sh)

2. Start the Starlake Server

./starlake.sh serve

Listens on port 9900 by default. Use --port PORT to change it.

3. Open Your Project

Open your Starlake project folder in VS Code. The extension activates automatically.

Commands

All commands are available via the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and context menus.

Command	Description
Starlake: New Project	Bootstrap a new Starlake project
Starlake: Validate Project	Validate project configuration
Starlake: Refresh server	Reconnect to the Starlake server
Starlake: Load File	Load a data file for ingestion
Starlake: Infer Schema	Auto-infer schema from data files
Starlake: Run Query	Execute a SQL query
Starlake: Preview Query	Preview transformation results
Starlake: Run Job	Execute a transformation job
Starlake: Dry Run	Validate a job without executing it
Starlake: Table relations	Generate an ER diagram
Starlake: Build Lineage	Visualize data lineage
Starlake: ACLs	View Access Control Lists
Starlake: Generate YML Files from XLS	Convert Excel definitions to YAML
Starlake: Build Excel Files	Export YAML definitions to Excel
Starlake: Open Results Viewer	Open the built-in viewer panel
Starlake: New Worksheet	Create a worksheet notebook in `metadata/worksheets/`
Starlake: Open Worksheet as Notebook	Switch a worksheet from the text editor to the notebook
Starlake: Open Worksheet as SQL Text	Switch a worksheet notebook back to the raw SQL file

Command	Description
Generate	Generate DAGs / workflow files
Dry Run	Dry-run the orchestration logic
Deploy	Deploy the orchestration workflow

Worksheet notebooks

Every .sql file under metadata/worksheets/ opens as a notebook. Cells are delimited with comment markers, so the file remains valid SQL:

-- %% [markdown]
-- # Customer analysis
-- Some *markdown* documentation.
-- %% [sql]
SELECT * FROM sales.customers;

A line matching -- %% (optionally followed by [sql] or [markdown]) starts a new cell; no tag means sql.
Markdown cell lines are stored with a -- prefix; a worksheet that is a single SQL cell is saved without any marker.
Run a cell with the play button or Ctrl+Enter / Shift+Enter; Run All runs cells top to bottom and stops at the first error.
The notebook is saved before running. Results show up to starlake.notebookPageSize rows (default 100).
Use Starlake: Open Worksheet as SQL Text (or right-click the file and choose Open With...) to edit the raw file.
The notebook is wired through a workbench.editorAssociations default for **/worksheets/**/*.sql. Set that key to "default" in your settings if you prefer worksheets to open as plain text.

Extension Settings

Configure the extension in your VS Code settings.json:

Setting	Default	Description
`starlake.serverHost`	`http://127.0.0.1`	Starlake server address
`starlake.serverPort`	`9900`	Starlake server port
`starlake.metadataDir`	`metadata`	Name of the metadata folder
`starlake.cliKey`	—	API key (must match `SL_API_CLI_KEY` env var)
`starlake.starlakeOutput`	—	Path to a log file to display progress from
`starlake.graphvizattrs`	`Primary and Foreign Keys only`	Diagram detail level (`All Attributes` or `Primary and Foreign Keys only`)
`starlake.notebookPageSize`	`100`	Maximum rows shown under a worksheet notebook cell

BigQuery Setup

If you are working with Google BigQuery, authenticate and set your project:

gcloud auth application-default login
gcloud config set core/project MY_PROJECT_ID

Install gcloud via brew install google-cloud-sdk (macOS) or sudo apt install google-cloud-sdk (Linux).

Starlake