Starlake for Visual Studio Code

Build data pipelines faster. The Starlake extension brings the full power of the Starlake data integration platform into VS Code — from schema inference and SQL transformations to ER diagrams, lineage visualization, and workflow orchestration.

Works with BigQuery, Snowflake, Redshift, Databricks, DuckDB, and more.

Key Features

Data Ingestion

Load files, automatically infer schemas from your data sources, and generate YAML configurations — no manual schema writing required.

SQL Transformations

Write, preview, and execute SQL transformations with Jinja2 templating support. Dry-run jobs to validate before execution.

Visualization

Generate interactive ER diagrams of table relationships, explore data lineage across your pipeline, and review Access Control Lists — all rendered in a built-in viewer panel.

Orchestration

Generate, dry-run, and deploy workflow DAGs for orchestrators like Airflow and Dagster directly from VS Code.

Project Management

Bootstrap new Starlake projects, validate configurations, and manage server connectivity without leaving the editor.

Quick Start

1. Install Prerequisites

Java 17+ is required:

# macOS
brew install openjdk@17

# Linux
sudo apt install openjdk-17-jdk

Starlake CLI:

bash <(curl https://raw.githubusercontent.com/starlake-ai/starlake/master/distrib/setup.sh)

2. Start the Starlake Server

./starlake.sh serve

Listens on port 9900 by default. Use --port PORT to change it.

3. Open Your Project

Open your Starlake project folder in VS Code. The extension activates automatically.

Commands

All commands are available via the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and context menus.

Command	Description
Starlake: New Project	Bootstrap a new Starlake project
Starlake: Validate Project	Validate project configuration
Starlake: Refresh server	Reconnect to the Starlake server
Starlake: Load File	Load a data file for ingestion
Starlake: Infer Schema	Auto-infer schema from data files
Starlake: Run Query	Execute a SQL query
Starlake: Preview Query	Preview transformation results
Starlake: Run Job	Execute a transformation job
Starlake: Dry Run	Validate a job without executing it
Starlake: Table relations	Generate an ER diagram
Starlake: Build Lineage	Visualize data lineage
Starlake: ACLs	View Access Control Lists
Starlake: Generate YML Files from XLS	Convert Excel definitions to YAML
Starlake: Build Excel Files	Export YAML definitions to Excel
Starlake: Open Results Viewer	Open the built-in viewer panel

Command	Description
Generate	Generate DAGs / workflow files
Dry Run	Dry-run the orchestration logic
Deploy	Deploy the orchestration workflow

Extension Settings

Configure the extension in your VS Code settings.json:

Setting	Default	Description
`starlake.serverHost`	`http://127.0.0.1`	Starlake server address
`starlake.serverPort`	`9900`	Starlake server port
`starlake.metadataDir`	`metadata`	Name of the metadata folder
`starlake.cliKey`	—	API key (must match `SL_API_CLI_KEY` env var)
`starlake.starlakeOutput`	—	Path to a log file to display progress from
`starlake.graphvizattrs`	`Primary and Foreign Keys only`	Diagram detail level (`All Attributes` or `Primary and Foreign Keys only`)

BigQuery Setup

If you are working with Google BigQuery, authenticate and set your project:

gcloud auth application-default login
gcloud config set core/project MY_PROJECT_ID

Install gcloud via brew install google-cloud-sdk (macOS) or sudo apt install google-cloud-sdk (Linux).

Starlake