Dataform tools - a vscode extension

🎥 Click here for YouTube video with installation steps and demo →

Installs

🎉 Officially recommended by Google[^1]. Though not an officially supported Google product.

VS Code extension for Dataform. Supports both Dataform version 2.x and 3.x

Feature	Description
Compiled Query & Dry run stats	Compiled query with dry run stats in a vertical split
Dependancy graph	Interative dependancy graph with external sources higlighted in distinct colors
Inline diagnostics on `.sqlx` file 🚨	Native LSP like experience with diagnostics being directly put on sqlx file
Preview query results	Preview query results in a table by running the file
Schema code generation	Edit the schema of the model to genrate code that can be used for documentation
Cost estimator 💸	Estimate the cost of running a Tag
Go to definition	Go to definition for source in `$ref{("my_source")}` and javascript blocks in `.sqlx` files
Auto-completion	- Column names of current model - Dependencies and declarations in `${ref("..")}` trigger when `$` character is typed - Dependencies when `"` or `'` is typed inside the config block which has `dependencies` keyword is in the line prefix - `tags` when `"` or `'` is typed inside the config block which has `tags` keyword is in the line prefix
Code actions	Apply dry run suggestions at the speed of thought
Run file(s)/tag(s)	Run file(s)/tag(s), optionally with dependencies/dependents/full refresh using vscode command pallet or compiled query web view
Format using Sqlfluff 🪄	Fromat `.sqlx` files using sqlfluff
BigQuery snippets	Code snippets for generic BigQuery functions taken from vscode-langauge-sql-bigquery extension
BigQuery hover definition provider	Hover definition for commonly used BigQuery functions

Requirements

Dataform cli
```
# requires nodejs
npm i -g @dataform/cli
```
Run dataform compile from the root of your Dataform project to ensure that you are able to use the cli

Install gcloud cli and run

gcloud init

gcloud auth application-default login

 gcloud config set project <project_id> #replace with your gcp project id

To enable formatting using sqlfluff install sqlfluff
```
# install python and run
pip install sqlfluff
```
To enable prettier diagnostics install Error Lens extension [ optional ]

[!NOTE] Trouble installing or looking for a specific customization ? Please see FAQ section, if you are still stuck, please raise an issue here

🎥 Installation on Windows
🎥 Installation and demo on Ubuntu

Features

Compiled query & Dry run stats

compilation

Inline diagnostics errors on `.sqlx` files ❗

diagnostics

Schema code generation

Generate code for schema of the model by editing description to generate documentation code for the columns. See documentation for how to use the generated code

schema_code_gen

Estimate cost of running a Tag

cost_estimator

Go to definition for source in $ref{("my_source")}. Takes you to my_source.sqlx or sources.js at the line where my_source is defined. There is also support for go to definiton from a javascript variable/module from a .sqlx file to js block or .js file where the virable or module declaration exsists

go-to-definition

Autocomplete model, tags, dependencies

Auto completion of declarations in ${ref("..")} trigger when $ character is typed and dependencies and tags in config block when " or ' is typed.

auto-completion

Formatting using sqlfluff

formatting

BigQuery hover definition provider

Hover over BigQuery functions to see their documentation, syntax, and examples making it easier to understand and use them correctly without leaving your editor.

go-to-definition

Commands

Most features can be invoked via command pallet by pressing CTLR + SHIFT + p or CMD + SHIFT + p on mac and searching for the following. These key bindings can also be attached to a keybinding to further streamline your workflow.

Command	Description
`vscode-dataform-tools.showCompiledQueryInWebView`	Show compiled Query in web view
`vscode-dataform-tools.runCurrentFile`	Run current file
`vscode-dataform-tools.runCurrentFileWtDeps`	Run current file with dependencies
`vscode-dataform-tools.runCurrentFileWtDownstreamDeps`	Run current file with dependents
`vscode-dataform-tools.runQuery`	Preview query results
`vscode-dataform-tools.runTag`	Run a tag
`vscode-dataform-tools.runTagWtDeps`	Run a tag with dependencies
`vscode-dataform-tools.runTagWtDownstreamDeps`	Run a tag with dependents
`vscode-dataform-tools.runFilesTagsWtOptions`	Run file(s) / tag(s) with options
`vscode-dataform-tools.dependencyGraphPanel`	Show dependency graph
`vscode-dataform-tools.formatDocument`	Format Document

Known Issues

[ ] Features such as go to definition / dependancy graph might not work with consistantly with ${ref("dataset", "table")} or when it is multiline or a different format works best with ${ref('table_name')} format

TODO

[ ] Add option to include dependents / dependencies when running cost estimator for tag
[ ] Add hover docs for config block elements. e.g. Assertions, type etc
[ ] Handle case where user is not connected to internet or on vpn where network request for dry run cannot be made

[^1]: Link to confirmation of official recommendation by Google: link

Dataform tools

ashishalex