Rotor

Rotor is a Visual Studio Code extension for analytical Jupyter notebooks that use SQL cells. It adds a notebook-focused workflow for naming paragraphs, assigning SQL connections, navigating notebook structure, formatting SQL, linting SQL, and viewing query results without leaving the notebook.

Features

• A context-aware Rotor: Paragraphs panel for notebook paragraphs and the full report structure of Motor report.md files.
• Paragraph titles stored in notebook metadata and shown as CodeLens entries.
• Drag paragraphs in Rotor: Paragraphs to reorder notebook cells. Drop onto a paragraph to insert before it, or onto empty space to move to the end. Code, metadata, and outputs are preserved; use the notebook's Undo to revert. A running paragraph must finish before it can be moved.
• SQL connection labels per paragraph, with configurable aliases and dialect mapping.
• One-click execution for the active cell or a selected paragraph.
• A manual Pipeline graph with Waterfall/Flat layouts, an optional native tree, dependency-aware runs and code-change tracking for Stator notebook tasks.
• A Rotor: Output webview for mirrored notebook output.
• A DataFrames tab with bounded pandas/Polars head previews from the active kernel.
• SQL formatting through SQLFluff, scoped to detected SQL cell bodies.
• SQL diagnostics through SQLFluff with configurable severity and ignored rules.
• OpenMetadata-aware SQL completion for configured metadata servers.
• Setup command that inserts a notebook bootstrap cell for required kernel packages.
• Native VS Code Outline navigation for Motor report.md files, including nested layouts, data sources, parameter types, SQL view/query kinds, component types, and each component's query datasource.
• Go to Definition for Motor report references through F12, Peek Definition, or Cmd+Click on macOS / Ctrl+Click on Windows and Linux.
• Find All References for Motor declarations and references through Shift+F12 or the editor context menu.

For a Motor report, use either the standard VS Code Outline view or the dedicated Rotor: Paragraphs panel. The Rotor panel switches automatically: Jupyter notebooks show paragraphs, while recognized Motor reports show data sources, parameters, SQL, templates, and nested layout. Clicking an entry jumps to its declaration. The custom Expand All title action opens every Motor branch; VS Code provides the adjacent Collapse All action. Expanded Motor branches are remembered separately for each report when switching editor tabs and are restored from workspace state after a VS Code reload. While the Rotor Paragraphs panel is visible, moving the editor cursor selects and reveals the deepest enclosing Motor item without moving focus out of the editor. The standard Outline and Go to Symbol in Editor (Ctrl+Shift+O) support both Markdown and Evidence Markdown (emd). Keep the standard Outline sorting set to Position to preserve source order.

Motor definition navigation works in both Markdown and Evidence Markdown (emd). Rotor links component query, template, and params attributes; parameter options.source; dimension default choices; SQL parameter helpers such as in_filter, between_filter, and dimension; and relations after SQL FROM or JOIN. Relation lookup understands report data sources, named SQL blocks, and scoped CTEs, including chained and nested CTEs. Local CTE names take priority while they are in scope. Find All References searches the current report and can be invoked from either a declaration or one of its references.

Requirements

Rotor activates for VS Code Jupyter notebooks. Install and configure the VS Code Jupyter extension separately.

SQL formatting and linting require sqlfluff to be available on your PATH, or configured through rotor.sqlLint.sqlfluffPath.

OpenMetadata completion requires a reachable OpenMetadata server and an API key configured through the Rotor command palette command.

Notebook output

By default, Rotor mirrors cell output into Rotor: Output and automatically collapses the output in the notebook. To keep standard Jupyter output behavior, open VS Code Settings, search for Rotor: Output Mode, and select mirror. Alternatively, add this to your user or workspace settings.json:

"rotor.outputMode": "mirror"

In mirror mode, Rotor leaves notebook output visibility unchanged while continuing to display a copy in Query Result. The other modes are mirror-and-collapse (the default) and mirror-and-clear (clear notebook output after mirroring).

Pipeline

Open Rotor: Open Pipeline, or select the separate Rotor: Pipeline panel. It shows only Stator tasks declared by %%sql task=..., %%task task=... and %%trino_loop task=.... Dependencies are explicit: root=true declares a root and after=orders,rates declares parents. %%sql part_of=orders adds another cell to the same logical task. Its cells always execute together, in order. Other notebook cells are not task nodes.

Install a Stator build providing parse_notebook_content, pipeline_to_dict and the stator-notebook-graph-v1 contract in the selected notebook kernel. The dagster package is not required. Missing/older Stator produces an update message and a Refresh action (Check again in the native tree); ordinary paragraphs and other Rotor features remain available. Run the notebook's stator.setup() cell before executing tasks.

Graph and tree presentations

The default graph shows each task once, including tasks with multiple parents. Arrows represent explicit dependencies, not SQL inference or elapsed time.

• Waterfall places dependency levels from left to right. Unbranched chains share a row; forks and joins get their own lanes.
• Flat places one task per row with compact dependency indentation.
• Click a task to select it; double-click it or use Reveal cell to open its notebook cell. Task details / cells shows its connection, code hash, freshness and every cell of a multipart task.
• Click empty graph space to deselect; dragging the background keeps the selection. Queued tasks have a separate purple status color and legend entry, distinct from running tasks.
• Upstream / Downstream in the highlighting selector emphasize the selected task's lineage without changing the execution plan.
• Use Fit, zoom controls, Ctrl/Cmd+wheel, scroll or drag the background to navigate. Layout, zoom, viewport and selection survive webview recreation for the current notebook.

Select Tree in the graph toolbar, or run Rotor: Show Pipeline Tree, to return to the original native tree. Use its Show Pipeline Graph title action or the command of that name to switch back. The graph/tree choice is remembered per workspace, and Rotor: Open Pipeline opens the selected presentation.

The native tree expands from roots toward downstream tasks. A shared task has one expandable occurrence and leaf references under its other parents. All occurrences share state and each task executes only once in a plan. Clicking an entry reveals its notebook cell; Cells lists the pieces of a multipart task.

Both presentations share the same controller, freshness records, confirmation preview and sequential Jupyter execution. Switching presentations does not refresh the kernel or stop an active queue.

Run actions are manual and preview the complete ordered plan before executing:

• Run Task forces the selected task.
• Run Task with Upstream forces its ancestors and the task.
• Run Task with Downstream forces the task and its descendants.
• Run Changed Tasks and Downstream selects changed, stale and never-run tasks, together with their descendants.
• Run All Pipeline Tasks forces the entire graph.

Missing or stale required parents from other branches are added to the plan and listed in the preview. Execution follows dependency order, even when notebook cell order differs. A failed or interrupted cell stops the queue; descendants are blocked. Native Jupyter execution owns cell outputs, errors and history. Stop Pipeline Queue lets the active cell finish and prevents further cells; use Jupyter's Stop action to interrupt the active query. Editing notebook source, moving/removing cells or changing Pipeline connection settings stops the plan before continuing with a different snapshot. Run without concurrently queuing other SQL cells while Pipeline is binding a cell's connection.

Connections

Pipeline sends every cell's source to Stator, including ordinary %sql gp switches. A switch does not appear as a task but determines the connection for subsequent %%sql tasks. connection=... overrides that connection and part_of=... inherits its owner's connection even if another %sql switch appears between parts. Rotor pins the resolved alias for the native cell run; it does not depend on whichever alias happened to be active in the live kernel, and does not rewrite cell source or notebook metadata.

For cells before the first %sql switch, use connection=... or configure rotor.pipeline.defaultAlias. Custom aliases require rotor.pipeline.profiles to match stator.setup(profiles=...), for example:

"rotor.pipeline.profiles": { "lake": "trino", "gp": "greenplum" },
"rotor.pipeline.defaultAlias": "lake"

An empty profile mapping uses Stator defaults. Setup Python code is not evaluated to discover settings. rotor.connections labels/metadata are display settings and do not select runtime connections.

Freshness and performance

Graph positions are cached by task names and dependencies. Status/code-hash updates reuse coordinates; status-only updates also reuse the existing SVG nodes. Switching Waterfall/Flat, zooming, selecting and highlighting tasks are local UI operations and do not inspect the kernel. The hidden graph performs no layout or state serialization, and its browser context is not retained.

changed compares Stator's code hash against the last successful Pipeline run. stale means an upstream is no longer current, or was rerun after the consumer's last success. never means there is no confirmed result in this session. Multipart tasks are confirmed only after every part succeeds. Records are kept in memory separately for up to eight notebooks. Kernel replacement/restart or a changed Stator partition window clears confirmed results. External execution of a task while Pipeline is visible conservatively invalidates its tracked result and downstream. Changes to database contents, arbitrary Python variables and external files are not inferred; use a force-run action for new data.

There is no automatic execution, typing subscription or polling. Graphs are checked on opening Pipeline, Refresh, before an explicit run, and after saving while the view is visible. Kernel metadata changes also trigger a compatibility check, or stop the remaining queue during a run. Save bursts are coalesced for 150 ms; unchanged source and output-only saves issue no new kernel request. Unsaved changes can be checked with Refresh. Hidden/collapsed views detach notebook/configuration/kernel listeners and start no background inspections. An already dispatched request may finish; its hidden response is discarded. Hiding the view stops any remaining explicit run queue after the active cell finishes and its one-shot binding is cleaned up. Retry and timeout magic parameters remain metadata in this notebook runner; there are no automatic retries or per-task timeout enforcement.

DataFrames

Open Rotor: Open DataFrames, or select DataFrames in Query Result. The list on the left shows existing pandas and Polars DataFrames in the active notebook's Python kernel. Select an entry to inspect its head. Jupyter may ask you to grant Rotor access to the kernel the first time. Rotor uses the public Jupyter extension API; it does not start a kernel, add cells or alter execution history. No Stator changes or additional kernel packages are needed.

The status dot has an accessible label and a tooltip:

• Green: the snapshot is current for the last observed execution.
• Yellow: the snapshot is stale; refresh is pending until the view is visible and the kernel is idle.
• Blue: an inspection is in progress.
• Red: inspection failed; the previous snapshot remains available. Use Refresh to retry, for example after granting kernel access.
• Gray: no preview is loaded, the kernel is unavailable, or a LazyFrame is selected.

There is no polling. The list/selected preview refresh after notebook execution or a manual Refresh. Typing in a cell does not invalidate the snapshot. Execution events are coalesced until 750 ms have passed without a new refresh event. Hidden, collapsed and disposed panels, and the Query Result tab, issue no new inspection requests. An existing request may finish; its result is discarded if the notebook, selection or visibility changed. Rotor never interrupts user code to stop an inspection. Changes made by external clients or background threads may require a manual Refresh.

The default head is 20 rows, with 5/20/50/100 available. Inspection is capped at 50 columns, 500 characters per scalar cell, 200 frames/5,000 namespace entries and a 256 KiB response. Large/nested/object/binary cells use bounded summaries; the interface reports when preview/discovery limits apply. Only eight head snapshots are retained in memory, and incompatible state is cleared on notebook switch or kernel restart. Data and caches are not written into the notebook. Sorting, filters, copy and CSV export operate on the preview only. Polars LazyFrames are listed but never collected or queried for their schema.

Development checks

npm test compiles and runs the Node tests and Python inspector checks (Python 3 is required). Tests of real pandas, Polars and IPython kernels run when those packages are installed in the selected interpreter; set ROTOR_TEST_PYTHON to its executable. Browser tests use the development-only playwright-core package and run when ROTOR_BROWSER_EXECUTABLE points to a Chromium/headless-shell executable. For example, with the Python packages and browser already installed:

ROTOR_TEST_PYTHON=/path/to/python \
ROTOR_BROWSER_EXECUTABLE=/path/to/chrome-headless-shell npm test

Commands

Settings