UI for Spark Declarative Pipeline (SDP)

Published VS Code extension to visualize Apache Spark Declarative Pipeline (SDP) YAML files.

Features

• Visualize pipeline DAGs from YAML, Python, and SQL files
• Click nodes to open source code directly in editor
• Preview code snippets with syntax highlighting in sidebar
• Live updates on file save

Installation

You can install the extension from the VSIX package or the VS Code Marketplace (if published there).

From VSIX

1. Download the latest .vsix file from the releases or build it locally.
2. In VS Code, run Extensions: Install from VSIX... and select the file.

From Marketplace

1. Search for "UI for Spark Declarative Pipeline" in the Extensions view.
2. Click Install.

Usage

1. Open a workspace containing a pipeline.yml, Python, or SQL pipeline file.
2. Click the SDP Pipeline Visualizer icon in the Activity Bar.
3. Select a pipeline from the automatically listed files in the sidebar.
4. View the pipeline DAG and entity details in the sidebar and webview.

Developer

Developed by Gergely Szécsényi

License

BSD 2-Clause License — see LICENSE for details.