Notebook Cell-Level Documentation AI

AI-powered cell-by-cell documentation for Jupyter and Databricks notebooks using GitHub Copilot.

Features

• AI-Generated Explanations - Generate clear explanations for each code cell using GitHub Copilot
• Side-by-Side Editor - View code and explanations together in an intuitive interface
• Enrich Explanation - Provide additional context and let AI regenerate richer explanations
• Business Context - Add business purpose per cell, included in final documentation
• Markdown Preview/Edit - GitHub-style preview/edit toggle for all text fields
• Notebook Overview - Edit or rewrite a high-level summary with AI assistance
• AI-Powered Reformat - Improve grammar and clarity of your notes with one click
• Change Detection - Detect when code has changed since documentation was created
• Comprehensive Documentation - Generate polished markdown with table of contents
• Multi-Format Support - Jupyter (.ipynb) and Databricks (.py, .sql) notebooks
• Master Documentation - Generate a unified master_documentation.md from all documented notebooks with a 2-step wizard (select → organize into sections)
• Section Organization - Group notebooks into named sections (e.g., Ingestion, Transformation) with drag-and-drop reordering
• Auto Dependency Detection - Automatically computes "Depends On" relationships by matching source tables to target tables
• Bulk Generation - Multi-select notebooks to generate documentation for all at once
• Customizable Prompts - Configure prompt templates via .notebook-cell-doc/prompts.yaml

Who Is This For?

• Data Scientists documenting analysis and ML workflows
• Data Engineers documenting ETL pipelines, Spark jobs, and data transformations
• Analytics Engineers documenting SQL-heavy notebooks and data models
• ML Engineers documenting training, evaluation, and inference notebooks

Requirements

• VS Code 1.84+
• GitHub Copilot Chat extension installed and activated
• Valid GitHub Copilot license

Installation

From VSIX (Local)

code --install-extension notebook-cell-doc-ai-0.1.0.vsix

From Marketplace

1. Open VS Code
2. Go to Extensions (Ctrl+Shift+X)
3. Search for "Notebook Cell-Level Documentation AI"
4. Click Install

Quick Start

1. Open any Jupyter notebook (.ipynb) or Databricks notebook (.py / .sql)
2. Right-click and select "Notebook Cell-Level Documentation AI: Generate Documentation - Cell by Cell"
3. Wait for AI to generate explanations
4. Enrich explanations with additional context or add business context
5. Click "Save & Generate Final Documentation"
6. Right-click the documentation/ folder and select "Generate Master Documentation" to create a unified document

Usage

Generate Documentation

Option 1: Context Menu

• Right-click on any notebook file in the Explorer
• Select "Notebook Cell-Level Documentation AI: Generate Documentation - Cell by Cell"

Option 2: Command Palette

• Open Command Palette (Ctrl+Shift+P)
• Type "Notebook Cell-Level Documentation AI: Generate Documentation"

Option 3: Keyboard Shortcut

• Press Ctrl+Shift+D (Windows/Linux) or Cmd+Shift+D (Mac)

Enrich Explanations

1. In the documentation editor, find a cell
2. Click "Enrich Explanation"
3. Enter additional context in the modal popup
4. AI regenerates the explanation incorporating your context

Business Context

Add a Business Context field to any cell describing its purpose (e.g., "Data ingestion from Delta Lake"). This is included in the final documentation.

Notebook Overview

• Markdown cells from the notebook automatically pre-populate the description
• Edit the description and click "Rewrite with AI" to format and improve it

Save Documentation

• Save - Saves the documentation file only
• Save & Generate Final Documentation - Saves and creates comprehensive markdown

Master Documentation

Generate a unified document covering all documented notebooks in a project:

1. Right-click the documentation/ folder in the Explorer
2. Select "Generate Master Documentation"
3. Step 1 - Select: Choose which notebooks to include
4. Step 2 - Organize: Drag notebooks into named sections (e.g., Ingestion, Transformation, Reporting)
5. Click "Generate Master Documentation" to produce master_documentation.md

The generated document includes:

• Notebooks grouped under section headings with purpose, description, source/target tables
• Automatic "Depends On" relationships computed from table references
• A summary table with all notebooks, sections, and dependencies

Output Files

Documentation is saved to a documentation/ folder in your project:

your-project/
├── notebook.ipynb
└── documentation/
    ├── master_documentation.md    # Unified master document (all notebooks)
    └── notebook/
        ├── cell_by_cell.yaml      # Detailed cell documentation
        └── final_documentation.md # Comprehensive markdown per notebook

Configuration

Extension Settings

Custom Prompts

To customize AI prompt templates for your team:

1. Open Command Palette (Ctrl+Shift+P)
2. Run "Notebook Cell-Level Documentation AI: Create Prompt Config"
3. Edit .notebook-cell-doc/prompts.yaml in your workspace

Commands

Keyboard Shortcuts

Privacy

• Does not collect user data
• Does not transmit notebooks to external servers (except GitHub Copilot)
• Uses GitHub Copilot's existing authentication
• Stores documentation locally in your workspace

Troubleshooting

"GitHub Copilot license is required"

1. Ensure GitHub Copilot Chat extension is installed
2. Verify you have an active Copilot subscription
3. Sign in to GitHub in VS Code

"No Copilot models available"

1. Sign out and sign back into GitHub
2. Check your Copilot subscription status
3. Restart VS Code

Documentation not saving

1. Check file permissions in the workspace
2. Verify disk space availability
3. Ensure output folder is not read-only

License

MIT License - See LICENSE for details.

For issues or feedback, email sraghu1993@gmail.com.