DocCrawler MCP Server

🚀 Simple Setup Install and configure in minutes.

A VS Code extension that automatically configures the DocCrawler MCP server for Claude Code. Crawl entire documentation websites with a single command.

✨ Features

• ✅ Simple Configuration - Add a .mcp.json file
• ✅ Auto-Installs Dependencies - Python package and Playwright installed automatically
• ✅ Works with Claude Code - Seamless integration
• ✅ Cross-Platform - Works on macOS, Windows, Linux
• Crawls entire documentation sites from a single URL
• Extracts clean content (strips navigation, sidebars, footers)
• Supports static sites and Single Page Applications (SPA)
• Returns structured content with titles and full text

📦 Installation

1. Install from VS Code Marketplace: "DocCrawler MCP Server"
2. Create a .mcp.json file in your project directory with the following content:

   {
     "mcpServers": {
       "doc-crawler": {
         "command": "python3",
         "args": ["-m", "doc_crawler"]
       }
     }
   }
   

3. Close Claude Code and reopen it
4. The MCP server will be available and you can start using fetch_docs

🚀 Usage

After restarting Claude Code, open a new chat and use the fetch_docs tool:

fetch_docs(
    url="https://docs.example.com/api",
    mode="auto",
    max_pages=100,
    max_depth=3
)

🔧 How It Works

The extension automatically:

1. ✅ Installs the Python doc-crawler package in your environment
2. ✅ Installs Playwright browsers (for JavaScript-heavy sites)
3. ✅ Verifies everything is working

You need to manually:

1. Create a .mcp.json file in your project directory (see Installation)
2. Restart Claude Code to load the MCP server

Once complete, the fetch_docs tool will be available in Claude Code.

📋 Requirements

• Python 3.10+ must be installed and available in PATH
• VS Code with Claude Code extension installed
• Internet connection (for first-time package installation)

⚠️ Troubleshooting

Python not found?

Make sure Python is installed:

• Mac: brew install python3
• Windows: Download from python.org
• Linux: sudo apt-get install python3 python3-pip

MCP server not showing?

1. Reload VS Code: Cmd+Shift+P → "Developer: Reload Window"
2. Check Extension Host output for errors
3. Verify extension is installed and running

📄 License

MIT

🙏 Credits

Built for the Claude Code community.