NL2Playwright-MCP

Turn plain English into writing.txt DSL and run via Playwright MCP.

What It Does

NL2Playwright-MCP is a VS Code extension that transforms natural language descriptions into structured test automation code. It bridges the gap between human-readable test requirements and executable browser automation.

Key Features:

Convert plain English → structured writing.txt DSL v2
NEW: Enhanced Test Spec DSL with comprehensive MCP recording instructions
NEW: Cucumber DSL with complete BDD project setup guidance
NEW: Excel Test Design generator with AMS2-2 compatible CSV format
Execute tests via Model Context Protocol (MCP) with Playwright
Smart UI with color feedback (buttons turn blue after generation)
LLM-enhanced generation with free/paid model support
Built-in validation with real-time error checking
Support for complex scenarios including email flows and stealth browsing

Quickstart

🔒 PRIVATE ACCESS: This extension is currently in private development mode. Public release will be announced soon.

Install the Extension

Current Installation Methods:

# Download and install VSIX manually
# 1. Download the .vsix file from releases
# 2. Install via: code --install-extension nl2playwright-mcp-private-0.1.0.vsix

# Or build from source (recommended for development)
git clone https://github.com/martin-ai-library/NL2Playwright-MCP.git
cd NL2Playwright-MCP
npm install
npm run build
npm run package  # Creates .vsix file
code --install-extension nl2playwright-mcp-private-0.1.0.vsix

Future Public Release:

# Will be available when public release is ready
code --install-extension martind.nl2playwright-mcp

Set Up MCP (Optional but Recommended)

// Add to your claude.code.json
{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@microsoft/mcp-playwright"]
    }
  }
}

Write Natural Language Test

Go to the EWFS registration page and sign up as a new operator. 
Fill out the company name as "Test Company LLC", use a random 9-digit TIN, 
and set the address to "123 Main Street, Phoenix, Arizona 85001". 
Use a unique Yopmail email address and complete the verification process.

Open NL2Playwright Panel
- Open VS Code Command Palette (Ctrl+Shift+P)
- Run: NL2Playwright: Show Panel
- Or click the NL2Playwright icon in the sidebar
Generate Test Content
- Enter your natural language test description
- Choose generation format:
  - Test Spec DSL: Comprehensive MCP recording instructions
  - Cucumber DSL: Complete BDD project setup with step definitions
  - Excel Test Design: AMS2-2 compatible CSV test cases
  - Feature File: Simple Gherkin feature files
  - Playwright: Direct TypeScript test code
  - MCP: Raw MCP action sequences
Smart UI Features
- Buttons turn blue after successful generation
- Colors reset to gray when input changes (indicating new generation needed)
- Model picker shows current LLM (Default, Claude, GPT-4o, etc.)
Execute Test
- Copy generated content to appropriate files
- Run via MCP server or traditional Playwright
- Review artifacts and results
View Results
- Check the output panel for execution logs
- Review screenshots and traces in configured directories

🤖 LLM Models (Free vs Paid)

NEW: NL2Playwright MCP now supports both traditional deterministic generation and AI-enhanced generation using various LLM models.

Model Categories

🆓 FREE MODELS (Local/Self-hosted)

Llama 3 8B (Ollama) - Fast local development
Llama 3 70B (Self-hosted) - High-quality reasoning
Gemma 2 9B (Ollama) - Concise DSL generation
Mistral Nemo (Ollama) - Fast code generation
DeepSeek V3 (Self-hosted) - Advanced reasoning
Qwen 2.5 14B (Ollama) - Code and multilingual support

💳 PAID MODELS (API Required)

GPT-4o (OpenAI) - Most capable for complex DSL
Claude 3.5 Sonnet (Anthropic) - Excellent for test scenarios
Mistral Large 2 (Mistral API) - Code generation specialist
Gemini 2.0 Flash Experimental (Google) - Massive context window

Quick Setup

1. Environment Variables

# Set API keys for paid models (optional)
export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"
export MISTRAL_API_KEY="your-mistral-api-key"
export COHERE_API_KEY="your-cohere-api-key"
export GOOGLE_API_KEY="your-google-api-key"

2. Install Ollama (for free local models)

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Pull models you want to use
ollama pull llama3:8b
ollama pull gemma2:9b
ollama pull mistral-nemo

3. Configure VS Code Settings

{
  "nl2pw.llm.allowNet": true,           // Enable paid API models
  "nl2pw.llm.selectedModelId": "default", // or specific model ID
  "nl2pw.llm.routerMode": "manual",     // or "auto"
  "nl2pw.llm.catalogFilter": "all",     // "free", "paid", or "all"
  "nl2pw.llm.maxTokens": 4096,
  "nl2pw.llm.temperature": 0.2
}

Using LLM Models

Option 1: UI Toggle

Click the 🟢 Default button in the extension header
Select from grouped Free/Paid model picker
Toggle shows current model: 🟢 Default → 🤖 GPT-4o

Option 2: Commands

Ctrl+Shift+P → "NL2Playwright: Pick LLM Model"
Ctrl+Shift+P → "NL2Playwright: Toggle Router Mode"

Option 3: Status Bar

Click NL2PW: DEFAULT in status bar
Shows current model and network status
Quick access to model picker and settings

Router Modes

Manual Mode (Default)

Uses your selected model for all generation
Falls back to deterministic pipeline if model unavailable
Consistent behavior across all tasks

Auto Mode (Intelligent)

Automatically selects best model for each task:
- Complex DSL generation → GPT-4o or Claude 3.5 Sonnet
- Smart locators → Local models (faster)
- Documentation → Claude or Gemini (better reasoning)
- Simple tasks → Free local models

Network Settings

Network Disabled (`allowNet: false`)

Only free local models available
Paid models shown but grayed out
Completely offline operation
Perfect for secure environments

Network Enabled (`allowNet: true`)

All models available (with valid API keys)
Automatic fallback to free models if APIs fail
Best generation quality

Generated Content Headers

All LLM-generated content includes metadata:

# Generated by: Claude 3.5 Sonnet
# Rationale: Auto-selected for complex DSL generation
# Timestamp: 2024-01-15T10:30:00.000Z
# 
Project: Login Test
...

🎯 Enhanced Output Formats

Test Spec DSL (Enhanced)

NEW: Comprehensive MCP-focused recording instructions that provide step-by-step guidance for creating robust Playwright tests.

Features:

Complete MCP recorder setup instructions
Detailed locator strategies (role, label, placeholder, testid, css)
Security guidelines (password redaction, no logging)
Guard rails for web-first assertions
Screenshot and artifact management
Project configuration templates

Use Case: When you need detailed recording instructions for manual test creation or MCP server integration.

Cucumber DSL (Enhanced)

NEW: Complete BDD project setup with comprehensive Cucumber-Playwright integration guidance.

Features:

Full project structure generation
Step definitions implementation code
Hooks and world setup
Feature file templates with scenarios
Configuration files (cucumber.config.js, playwright.config.ts)
MCP integration workflow
Execution and reporting commands

Use Case: When setting up a new Cucumber-Playwright BDD testing framework or converting existing tests to BDD structure.

Excel Test Design (New)

NEW: AMS2-2 compatible CSV format for structured test case documentation.

Features:

Professional test case format with columns: Test Case ID, Scenario, Preconditions, Steps, Data, Expected Result, Pass/Fail, Notes
Comprehensive test coverage (positive, negative, edge cases)
Security and performance considerations
Realistic test data examples
S1-PROJECT-VERSION-### ID format

Use Case: When you need formal test documentation, test case management, or QA review processes.

Smart UI Features

Color Feedback: Buttons change from gray → blue after generation → gray when input changes
2x2 Layout: Clean interface with Test Spec DSL, Cucumber DSL, Excel Test Design, and Feature File
Model Status: Always visible current LLM model (Default, Claude, GPT-4o, etc.)
Input Change Detection: Automatically resets button colors when test description is modified

DSL v2 Reference

The writing.txt DSL v2 format provides a human-readable way to define test automation scenarios.

Basic Structure

Project: My Test Project

Meta:
  Author: Martin D
  Created: 2024-01-15T10:30:00Z
  Tags: signup,email,verification

Config:
  BrowserChannel: chrome|chromium|firefox|webkit
  Headless: true|false
  StartMaximized: true|false
  StealthArgs: on|off
  Humanize: on|off

Env:
  AppURL: https://example.com
  MailURL: https://yopmail.com

Test: Test Description

Scope:
  - PublicSignupOnly
  - EmailFlow
  - NegativeDuplicate

Data:
  CompanyName: Test Co LLC
  Email: generate:yopmail:unique
  TIN: generate:unique:9digits
  FirstName: auto:Tesla<2CharRandom>
  LastName: seq:One|Two|Three

Steps:
  - goto: AppURL
  - click: SignUpButton
  - fill: Email -> Data.Email
  - assertVisible: /success/i

Data Generation Patterns

generate:unique:9digits - Random 9-digit number
generate:yopmail:unique - Unique Yopmail email address
auto:Base<2CharRandom> - Base text with random characters
seq:Option1|Option2|Option3 - Sequential selection from list
<last:FieldName> - Reference to previously generated value
<reuse:last:FieldName> - Reuse value from previous test

Locator Strategy

Locators follow a strategy order for maximum reliability:

role - role:button[name="Submit"]
label - label:"Submit Form"
placeholder - placeholder:"Enter email"
testid - testid:submit-btn
css - css:#submit-button

Step Primitives

goto: URL - Navigate to page
click: LocatorKey - Click element
fill: LocatorKey -> Value - Fill input field
select: LocatorKey -> Option - Select dropdown option
assertVisible: Text|Pattern - Assert element visibility
waitFor: LocatorKey timeout=ms - Wait for element
screenshot: Label -> Path - Take screenshot
email.pollLatest: Mailbox timeout=ms - Poll for email
email.openLink: regex=/pattern/ - Open link from email
password.set: value=Password - Set password
context.new: clean|humanized - Create new browser context

Examples

Project: Login Test
Test: Basic user authentication
Steps:
  - goto: https://app.example.com/login
  - fill: Username -> user@example.com
  - fill: Password -> SecurePass123
  - click: LoginButton
  - assertVisible: Dashboard

Registration with Email Verification

Project: Registration Flow
Test: Complete signup with email verification
Scope:
  - EmailFlow
Data:
  Email: generate:yopmail:unique
  Password: SecurePass123
Steps:
  - goto: https://app.example.com/signup
  - fill: Email -> Data.Email
  - fill: Password -> Data.Password
  - click: RegisterButton
  - assertVisible: /check your email/i
  - context.new: clean
  - goto: https://yopmail.com
  - email.pollLatest: <parse:mailboxFromLastEmail> timeout=60000
  - email.openLink: regex=/verify[^\s]+/
  - assertVisible: /account verified/i

Negative Test (Duplicate Registration)

Project: Duplicate Prevention
Test: Reject duplicate email registration
Scope:
  - NegativeDuplicate
Data:
  Email: <reuse:last:Email>
Steps:
  - goto: https://app.example.com/signup
  - fill: Email -> Data.Email
  - click: RegisterButton
  - assertVisible: /already exists|duplicate/i

Configuration Options

Browser Settings

BrowserChannel: chrome, chromium, firefox, webkit
Headless: Run without UI (true/false)
StartMaximized: Start browser maximized
StealthArgs: Anti-bot detection prevention
Humanize: Human-like interaction patterns

Execution Modes

MCP: Execute via Model Context Protocol (recommended)
Playwright: Generate and run traditional Playwright tests

Artifacts

Screenshots: Capture on failure, always, or never
Traces: Browser interaction recordings
Videos: Full test execution videos

Known Issues

Email Polling: Yopmail iframe detection requires specific handling
Stealth Mode: Some sites may still detect automation despite stealth args
Dynamic Selectors: Complex dynamic content may need manual locator refinement
MCP Dependency: Full MCP functionality requires @microsoft/mcp-playwright setup

Troubleshooting

Extension Not Loading

Check VS Code version (requires 1.90.0+)
Verify extension is enabled in Extensions panel
Check Output panel for error messages

Conversion Failures

Ensure text selection is not empty
Check natural language is descriptive enough
Review Output panel for parsing errors

Test Execution Issues

Verify Playwright is installed: npx playwright install
Check browser compatibility
Review generated locators for accuracy

MCP Connection Problems

Ensure claude.code.json is properly configured
Verify MCP server is running
Check network connectivity

💖 Support the Project

Love NL2Playwright MCP? Support its development!

Your support helps maintain and improve this extension, add new features, and provide better documentation.

Every coffee ☕ helps fuel more awesome features!

Thank you for supporting open source development! 🙏

👨‍💻 Author

Martin D — QA Team Lead & Automation Specialist
GitHub: MartinGitLibrary

Specialized in test automation frameworks, browser testing, and developer productivity tools. Created NL2Playwright-MCP to bridge the gap between business requirements and technical test implementation.

Contributing

This project welcomes contributions! Please:

Fork the repository
Create a feature branch
Follow the existing code style
Add tests for new functionality
Update documentation as needed
Submit a pull request

License

MIT License - see LICENSE file for details.

Changelog

v0.2.0 (Enhanced UI & Formats Release)

NEW: Enhanced Test Spec DSL with comprehensive MCP recording instructions
NEW: Cucumber DSL with complete BDD project setup guidance
NEW: Excel Test Design generator with AMS2-2 compatible CSV format
NEW: Smart UI with color feedback system (gray → blue → gray on input change)
NEW: Clean 2x2 button layout (removed Smart Locators, Test Docs, API Tests)
ENHANCED: MCP-focused converters for detailed project setup instructions
IMPROVED: Better fallback handling and error management
IMPROVED: Security-aware generation with password redaction
LLM catalog system with free/paid model support
Model picker UI with grouped Free/Paid options
Comprehensive debugging and logging system

v0.1.0 (Initial Release)

Natural language to writing.txt DSL conversion
MCP integration with Playwright execution
EWFS example test scenarios
VS Code extension with commands and validation
Support for email flows and stealth browsing
Comprehensive locator strategy system

NL2Playwright MCP

Martin D

NL2Playwright-MCP

What It Does

Quickstart

🤖 LLM Models (Free vs Paid)

Model Categories

🆓 FREE MODELS (Local/Self-hosted)

💳 PAID MODELS (API Required)

Quick Setup

1. Environment Variables

2. Install Ollama (for free local models)

3. Configure VS Code Settings

Using LLM Models

Option 1: UI Toggle

Option 2: Commands

Option 3: Status Bar

Router Modes

Manual Mode (Default)

Auto Mode (Intelligent)

Network Settings

Network Disabled (allowNet: false)

Network Enabled (allowNet: true)

Generated Content Headers

🎯 Enhanced Output Formats

Test Spec DSL (Enhanced)

Cucumber DSL (Enhanced)

Excel Test Design (New)

Smart UI Features

DSL v2 Reference

Basic Structure

Data Generation Patterns

Locator Strategy

Step Primitives

Examples

Simple Login Test

Registration with Email Verification

Negative Test (Duplicate Registration)

Configuration Options

Browser Settings

Execution Modes

Artifacts

Known Issues

Troubleshooting

Extension Not Loading

Conversion Failures

Test Execution Issues

MCP Connection Problems

💖 Support the Project

👨‍💻 Author

Contributing

License

Changelog

v0.2.0 (Enhanced UI & Formats Release)

v0.1.0 (Initial Release)

Network Disabled (`allowNet: false`)

Network Enabled (`allowNet: true`)