Behave Test Runner

A VS Code extension for discovering, running, and debugging Behave and pytest-bdd tests with full integration into the VS Code testing framework.

Features

🔍 Automatic Test Discovery: Automatically finds and parses .feature files in your workspace
▶️ Run Tests: Run individual scenarios or entire feature files with a single click
🐛 Debug Support: Debug scenarios with full VS Code debugging capabilities
📊 Test Explorer Integration: View and manage tests in the VS Code Test Explorer with organized hierarchy
🎯 CodeLens Support: Run/debug scenarios directly from the editor with inline buttons
🏷️ Tag-based Organization: Automatically groups scenarios by tags for better organization
📋 Scenario Outline Support: Full support for scenario outlines with individual example tracking
🔢 Smart Example Naming: Clean, numbered examples (1:, 2:, 3:) instead of verbose names
📏 Long Column Names Handling: Intelligently truncates long table column names for readability
⚙️ Configurable: Customize behave command, working directory, and more
🔄 Auto-refresh: Automatically updates when feature files change
🚀 Parallel Execution: Run tests in parallel for faster execution
🤝 Extension Compatibility: Designed to work seamlessly with Cucumber (Gherkin) Full Support
🔧 Multi-Framework Support: Support for both Behave and pytest-bdd frameworks with automatic detection

Installation

From VSIX

Download the latest .vsix file from the releases page
In VS Code, go to Extensions (Ctrl+Shift+X)
Click the "..." menu and select "Install from VSIX..."
Select the downloaded file

From Source

git clone https://github.com/your-username/behave-test-runner.git
cd behave-test-runner
npm install
npm run compile
npm run package

Usage

Prerequisites

Python with your preferred BDD framework installed:
- Behave: pip install behave
- pytest-bdd: pip install pytest-bdd
VS Code 1.99.0 or later

Basic Usage

Open a workspace containing .feature files
Framework Detection: The extension automatically detects your BDD framework (Behave or pytest-bdd) based on configuration files
Discover tests by running the "Behave: Discover Tests" command
View tests in the Test Explorer (Ctrl+Shift+P → "Testing: Focus on Test Explorer View")
Run tests by clicking the play button next to any test
Debug tests by clicking the debug button next to any test

Test Organization/Test View

The extension automatically organizes your tests in the Test Explorer with multiple organization strategies:

Default: Feature-Based (Hierarchical) Organization

The extension uses Feature-Based (Hierarchical) Organization as the default strategy:

Feature Files: Top-level containers for your feature files
Scenarios: Nested as children under their respective feature files
Scenario Outlines: Special groups for scenario outlines with their examples
Individual Scenarios: Regular scenarios and scenario outline examples

Note: The default organization strategy is set in the extension code and cannot be changed via configuration. You can switch between strategies using the context menu in the Test Explorer.

Switching views

switching views via test explorer

Alternative Organization Strategies

Tag-Based Organization: Groups scenarios by their tags (e.g., @smoke, @regression)
File-Based Organization: Groups scenarios by their file paths
Scenario Type Organization: Groups by scenario type (regular vs outline)
Flat Organization: All scenarios at the same level

Switch Organization: Right-click in the Test Explorer → "Organization Strategy" → Choose your preferred organization strategy

The extension provides enhanced support for scenario outlines:

Clean Example Names: Examples are named as "1:", "2:", "3:" instead of "Example 1", "Example 2"
Long Column Name Handling: Column names longer than 15 characters are truncated for readability
Multiple Outlines: Support for multiple scenario outlines in the same feature file
Sorted Examples: Examples are automatically sorted by their number
Organized Hierarchy: Scenario outline examples are grouped under their parent outline

Extended support for scenario outline

Execute test from scenario outline level or example
Hierachichal view of scenario outline examples for easy navigation

CodeLens Features

When you open a .feature file, you'll see inline buttons above each scenario:

▶️ Run Scenario: Runs the specific scenario
🐛 Debug Scenario: Debugs the specific scenario

Code lens inline buttons

Feature inline buttons [Run Feature File]| Run with <@tags-available> | ...
Play button on feature level will execute feature and update test explorer on test outcome
Play button scenario level will execute an individual scenario

Note: Right click on play button will give additional options like debugging test

Click on any step in a .feature file and press F12 (or right-click → "Go to Definition") to jump to the matching Python @given/@when/@then/@step decorator.

Supports both styles:

behave: @given('I have <count> users')
pytest-bdd: @given(parsers.parse('I enter {name}')) and @then(parsers.re(r'^count is (\d+)$'))

By default the extension searches features/steps/**/*.py, tests/**/*.py, and tests/steps/**/*.py. Override via behaveTestRunner.stepDefinitionPaths:

{
  "behaveTestRunner.stepDefinitionPaths": [
    "src/acceptance/steps/**/*.py",
    "tests/bdd/steps/**/*.py"
  ]
}

If you prefer the Cucumber (Gherkin) Full Support extension's navigation, disable ours to avoid duplicate results:

{ "behaveTestRunner.enableStepDefinitionNavigation": false }

Recommended companion settings (Cucumber Full Support)

This extension is designed to coexist with Cucumber (Gherkin) Full Support (auto-installed via extensionPack). To get its syntax highlighting and autocomplete on Python step defs, add to your workspace .vscode/settings.json:

{
  "cucumberautocomplete.steps": [
    "features/steps/**/*.py",
    "tests/**/*.py"
  ],
  "cucumberautocomplete.syncfeatures": "features/**/*.feature",
  "cucumberautocomplete.strictGherkinCompletion": true
}

Commands

Command	Description
`Behave: Discover Tests`	Manually discover and load test files
`Behave: Refresh Tests`	Refresh and reload all test files
`Behave: Run All Tests`	Run all tests in the workspace
`Behave: Run All Tests in Parallel`	Run all tests in parallel for faster execution
`Behave: Run Scenario`	Run a specific scenario (used by CodeLens)
`Behave: Debug Scenario`	Debug a specific scenario (used by CodeLens)
`Behave: Run Feature File`	Run an entire feature file
`Behave: Run Feature File with Tags`	Run a feature file with specific tags

Configuration

Extension Settings

Setting	Default	Description
`behaveTestRunner.behaveCommand`	`"behave"`	Command to run Behave tests (auto-detects Python if needed)
`behaveTestRunner.workingDirectory`	`""`	Working directory for test execution (empty = workspace root)
`behaveTestRunner.autoDiscoverTests`	`true`	Automatically discover tests on startup
`behaveTestRunner.testFilePattern`	`"*/.feature"`	Glob pattern for test file discovery
`behaveTestRunner.enableCodeLens`	`true`	Enable CodeLens buttons on feature files (disable if conflicting with other extensions)
`behaveTestRunner.enableStepDefinitionNavigation`	`true`	Enable "Go to Definition" from a Gherkin step to its Python `@given`/`@when`/`@then`
`behaveTestRunner.stepDefinitionPaths`	see below	Globs for Python files containing step definitions (used by step navigation)
`behaveTestRunner.parallelExecution`	`false`	Enable parallel execution of feature files
`behaveTestRunner.maxParallelProcesses`	`4`	Maximum number of parallel processes for test execution
`behaveTestRunner.outputFormat`	`"pretty"`	Output format for behave test results (pretty, plain, json, junit, progress, steps)
`behaveTestRunner.tags`	`""`	Default tags to filter tests (e.g., '@smoke,@regression')
`behaveTestRunner.dryRun`	`false`	Enable dry run mode to see what tests would be executed without running them

Intelligent Command Detection

The extension automatically detects the correct Python command for your system:

macOS/Linux: Tries python3, python, then py
Windows: Tries py, python, then python3
Fallback: Uses python3 -m behave if direct behave command is not available

You can override this by setting a specific command in your configuration.

Example Configuration

{
  "behaveTestRunner.behaveCommand": "python3 -m behave",
  "behaveTestRunner.workingDirectory": "${workspaceFolder}/tests",
  "behaveTestRunner.autoDiscoverTests": true,
  "behaveTestRunner.testFilePattern": "**/features/**/*.feature",
  "behaveTestRunner.enableCodeLens": true,
  "behaveTestRunner.enableTestExplorer": true,
  "behaveTestRunner.priority": "normal",
  "behaveTestRunner.parallelExecution": false,
  "behaveTestRunner.maxParallelProcesses": 4,
  "behaveTestRunner.outputFormat": "pretty",
  "behaveTestRunner.tags": "@smoke,@regression",
  "behaveTestRunner.dryRun": false
}

Example Feature Files

Basic Scenario Outline:

@feature @login
Feature: User Login

  @smoke @critical
  Scenario Outline: Login with different credentials
    Given I am on the login page
    When I enter username "<username>" and password "<password>"
    And I click the login button
    Then I should see "<expected_result>"

    Examples:
      | username | password | expected_result |
      | admin    | admin123 | dashboard       |
      | user     | user123  | dashboard       |
      | invalid  | wrong    | error message   |

Multiple Scenario Outlines:

@feature @api
Feature: API Testing

  @api @validation
  Scenario Outline: Data validation with various inputs
    Given I have a valid API endpoint
    When I send "<data_type>" data with value "<input_value>"
    Then the response should contain "<validation_result>"

    Examples:
      | data_type | input_value | validation_result |
      | string    | hello       | valid             |
      | number    | 42          | valid             |
      | email     | test@test   | invalid           |

  @performance @load
  Scenario Outline: Load testing with different user loads
    Given I have configured the load testing environment
    When I simulate "<concurrent_users>" concurrent users
    Then the system should handle the load successfully

    Examples:
      | concurrent_users |
      | 10               |
      | 50               |
      | 100              |

In the Test Explorer, these will appear as:

Feature-Based (Hierarchical) Organization (Default):

Feature: User Login
- Scenario Outline: Login with different credentials
  - 1: Login with different credentials - username: admin, password: admin123, expected_result: dashboard
  - 2: Login with different credentials - username: user, password: user123, expected_result: dashboard
  - 3: Login with different credentials - username: invalid, password: wrong, expected_result: error message

Tag-Based Organization:

@smoke (group)
- 1: Login with different credentials - username: admin, password: admin123, expected_result: dashboard
- 2: Login with different credentials - username: user, password: user123, expected_result: dashboard

Extension Compatibility

This extension is designed to work alongside other VS Code Gherkin extensions.

Recommended Configuration with Cucumber (Gherkin) Full Support

Note: This extension is developed and tested with Cucumber (Gherkin) Full Support extension. Since you're using Cucumber (Gherkin) Full Support for syntax highlighting and autocomplete, this extension focuses on test execution and discovery. The extensions complement each other perfectly:

{
  "behaveTestRunner.enableCodeLens": true,
  "behaveTestRunner.enableTestExplorer": true,
  "behaveTestRunner.priority": "low",
  "behaveTestRunner.defaultOrganizationStrategy": "FeatureBasedOrganization"
}

Why this works well:

Cucumber Extension: Handles syntax highlighting, autocomplete, and Gherkin language support
Behave Test Runner: Handles test discovery, execution, and Test Explorer integration
No Conflicts: Each extension has its own domain of responsibility

Extension Priority

The priority setting determines which extension takes precedence when multiple extensions can handle the same file type:

"low": Let other extensions handle .feature files first
"normal": Standard priority
"high": Take precedence over other extensions

Framework Selection Best Practices

⚠️ Important: A project should use only one BDD framework (either behave or pytest-bdd) to avoid conflicts.

Automatic Detection Priority

Explicit Configuration: Framework with configuration files takes precedence
- behave.ini, .behaverc → behave
- pytest.ini with pytest-bdd → pytest-bdd
File Pattern Fallback: When no explicit config found
- .feature files + Python test files → defaults to behave
Default: Always behave (for backward compatibility)

Conflict Prevention

The extension will warn you if:

✅ Multiple framework configs detected: Remove unused framework configuration
✅ Ambiguous file patterns: Add explicit framework configuration
✅ Mixed test files: Choose one framework and organize accordingly

Manual Override

Always use manual selection for complex projects:

{
  "behaveTestRunner.framework": "behave",           // Explicit selection
  "behaveTestRunner.autoDetectFramework": false     // Disable auto-detection
}

Configuration

The extension supports various configuration options for customizing behavior:

Framework Selection

The extension automatically detects your BDD framework, but you can also configure it manually:

{
  "behaveTestRunner.framework": "behave",              // "behave" or "pytest-bdd"
  "behaveTestRunner.autoDetectFramework": true,        // Enable automatic framework detection
  "behaveTestRunner.behaveCommand": "behave",          // Command for Behave framework
  "behaveTestRunner.pytestCommand": "pytest",          // Command for pytest-bdd framework
  "behaveTestRunner.pytestBddFilePattern": "**/*_test.py"  // File pattern for pytest-bdd tests
}

Framework Auto-Detection

The extension automatically detects your framework based on:

Behave: Looks for .behaverc, behave.ini, or behave configuration in setup.cfg/pyproject.toml
pytest-bdd: Looks for pytest.ini, pytest-bdd in pyproject.toml, or pytest-bdd imports

General Settings

{
  "behaveTestRunner.workingDirectory": "",             // Working directory for tests
  "behaveTestRunner.autoDiscoverTests": true,          // Auto-discover tests on startup
  "behaveTestRunner.testFilePattern": "**/*.feature",  // Pattern for feature files
  "behaveTestRunner.enableCodeLens": true,             // Enable CodeLens buttons
  "behaveTestRunner.enableTestExplorer": true,         // Enable Test Explorer integration
  "behaveTestRunner.priority": "normal",               // Extension priority
  "behaveTestRunner.parallelExecution": false,         // Enable parallel execution
  "behaveTestRunner.maxParallelProcesses": 4,          // Max parallel processes
  "behaveTestRunner.outputFormat": "pretty",           // Output format
  "behaveTestRunner.tags": "",                         // Default tags filter
  "behaveTestRunner.dryRun": false                     // Enable dry run mode
}

Project Structure

src/
├── core/                   # Core functionality
│   └── test-executor.ts    # Test execution logic
├── parsers/                # File parsing
│   └── feature-parser.ts   # Gherkin feature file parser
├── test-providers/         # VS Code test integration
│   └── behave-test-provider.ts
├── commands/               # VS Code commands
│   └── command-manager.ts  # Command registration and handling
├── utils/                  # Utilities
│   └── logger.ts           # Logging utility
├── types/                  # TypeScript type definitions
│   └── index.ts            # Shared interfaces and types
└── extension.ts            # Main extension entry point

Development

Prerequisites

Node.js 20.x or later
npm or yarn

Setup

# Install dependencies
npm install

# Compile TypeScript
npm run compile

# Run tests
npm test

# Package extension
npm run package

Development Scripts

Script	Description
`npm run compile`	Compile TypeScript and build extension
`npm run watch`	Watch for changes and recompile
`npm run lint`	Run ESLint
`npm run lint:fix`	Fix ESLint issues
`npm test`	Run all tests
`npm run package`	Create VSIX package
`npm run package:vsix`	Create VSIX package with version
`npm run dev`	Start development mode
`npm run test:unit`	Run unit tests only
`npm run test:integration`	Run integration tests only
`npm run test:parser`	Run parser tests only
`npm run test:execution`	Run execution tests only

Testing

The extension includes comprehensive tests:

Unit Tests: Test individual components
Integration Tests: Test VS Code integration
Parser Tests: Test Gherkin parsing
Execution Tests: Test test execution

Run tests with:

npm test                   # Run all tests
npm run test:unit          # Run unit tests only
npm run test:integration   # Run integration tests only
npm run test:parser        # Run parser tests only
npm run test:execution     # Run execution tests only

Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Style

Follow TypeScript best practices
Use ESLint for code linting
Write comprehensive tests
Follow conventional commit messages

Troubleshooting

Common Issues

Tests not discovered:

Ensure .feature files exist in your workspace
Check the testFilePattern setting
Try running "Behave: Discover Tests" manually

Tests not running:

Verify your BDD framework is installed:
- Behave: behave --version
- pytest-bdd: pytest --version and check for pytest-bdd plugin
Check the framework command settings (behaveCommand or pytestCommand)
Ensure the working directory is correct
Verify the framework auto-detection or set it manually

Debug not working:

Make sure Python debugger is configured
Check that the Python extension is installed
Verify the debug configuration

Logs

Enable detailed logging by opening the Output panel (View → Output) and selecting "Behave Test Runner" from the dropdown.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Changelog

See CHANGELOG.md for a list of changes and version history.

Support

"Code, coffee, repeat. Support the cycle! : If you enjoy using Behave Test Runner, consider contributing to fuel my coding dreams!

Acknowledgments

Behave - Python BDD framework
pytest-bdd - pytest plugin for BDD
VS Code Extension API - Extension development framework
Gherkin - BDD syntax specification
Cucumber (Gherkin) Full Support - VS Code extension for Gherkin syntax highlighting and autocomplete
Freepik - Extension icon design

Behave Test Runner

upscaled.dev

Behave Test Runner

Features

Installation

From VSIX

From Source

Usage

Prerequisites

Basic Usage

Test Organization/Test View

Default: Feature-Based (Hierarchical) Organization

Switching views

Alternative Organization Strategies

Extended support for scenario outline

CodeLens Features

Code lens inline buttons

Step Definition Navigation

Recommended companion settings (Cucumber Full Support)

Commands

Configuration

Extension Settings

Intelligent Command Detection

Example Configuration

Example Feature Files

Extension Compatibility

Recommended Configuration with Cucumber (Gherkin) Full Support

Extension Priority

Framework Selection Best Practices

Automatic Detection Priority

Conflict Prevention

Manual Override

Configuration

Framework Selection

Framework Auto-Detection

General Settings

Project Structure

Development

Prerequisites

Setup

Development Scripts

Testing

Contributing

Code Style

Troubleshooting

Common Issues

Logs

License

Changelog

Support

Acknowledgments