Azure DevOps Test Coverage Analyzer & Generator

A powerful Visual Studio Code extension that leverages Azure OpenAI to automatically generate comprehensive test cases for your Azure DevOps work items. This extension helps quality assurance teams and developers create thorough test coverage by analyzing work item requirements and generating intelligent, context-aware test cases.

Features

� Revolutionary Test Coverage Analysis

• Smart Coverage Assessment: Analyzes existing test cases to calculate comprehensive coverage scores (0-100%)
• Gap Detection: Identifies missing test scenarios and coverage gaps using AI-powered analysis
• Risk Assessment: Automatic risk level evaluation (Low/Medium/High) based on current coverage
• Visual Coverage Dashboard: Interactive webview panel with rich metrics and analytics
• Fallback Analysis: Robust heuristic-based analysis when AI services are unavailable

�🚀 AI-Powered Test Generation

• Utilizes Azure OpenAI models to generate intelligent, context-aware test cases
• Analyzes work item details including title, description, and acceptance criteria
• Generates multiple test case types: functional, negative, edge case, integration, security, performance, and UI/UX tests
• Coverage-Driven Generation: Creates test cases specifically targeting identified coverage gaps

🔗 Seamless Azure DevOps Integration

• Direct integration with Azure DevOps REST API v7.0
• Fetches work item details and creates test cases automatically
• Links generated test cases to source work items with proper relationships
• Supports all work item types: User Stories, Features, Requirements, Bugs, Tasks, and custom types
• Enhanced Work Item Validation: Ensures only valid work items are processed

🎯 Smart Duplicate Detection

• Enhanced duplicate avoidance using comprehensive relationship analysis
• Analyzes existing test case titles, steps, and scenarios
• Prevents creation of redundant test cases

📊 Advanced Analytics & Export Capabilities

• Interactive Coverage Dashboard: Real-time coverage metrics and visual analytics
• Rich Reporting: Detailed breakdown of covered vs missing scenarios
• Educational Guidance: Comprehensive notes about AI limitations and testing best practices
• Export test cases in multiple formats: CSV, Excel (XLSX)
• Rich webview panel with syntax highlighting and interactive elements

🛡️ Robust Error Handling

• Comprehensive error handling with detailed logging
• Graceful fallback mechanisms when AI analysis fails
• Real-time progress indicators
• User-friendly feedback and diagnostic tools

Getting Started

What's New in v0.0.3

🎉 Revolutionary Coverage Analysis - The biggest update yet!

• Smart Analysis Before Generation: Now analyzes existing test coverage before generating new test cases
• Visual Coverage Dashboard: Beautiful, interactive dashboard showing coverage metrics and gaps
• AI-Powered Gap Detection: Identifies exactly what's missing from your test coverage
• Risk Assessment: Automatic evaluation of testing risk levels
• Targeted Test Generation: Creates test cases specifically for identified coverage gaps
• Enhanced Robustness: Fallback analysis when AI services are unavailable

Prerequisites

1. Visual Studio Code (version 1.103.0 or higher)
2. Azure DevOps Organization with appropriate permissions
3. Azure OpenAI Service with a deployed model (e.g., GPT-4)

Installation

1. Install the extension from the VS Code marketplace or install the .vsix file
2. Open VS Code and navigate to the Command Palette (Ctrl+Shift+P)
3. Run the command: Azure DevOps: Configure Settings

Configuration

The extension requires the following configuration parameters:

Setting up Azure DevOps Personal Access Token

1. Go to your Azure DevOps organization
2. Click on your profile → Personal Access Tokens
3. Create a new token with the following scopes:
   • Work Items: Read & Write
   • Test Management: Read & Write

Setting up Azure OpenAI

1. Create an Azure OpenAI resource in Azure Portal
2. Deploy a model (GPT-4 recommended)
3. Get the endpoint URL and API key from the resource

Usage

New Workflow with Coverage Analysis

1. Configure the Extension

   Command Palette → Azure DevOps: Configure Settings
   

2. Analyze Test Coverage

   Command Palette → Azure DevOps: Analyze Test Coverage
   

   • Enter the work item ID when prompted
   • View Coverage Analysis: Review comprehensive coverage dashboard showing:
     • Current coverage percentage
     • Existing test cases
     • Missing scenarios and coverage gaps
     • Risk assessment
     • Recommendations for improvement

3. Generate Targeted Test Cases

   • After reviewing coverage analysis, click "Generate Test Cases" to create tests targeting specific gaps
   • Review AI-generated test cases in the interactive panel with educational guidance
   • Choose to create test cases in Azure DevOps or export them

4. Additional Commands

   • Azure DevOps: Reset Settings - Clear all configuration
   • Azure DevOps: Show Diagnostics - View extension diagnostics

Key Workflow Benefits

• Intelligence First: Understand existing coverage before generating new tests
• Gap-Focused Generation: Generate tests specifically for identified coverage gaps
• Data-Driven Decisions: Make informed decisions about test case priorities
• Visual Analytics: Rich dashboard provides immediate insights into test coverage health

Pro Tips for Maximum Effectiveness

1. Start with Coverage Analysis: Always run coverage analysis first to understand your current testing landscape
2. Review Coverage Gaps: Pay attention to the "Missing Scenarios" section for targeted test generation
3. Monitor Risk Levels: High-risk work items need immediate attention for additional test coverage
4. Use Fallback Analysis: When AI services are unavailable, the extension provides heuristic-based analysis
5. Export for Documentation: Use the export features to create comprehensive test documentation
6. Iterative Improvement: Run analysis regularly as work items evolve to maintain coverage health

Coverage Analysis Features

Comprehensive Coverage Assessment

The extension provides sophisticated test coverage analysis capabilities:

Coverage Metrics

• Coverage Score: Intelligent 0-100% scoring based on existing test case depth and breadth
• Test Case Count: Total number of existing test cases linked to the work item
• Missing Scenarios: AI-identified gaps in test coverage
• Risk Level: Automatic assessment (Low/Medium/High) based on coverage analysis

Visual Analytics Dashboard

• Interactive Coverage Score: Large, color-coded percentage with contextual styling
• Coverage vs Missing Grid: Side-by-side comparison of covered and missing scenarios
• Existing Test Cases: Detailed list of current test cases with states
• Recommendations: AI-powered suggestions for improving coverage

Intelligent Analysis Types

• AI-Powered Analysis: Uses Azure OpenAI to deeply analyze coverage gaps
• Fallback Heuristic Analysis: Robust backup analysis when AI services are unavailable
• Relationship Analysis: Comprehensive analysis of work item relationships and dependencies

Supported Test Case Types

The extension generates various types of test cases based on work item analysis and coverage gaps:

• Functional Tests: Happy path scenarios and core functionality validation
• Negative Tests: Error handling and invalid input scenarios
• Edge Case Tests: Boundary conditions and limit testing
• Integration Tests: Component interaction and system integration testing
• Security Tests: Security considerations and vulnerability assessment
• Performance Tests: Performance criteria validation and load testing
• UI/UX Tests: User experience and interface validation
• Regression Tests: Ensuring existing functionality remains intact
• Accessibility Tests: Compliance with accessibility standards
• Data Validation Tests: Data integrity and validation scenarios

Build and Development

Prerequisites for Development

• Node.js (22.x or higher)
• npm

Development Setup

1. Clone the repository

2. Install dependencies:

   npm install
   

3. Run linting:

   npm run lint
   

4. Run tests:

   npm test
   

Build Extension

# Install vsce (Visual Studio Code Extension CLI)
npm install -g vsce

# Package the extension
vsce package

Project Structure

├── extension.js              # Main extension entry point
├── package.json             # Extension manifest and dependencies
├── utils/
│   ├── azureDevOpsClient.js # Azure DevOps API client
│   ├── azureOpenAIClient.js # Azure OpenAI API client
│   └── testCaseFormatter.js # Test case formatting utilities
├── README.md
├── CHANGELOG.md
└── eslint.config.mjs

Contributing

We welcome contributions! Here's how you can help:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and ensure they follow the coding standards
4. Run tests: npm test
5. Run linting: npm run lint
6. Commit your changes: git commit -m 'Add amazing feature'
7. Push to the branch: git push origin feature/amazing-feature
8. Open a Pull Request

Development Guidelines

• Follow existing code style and conventions
• Add tests for new functionality
• Update documentation as needed
• Ensure all tests pass before submitting

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

If you encounter any issues or have suggestions:

1. Check the CHANGELOG.md for recent updates
2. Use the Azure DevOps: Show Diagnostics command for troubleshooting
3. Create an issue in the repository with detailed information

Version History

• v0.0.3 (Current) - Major update with test coverage analysis, enhanced UI, and intelligent gap detection
• v0.0.2 - Enhanced features with marketplace release and improved commands
• v0.0.1 - Initial development version with core test case generation

For detailed version history, see CHANGELOG.md.