Azure DevOps Copilot CLI Extension

A dual-purpose extension that integrates GitHub Copilot CLI into both Visual Studio Code and Azure DevOps pipelines for automated integration testing and enhanced software quality assurance.

🚀 Features

AI-Powered Integration Testing: Leverage GitHub Copilot CLI to generate comprehensive integration tests
Multi-Framework Support: Intelligent support for Playwright, RestAssured, SpecFlow, pytest, and Go testify
Smart Application Discovery: Automatically analyzes Node.js, ASP.NET, Java, Python, and Docker applications
Dual Integration: Works as both VS Code extension and Azure DevOps pipeline task
Azure DevOps Pipeline Integration: Seamlessly integrate with your existing CI/CD workflows
Quality Gate Automation: Automated quality validation with configurable thresholds
Post-Deployment Validation: Validate deployments on LOCAL/INT/QA environments automatically

📦 Installation Options

Option 1: VS Code Extension

Install from the VS Code marketplace or manually:

From VS Code Marketplace: Search for "Azure DevOps Copilot CLI Extension"
Manual installation: Download the .vsix file and install using code --install-extension

Option 2: Azure DevOps Extension

Install in your Azure DevOps organization:

Go to Organization Settings → Extensions → Browse Marketplace
Search for "Azure DevOps Copilot CLI Extension"
Click Get it free and install to your organization
Or upload the Azure DevOps .vsix file manually

📋 Prerequisites

Node.js (v20 or higher)
PowerShell (v7.0 or higher) - for Azure DevOps tasks
GitHub Copilot CLI (npm install -g @github/copilot)
Azure DevOps account with pipeline access (for Azure DevOps integration)
GitHub Copilot subscription

🚀 Quick Start

VS Code Usage

Install the extension from VS Code marketplace
Install GitHub Copilot CLI:
```
npm install -g @github/copilot
```

Authenticate with GitHub:

export GITHUB_TOKEN="your_github_token"

Run integration tests:
- Open Command Palette (Ctrl+Shift+P)
- Run Azure DevOps Copilot: Run Integration Tests
- Select target environment

Azure DevOps Pipeline Usage

Add the Copilot CLI task to your Azure DevOps pipeline:

# azure-pipelines.yml
stages:
- stage: IntegrationTests
  jobs:
  - job: CopilotTests
    steps:
    - task: CopilotCLITask@1
      displayName: 'Run AI-Powered Integration Tests'
      inputs:
        testEnvironment: 'QA'
        copilotCliPath: 'npx'
        githubToken: '$(GITHUB_TOKEN)'
        qualityGateThreshold: '85'
        integrationTestTimeout: '300'
        failOnError: true

Advanced Pipeline Configuration

- task: CopilotCLITask@1
  displayName: 'Comprehensive Integration Testing'
  inputs:
    testEnvironment: 'INT'
    testSuite: 'api-tests'
    copilotCliPath: 'npx'
    githubToken: '$(GITHUB_TOKEN)'
    customPrompt: |
      Focus on API endpoint testing and database integration.
      Include performance benchmarks and security validation.
      Test error handling for edge cases.
    integrationTestTimeout: '600'
    qualityGateThreshold: '90'
    mcpServerUrl: 'https://mcp.example.com'
    workingDirectory: '$(System.DefaultWorkingDirectory)'
    failOnError: true

⚙️ Configuration

VS Code Settings

Configure through VS Code settings:

{
  "azdo-copilot.copilotCliPath": "npx",
  "azdo-copilot.githubToken": "your_github_token",
  "azdo-copilot.integrationTestTimeout": 300,
  "azdo-copilot.qualityGateThreshold": 80,
  "azdo-copilot.defaultEnvironment": "LOCAL",
  "azdo-copilot.mcpServerUrl": "https://mcp.example.com"
}

Azure DevOps Task Inputs

Input	Type	Required	Default	Description
`testEnvironment`	pickList	Yes	LOCAL	Target environment (LOCAL/INT/QA/STAGING/PRODUCTION)
`testSuite`	string	No	-	Specific test suite to execute
`copilotCliPath`	string	Yes	npx	Path to GitHub Copilot CLI executable
`githubToken`	string	No	$(GITHUB_TOKEN)	GitHub personal access token
`customPrompt`	multiLine	No	-	Custom prompt to guide test generation
`integrationTestTimeout`	string	No	300	Test timeout in seconds
`qualityGateThreshold`	string	No	80	Quality gate threshold percentage
`mcpServerUrl`	string	No	-	MCP server URL for enhanced context
`workingDirectory`	filePath	No	$(System.DefaultWorkingDirectory)	Working directory for execution
`failOnError`	boolean	No	true	Fail build on test failures

🔧 Setup for Azure DevOps

1. Install the Extension

Download the Azure DevOps extension from releases or build from source
Upload to your Azure DevOps organization via Organization Settings → Extensions
Install to your organization

2. Configure Authentication

Set up GitHub authentication in your pipeline:

variables:
  - name: GITHUB_TOKEN
    value: 'your_github_token' # Use Azure DevOps secret variables

3. Agent Requirements

Ensure your build agents have:

Node.js v20+
PowerShell 7+
Internet access for GitHub Copilot CLI

4. Pipeline Integration Examples

Basic Integration Testing

- stage: Test
  jobs:
  - job: IntegrationTests
    steps:
    - task: CopilotCLITask@1
      inputs:
        testEnvironment: 'INT'
        copilotCliPath: 'npx'
        githubToken: '$(GITHUB_TOKEN)'

Multi-Environment Testing

- stage: Testing
  jobs:
  - job: QATests
    steps:
    - task: CopilotCLITask@1
      inputs:
        testEnvironment: 'QA'
        qualityGateThreshold: '90'
  
  - job: StagingTests
    dependsOn: QATests
    steps:
    - task: CopilotCLITask@1
      inputs:
        testEnvironment: 'STAGING'
        qualityGateThreshold: '95'

🎯 Output Variables

The Azure DevOps task sets the following output variables for use in subsequent tasks:

CopilotTestsPassed - Number of tests that passed
CopilotTestsFailed - Number of tests that failed
CopilotQualityScore - Overall quality score percentage
CopilotTestSuccess - Boolean indicating overall success

Using Output Variables

- task: CopilotCLITask@1
  name: 'CopilotTests'
  inputs:
    testEnvironment: 'QA'

- script: |
    echo "Tests passed: $(CopilotTestsPassed)"
    echo "Quality score: $(CopilotQualityScore)%"
  displayName: 'Display test results'

🧪 How It Works

Project Detection: Automatically analyzes your workspace (Node.js, Java, Python, .NET, Go)
Framework Detection: Identifies test frameworks (Playwright, Jest, MSTest, pytest, etc.)
Test Generation: Uses GitHub Copilot CLI to create environment-specific tests
Environment Configuration: Loads API URLs from OpenAPI specs or config files
Test Execution: Runs tests with real-time progress tracking and timeout management
Quality Validation: Provides quality scores and actionable insights
Result Reporting: Sets Azure DevOps variables and build status

📁 Project Structure

├── vss-extension.json     # Azure DevOps extension manifest
├── package.json          # VS Code extension configuration
├── azure-pipelines.yml   # CI/CD pipeline for the extension
├── task/                 # Azure DevOps task definition
│   ├── task.json         # Task metadata and inputs
│   └── task.ps1          # PowerShell execution script
├── src/                  # VS Code extension source
│   ├── extension.ts      # Core extension implementation
│   ├── mcp-client.ts     # MCP integration
│   └── utils/
│       └── exec.ts       # Command execution utilities
└── config/
    ├── environments.json # Environment configurations
    └── example-openapi.yaml # OpenAPI specification template

🔍 Troubleshooting

Common Issues

GitHub Copilot CLI not found

npm install -g @github/copilot

GitHub authentication failed

# Option 1: Environment variable
export GITHUB_TOKEN="your_token"

# Option 2: GitHub CLI
gh auth login

# Option 3: Azure DevOps variable
# Set GITHUB_TOKEN in pipeline variables

PowerShell execution policy (Windows)

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Debug Mode

Enable verbose logging by setting the Azure DevOps variable:

variables:
  system.debug: true

🤝 Contributing

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Commit changes: git commit -m 'Add amazing feature'
Push to branch: git push origin feature/amazing-feature
Open a Pull Request

🚀 Feature Enhancement

Have ideas for new features or improvements? We'd love to hear from you!

Contact us for feature enhancements: info@jariyah.us

Whether you need custom integrations, additional test frameworks, or specialized Azure DevOps workflows, our team can help enhance this extension to meet your specific requirements.

⭐ Rate & Review

Don't forget to send rating and review for this extension!

If this extension has helped improve your development workflow and Azure DevOps pipelines, please consider:

⭐ Rating it on VS Code Marketplace - Help others discover this tool
⭐ Rating it on Azure DevOps Marketplace - Share your experience with the community
� Writing a review - Let others know how it's working for you
🐛 Reporting issues - Help us improve by sharing any bugs or suggestions

Your feedback helps us continue to improve and helps other developers find useful tools!

�📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📚 Documentation

VS Code Extension Guide - VS Code specific documentation
Configuration Templates - Environment and OpenAPI examples
Publishing Guide - Extension publishing instructions
Changelog - Version history and updates

🚀 Ready to supercharge your Azure DevOps pipelines with AI-powered testing!

🔧 Key Commands

Run Integration Tests: Execute AI-generated integration tests
Setup Pipeline Task: Generate Azure DevOps pipeline configurations
Configure MCP Server: Set up Model Context Protocol integration
Validate Deployment: Perform post-deployment validation

🧪 How It Works

Project Detection: Automatically analyzes your workspace (Node.js, Java, Python, .NET, Go)
Test Generation: Uses GitHub Copilot CLI to create framework-specific tests
Environment Configuration: Loads API URLs from OpenAPI specs or config files
Test Execution: Runs tests with real-time progress tracking
Quality Validation: Provides quality scores and actionable insights

🚀 Ready to supercharge your Azure DevOps pipelines with AI-powered testing!

Azure DevOps Copilot Pipeline Extension

Jariyah Foundation

Azure DevOps Copilot CLI Extension

🚀 Features

📦 Installation Options

Option 1: VS Code Extension

Option 2: Azure DevOps Extension

📋 Prerequisites

🚀 Quick Start

VS Code Usage

Azure DevOps Pipeline Usage

Advanced Pipeline Configuration

⚙️ Configuration

VS Code Settings

Azure DevOps Task Inputs

🔧 Setup for Azure DevOps

1. Install the Extension

2. Configure Authentication

3. Agent Requirements

4. Pipeline Integration Examples

Basic Integration Testing

Multi-Environment Testing

🎯 Output Variables

Using Output Variables

🧪 How It Works

📁 Project Structure

🔍 Troubleshooting

Common Issues

Debug Mode

🤝 Contributing

🚀 Feature Enhancement

⭐ Rate & Review

�📄 License

📚 Documentation

🔧 Key Commands

🧪 How It Works

Azure DevOps Copilot Pipeline Extension

Jariyah Foundation

Azure DevOps Copilot CLI Extension

🚀 Features

📦 Installation Options

Option 1: VS Code Extension

Option 2: Azure DevOps Extension

📋 Prerequisites

🚀 Quick Start

VS Code Usage

Azure DevOps Pipeline Usage

Advanced Pipeline Configuration

⚙️ Configuration

VS Code Settings

Azure DevOps Task Inputs

🔧 Setup for Azure DevOps

1. Install the Extension

2. Configure Authentication

3. Agent Requirements

4. Pipeline Integration Examples

Basic Integration Testing

Multi-Environment Testing

🎯 Output Variables

Using Output Variables

🧪 How It Works

📁 Project Structure

🔍 Troubleshooting

Common Issues

Debug Mode

🤝 Contributing

🚀 Feature Enhancement

⭐ Rate & Review

�📄 License

📚 Documentation

🔗 Related Links

🔧 Key Commands

🧪 How It Works