AI Code Review for Azure DevOps (Powered by GitHub Copilot)

Automatically review Pull Requests using GitHub Models API. Get instant feedback on security vulnerabilities, performance issues, and best practices.

Features

• GitHub Models API — Uses GPT-4o, Llama 3.1, Mistral models via your GitHub account
• Multi-Model Support — Choose from GPT-4o, GPT-4o-mini, Llama 3.1, Mistral Large
• Automated PR Reviews — Runs automatically on every Pull Request
• Multi-Language Support — Python, JavaScript/TypeScript, C#, Java, Go, Rust, C++, and more
• Framework-Aware — React, Vue, Angular, FastAPI, Flask, Express, Spring, ASP.NET
• Security Analysis — OWASP Top 10, injection vulnerabilities, authentication issues
• Performance Review — Algorithm complexity, database queries, memory management
• Custom Prompts — Tailor review focus to your team's needs
• Inline Comments — Posts structured review comments directly on your PR

Quick Start

1. Install the Extension

Install from the Visual Studio Marketplace.

2. Create a GitHub PAT

1. Go to GitHub Settings > Developer Settings > Personal Access Tokens
2. Generate a new Fine-grained token:
   • Repository access: Public
   • Permission: Copilot Requests
3. Store as a secret variable GITHUB_PAT in your Azure DevOps pipeline

3. Add to Your Pipeline

trigger: none

pr:
  branches:
    include:
      - main
      - develop

pool:
  vmImage: 'ubuntu-latest'

steps:
- checkout: self
  fetchDepth: 0  # Required for full diff

- task: AICodeReview@2
  inputs:
    githubPat: $(GITHUB_PAT)
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

4. Enable OAuth Token

In your pipeline settings, ensure "Allow scripts to access the OAuth token" is enabled.

Configuration Options

Available Models

Custom Review Prompts

Inline Prompt

- task: AICodeReview@2
  inputs:
    githubPat: $(GITHUB_PAT)
    copilotModel: 'gpt-4o'
    customPrompt: |
      Review this code focusing only on:
      - Security vulnerabilities (OWASP Top 10)
      - .NET 8 migration best practices
      - BACnet protocol handling patterns
      - Async/await correctness
      Keep comments concise and actionable.

Prompt File

Create a .copilot/review-prompt.txt in your repo:

- task: AICodeReview@2
  inputs:
    githubPat: $(GITHUB_PAT)
    promptFile: '$(Build.SourcesDirectory)/.copilot/review-prompt.txt'

Supported Languages

Supported Frameworks

Review Output

The task posts a structured review comment on your PR:

## AI Code Review (Powered by GitHub Copilot)

Overall Assessment: CHANGES REQUESTED

> Found 2 security issues and 1 performance concern in the authentication module.

### Critical Issues

**1. SQL Injection Vulnerability** [security] `src/repository.py:45`
  - **Problem**: String concatenation used in SQL query
  - **Suggestion**: Use parameterized queries with SqlParameter

### Medium Priority

**1. Consider async/await** [performance] `src/service.cs:23`
  - **Problem**: Synchronous database calls blocking thread pool
  - **Suggestion**: Use async overloads with await for I/O operations

How It Works

┌─────────────────┐     ┌─────────────────┐     ┌────────────────┐
│   PR Created     │────▶│  Pipeline Runs   │────▶│  Fetch Diff    │
│  (Azure DevOps)  │     │  (Build Agent)   │     │  (git / API)   │
└─────────────────┘     └─────────────────┘     └───────┬────────┘
                                                        │
                        ┌─────────────────┐     ┌───────▼────────┐
                        │ Post PR Comment  │◀────│ Copilot CLI    │
                        │ (ADO REST API)   │     │ Reviews Diff   │
                        └─────────────────┘     └────────────────┘

1. PR trigger — Branch policy triggers the review pipeline
2. Fetch diff — Gets code changes via git diff (preferred) or Azure DevOps API
3. Copilot review — Sends diff to GitHub Copilot CLI with review prompt
4. Post results — Formats findings and posts as a PR comment thread

Privacy & Security

• Code is sent to GitHub Copilot for analysis (same as using Copilot in your IDE)
• GitHub PAT is handled as a secret and never logged
• No code is stored by this extension beyond the pipeline run
• Review results are posted only to your PR

Troubleshooting

"GitHub Copilot CLI not found"

• Linux: The task auto-installs via https://gh.io/copilot-install. Ensure the agent has internet access and curl/bash are available.
• Windows: Requires winget. MS-hosted agents have this by default.

Authentication errors

• Verify your GitHub PAT has Copilot Requests permission
• If your GitHub account is in an organization, ensure the admin has enabled Copilot CLI under GitHub Policies > Copilot
• Ensure "Allow scripts to access OAuth token" is enabled in the pipeline

Empty review or no comments

• Add fetchDepth: 0 to the checkout step for complete git history
• Check pipeline logs for Copilot CLI output
• For very large PRs, try setting a lower maxFiles value or use claude-haiku-4.5 for better handling of large context

Timeout errors

• Large PRs may take longer. The default timeout is 10 minutes.
• Consider using claude-haiku-4.5 for faster reviews on large PRs
• Break large PRs into smaller, focused changes

Build Service permissions

If using System.AccessToken, the Build Service identity needs "Contribute to pull requests" permission:

1. Go to Repos > Manage repositories > Security
2. Find "[Project] Build Service ([Org])"
3. Set "Contribute to pull requests" to Allow

Support

• GitHub Issues
• Documentation

License

MIT License - See LICENSE for details.