English | 繁體中文

AI Code Review for Azure DevOps

This is an Azure DevOps Pipeline extension that leverages the power of Large Language Models (LLMs) to automatically perform code reviews on Pull Request (PR) changes. It acts as an intelligent coding assistant, analyzing diffs and posting insightful comments directly to the PR.

Now supporting all major AI providers:

• GitHub Copilot (All versions supported)
• OpenAI (GPT-4o, etc.)
• Google Gemini
• Anthropic Claude
• xAI Grok

Highlight: Maximize the value of your existing GitHub Copilot subscription by integrating it directly into your Azure DevOps PR workflow! This extension also supports GitHub repository Pull Request CI.

✨ Main Features

• Automated PR review: Automatically triggers during PR build validation, acting as a diligent 24/7 reviewer.
• Universal AI Support: Seamlessly switch between Google Gemini, OpenAI, Grok, Claude, and GitHub Copilot based on your needs.
• GitHub Copilot Integration: Connect to GitHub Copilot CLI to perform reviews using your existing subscription (Individual, Business, or Enterprise), ensuring data privacy and cost-efficiency.
• Direct Feedback: Publishes AI review suggestions directly to the PR as comments, threading into the conversation.
• Highly Customizable: Tailor the System Prompts (Inline or File-based), adjust creativity (Temperature), and control token usage.
• Smart Filtering: configure included/excluded file extensions to focus the review on what matters.

Installation

You can install this extension from the Azure DevOps Marketplace: https://marketplace.visualstudio.com/items?itemName=LawrenceShen.ai-pr-autoreview

🛠️ Setup steps

📊 AI Provider Prerequisites Comparison

Different AI Providers have different prerequisites:

GitHub Copilot Prerequisites

If you or your organization has a GitHub Copilot subscription (Individual, Business, or Enterprise), you can use GitHub Copilot for PR Code Review in three different modes.

Three Usage Modes

The extension automatically detects which mode to use based on the parameters you provide:

Important: GitHub Token and CLI Server Address cannot be used together. Choose one authentication method only.

Token Mode Setup (Recommended for Cloud CI)

1. Obtain Fine-grained Personal Access Token

   Go to GitHub Settings → Developer settings → Personal access tokens → Fine-grained tokens

   Create a new token with:

   • Permissions: Account permissions → Copilot Requests → Access: Read-only
   • Token format: github_pat_ (Fine-grained) or gho_, ghu_ (other supported types)
   • Not supported: Classic tokens (ghp_)

2. Store Token as Secret Variable

   In Azure DevOps Pipeline, add the token as a secret variable to prevent exposure in logs.

3. Configure CI Pipeline (Token Mode)

   Important for CI Token Mode: Before using this task, install GitHub Copilot CLI:

   - script: npm install -g @github/copilot
     displayName: 'Install GitHub Copilot CLI'
   

   Reference: GitHub Copilot CLI Installation Guide

4. Configure Pipeline Task

   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Enter your Fine-grained Personal Access Token (or use secret variable)
   • CLI Server Address: Leave empty
   • Model Name: (Optional) Defaults to gpt-4o

Remote CLI Server & Local CLI Mode Setup

Both Remote CLI Server and Local CLI modes require the same initial setup:

1. Install GitHub Copilot CLI

   npm install -g @github/copilot
   

2. Authenticate with GitHub

   copilot auth login
   

For Remote CLI Server Mode Only:

3. Start CLI Server

   Start the CLI Server on an internal network server:

   copilot --headless --port 8080
   

4. Configure Pipeline Task

   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Leave empty
   • CLI Server Address: Enter your-server-ip:8080 or your-domain:8080
   • Model Name: (Optional) Defaults to gpt-4o

For Local CLI Mode:

3. Configure Pipeline Task
   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Leave empty
   • CLI Server Address: Leave empty
   • Model Name: (Optional) Defaults to gpt-4o

Security Best Practices

• Always use Secret Variables to store GitHub Tokens
• Use minimum required permissions (Copilot Requests Read-only)
• Rotate tokens periodically
• Never commit tokens to source control

Before using this Task, you also need to complete the following configuration steps:

Step 1: Configure CI service permissions

To allow the Pipeline service to write AI comments back to the PR, you must grant it the required permissions. If this permission is not set, the Pipeline will fail and display the error Error: TF401027: You need the Git 'PullRequestContribute' permission....

• Configure the CI build service to write back PR comments: Projects Settings -> Repositories -> Security.
• In the user list, find your Project Collection Build Service (YourCollectionName) account (or the specific service account your Pipeline uses).
• Set the "Contribute to pull request" permission to Allow.

Step 2: Create a Pull Request (PR) Pipeline

Set up branch policies so that this Pipeline is automatically triggered when a PR is created. This extension only triggers the Code Review process during PR builds; it will be skipped in standard build runs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure Build Validation within Branch Policies according to your team's rules.
• Ensure your Pipeline includes the normal CI Build Tasks, then add this extension.
• Enter the Task parameters and adjust them according to your needs.

Step 3: (Recommended) Enforce PR-based code merges

To ensure all code is code-reviewed, we recommend configuring branch policies to require merges through PRs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure the branch policy and enable Require a minimum number of reviewers. For demonstration purposes we allowed users to approve their own changes; set this according to your team's policies.

📋 Task input parameters explained

Below are all input parameters supported by this Task:

🎉 Result display

Gemini

OpenAI

Grok (xAI)

Claude (Anthropic)