English | 繁體中文

AI Code Review for Azure DevOps

This is an Azure DevOps Pipeline extension that leverages the power of Large Language Models (LLMs) to automatically perform code reviews on Pull Request (PR) changes. It acts as an intelligent coding assistant, analyzing diffs and posting insightful comments directly to the PR.

Now supporting all major AI providers:

• GitHub Copilot (All versions supported)
• OpenAI (gpt-5-mini, etc.)
• Google Gemini
• Anthropic Claude
• xAI Grok
• Ollama (Local deployment, any Ollama-compatible model)

Highlight: Maximize the value of your existing GitHub Copilot subscription by integrating it directly into your Azure DevOps PR workflow! This extension also supports GitHub repository Pull Request CI.

✨ Main Features

• Automated PR review: Automatically triggers during PR build validation, acting as a diligent 24/7 reviewer.
• Universal AI Support: Seamlessly switch between Google Gemini, OpenAI, Grok, Claude, GitHub Copilot, and self-hosted Ollama based on your needs.
• GitHub Copilot Integration: Connect to GitHub Copilot CLI to perform reviews using your existing subscription (Individual, Business, or Enterprise), ensuring data privacy and cost-efficiency.
• Direct Feedback: Publishes AI review suggestions directly to the PR as comments, threading into the conversation.
• Highly Customizable: Tailor the System Prompts (Built-in, Inline, or File-based), adjust creativity (Temperature), and control token usage.
• Smart Filtering: configure included/excluded file extensions to focus the review on what matters.
• Precise Line Annotation (Inline Comments): AI comments are anchored to specific changed lines in the Files Changed tab, making it easier to pinpoint code issues.

Installation

You can install this extension from the Azure DevOps Marketplace: https://marketplace.visualstudio.com/items?itemName=LawrenceShen.ai-pr-autoreview

🛠️ Setup steps

📊 AI Provider Prerequisites Comparison

Different AI Providers have different prerequisites:

GitHub Copilot Prerequisites

If you or your organization has a GitHub Copilot subscription (Individual, Business, or Enterprise), you can use GitHub Copilot for PR Code Review in three different modes.

Three Usage Modes

The extension automatically detects which mode to use based on the parameters you provide:

Important: GitHub Token and CLI Server Address cannot be used together. Choose one authentication method only.

Token Mode Setup (Recommended for Cloud CI)

1. Obtain Fine-grained Personal Access Token

   Go to GitHub Settings → Developer settings → Personal access tokens → Fine-grained tokens

   Create a new token with:

   • Permissions: Account permissions → Copilot Requests → Access: Read-only
   • Token format: github_pat_ (Fine-grained) or gho_, ghu_ (other supported types)
   • Not supported: Classic tokens (ghp_)

2. Store Token as Secret Variable

   In Azure DevOps Pipeline, add the token as a secret variable to prevent exposure in logs.

3. Configure Pipeline Task

   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Enter your Fine-grained Personal Access Token (or use secret variable)
   • CLI Server Address: Leave empty
   • Model Name: (Optional) Defaults to gpt-5-mini

Remote CLI Server & Local CLI Mode Setup

For Remote CLI Server Mode, the CLI must be installed and authenticated on the server:

1. Install GitHub Copilot CLI (on the server)

   npm install -g @github/copilot
   

   Reference: GitHub Copilot CLI Installation Guide

2. Authenticate with GitHub (on the server)

   copilot auth login
   

For Remote CLI Server Mode Only:

3. Start CLI Server

   Start the CLI Server on an internal network server:

   copilot --headless --port 8080
   

4. Configure Pipeline Task

   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Leave empty
   • CLI Server Address: Enter your-server-ip:8080 or your-domain:8080
   • Model Name: (Optional) Defaults to gpt-5-mini

For Local CLI Mode, the task automatically installs the CLI. You only need to pre-authenticate on the Build Agent:

1. Authenticate with GitHub (on the Build Agent, one-time setup)

   npx @github/copilot auth login
   

2. Configure Pipeline Task

   • AI Provider: Select GitHub Copilot
   • GitHub Copilot Token: Leave empty
   • CLI Server Address: Leave empty
   • Model Name: (Optional) Defaults to gpt-5-mini

Security Best Practices

• Always use Secret Variables to store GitHub Tokens
• Use minimum required permissions (Copilot Requests Read-only)
• Rotate tokens periodically
• Never commit tokens to source control

Before using this Task, you also need to complete the following configuration steps:

⚠️ (GitHub Repositories Only) Configure GitHub Access Token

This step only applies when your Azure DevOps Pipeline sources code from a GitHub repository. If you are using Azure DevOps Git repositories, skip to Step 1.

When a pipeline's source code is hosted on GitHub, the Azure DevOps built-in identity (SystemVssConnection) cannot be used to call the GitHub API. This extension reads PR diffs and posts review comments via the GitHub REST API, which requires a manually configured GitHub Personal Access Token (PAT).

Why this error occurs: Without this configuration, the extension cannot obtain an access token and will fail with:

Task failed with error: ⛔ Unable to get DevOps access token

Step-by-Step Setup

1. Create a GitHub Personal Access Token

   Option A — Classic Token (recommended for simplicity)

   Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)

   Generate a new token and select the appropriate scope based on your repository type:

   

   • Public repositories: Check public_repo — Access public repositories (grants read/write access to public repos, including PR comments)
   • Private repositories: Check repo — Full control of private repositories (the entire repo scope must be selected; selecting only sub-scopes like public_repo or repo:status is not sufficient for private repos)

   Important: For private GitHub repositories, you must select the top-level repo scope. Selecting only sub-items (e.g., repo:status, public_repo) will not grant the required write access to post PR comments.

   Option B — Fine-grained Token

   Go to GitHub Settings → Developer settings → Personal access tokens → Fine-grained tokens

   Create a token scoped to the target repository with:

   • Repository permissions → Pull requests: Read and write (allows the extension to post comments)
   • Repository permissions → Contents: Read-only (allows the extension to read file content)

2. Add the Token as a Secret Pipeline Variable

   In your Azure DevOps Pipeline:

   • Click Edit on your pipeline
   • Click Variables (top-right corner)
   • Click + Add and configure:
     • Name: AccessToken
     • Value: Your GitHub PAT
     • ✅ Enable Keep this value secret to prevent the token from appearing in build logs
   • Click Save

   

3. Notes

   • The AccessToken variable name is case-sensitive and must be exactly AccessToken.
   • Once this token is configured, the extension will use it to read PR diffs from GitHub and post review comments — no additional Azure DevOps permission setup (Steps 1–3 below) is needed for GitHub repositories.

Step 1: Configure CI service permissions

To allow the Pipeline service to write AI comments back to the PR, you must grant it the required permissions. If this permission is not set, the Pipeline will fail and display the error Error: TF401027: You need the Git 'PullRequestContribute' permission....

• Configure the CI build service to write back PR comments: Projects Settings -> Repositories -> Security.
• In the user list, find your Project Collection Build Service (YourCollectionName) account (or the specific service account your Pipeline uses).
• Set the "Contribute to pull request" permission to Allow.

Step 2: Create a Pull Request (PR) Pipeline

Set up branch policies so that this Pipeline is automatically triggered when a PR is created. This extension only triggers the Code Review process during PR builds; it will be skipped in standard build runs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure Build Validation within Branch Policies according to your team's rules.
• Ensure your Pipeline includes the normal CI Build Tasks, then add this extension.
• Enter the Task parameters and adjust them according to your needs.

Step 3: (Recommended) Enforce PR-based code merges

To ensure all code is code-reviewed, we recommend configuring branch policies to require merges through PRs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure the branch policy and enable Require a minimum number of reviewers. For demonstration purposes we allowed users to approve their own changes; set this according to your team's policies.

📋 Task input parameters explained

Below are all input parameters supported by this Task:

For detailed review mode parameter combinations, see PARAMETER-COMBINATIONS.md.

🎉 Result display

Gemini

OpenAI

Grok (xAI)

Claude (Anthropic)