English | 繁體中文

AI Code Review for Azure DevOps

This is an Azure DevOps Pipeline extension that leverages the power of Large Language Models (LLMs) to automatically perform code reviews on Pull Request (PR) changes. It acts as an intelligent coding assistant, analyzing diffs and posting insightful comments directly to the PR.

Now supporting all major AI providers:

• GitHub Copilot (All versions supported)
• OpenAI (GPT-4o, etc.)
• Google Gemini
• Anthropic Claude
• xAI Grok

Highlight: Maximize the value of your existing GitHub Copilot subscription by integrating it directly into your Azure DevOps PR workflow! This extension also supports GitHub repository Pull Request CI.

✨ Main Features

• Automated PR review: Automatically triggers during PR build validation, acting as a diligent 24/7 reviewer.
• Universal AI Support: Seamlessly switch between Google Gemini, OpenAI, Grok, Claude, and GitHub Copilot based on your needs.
• GitHub Copilot Integration: Connect to GitHub Copilot CLI to perform reviews using your existing subscription (Individual, Business, or Enterprise), ensuring data privacy and cost-efficiency.
• Direct Feedback: Publishes AI review suggestions directly to the PR as comments, threading into the conversation.
• Highly Customizable: Tailor the System Prompts (Inline or File-based), adjust creativity (Temperature), and control token usage.
• Smart Filtering: configure included/excluded file extensions to focus the review on what matters.

Installation

You can install this extension from the Azure DevOps Marketplace: https://marketplace.visualstudio.com/items?itemName=LawrenceShen.ai-pr-autoreview

🛠️ Setup steps

📊 AI Provider Prerequisites Comparison

Different AI Providers have different prerequisites:

GitHub Copilot CLI Prerequisites

If you or your organization has a GitHub Copilot subscription (Individual, Business, or Enterprise), you can use the internal CLI Server for PR Code Review.

Applicable Scenarios

• Have an active GitHub Copilot subscription
• Want to reuse existing Copilot infrastructure
• Need unified AI toolchain experience

CLI Server Setup Steps

1. Install GitHub Copilot CLI

   npm install -g @github/copilot-sdk
   

2. Start CLI Server Mode

   Start the CLI Server on an internal network server:

   copilot --headless --port 8080
   

3. Configure in Pipeline Task

   • AI Provider: Select GitHub Copilot
   • Network Type: Select Intranet
   • CLI Server Address: (Optional) Enter your-server-ip:8080 or your-domain:8080. If not provided, will use GitHub Copilot CLI in Build Agent
   • Model Name: (Optional) Defaults to gpt-4o

Important Notes

• Remote Server Mode: When CLI Server Address is provided, ensure Pipeline Agent and CLI Server are on the same internal network or can connect to each other. CLI Server must be running when Pipeline executes.
• Local CLI Mode: When CLI Server Address is not provided, will use the authenticated GitHub Copilot CLI in the Build Agent. Requirements:
  • GitHub Copilot CLI must be installed on Build Agent
  • Must have completed authentication via copilot auth login
  • Agent must have GitHub Copilot access permission
• Currently only supports Intranet mode; Internet mode will be available in future versions

Before using this Task, you also need to complete the following configuration steps:

Step 1: Configure CI service permissions

To allow the Pipeline service to write AI comments back to the PR, you must grant it the required permissions. If this permission is not set, the Pipeline will fail and display the error Error: TF401027: You need the Git 'PullRequestContribute' permission....

• Configure the CI build service to write back PR comments: Projects Settings -> Repositories -> Security.
• In the user list, find your Project Collection Build Service (YourCollectionName) account (or the specific service account your Pipeline uses).
• Set the "Contribute to pull request" permission to Allow.

Step 2: Create a Pull Request (PR) Pipeline

Set up branch policies so that this Pipeline is automatically triggered when a PR is created. This extension only triggers the Code Review process during PR builds; it will be skipped in standard build runs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure Build Validation within Branch Policies according to your team's rules.
• Ensure your Pipeline includes the normal CI Build Tasks, then add this extension.
• Enter the Task parameters and adjust them according to your needs.

Step 3: (Recommended) Enforce PR-based code merges

To ensure all code is code-reviewed, we recommend configuring branch policies to require merges through PRs.

• Select Projects Settings -> Repositories -> YourGitProject -> Policies -> Branch Policies -> select the target branch (for example main or master).
• Configure the branch policy and enable Require a minimum number of reviewers. For demonstration purposes we allowed users to approve their own changes; set this according to your team's policies.

📋 Task input parameters explained

Below are all input parameters supported by this Task:

🎉 Result display

Gemini

OpenAI

Grok (xAI)

Claude (Anthropic)