Claude AI Code Review

Automatically review pull request code changes using Claude (Anthropic) AI. The extension adds a pipeline task that analyzes code diffs, detects bugs, security vulnerabilities, performance issues, and style problems, then posts its findings as inline comments directly on the pull request.

Compatible with Azure DevOps Services (cloud) and Azure DevOps Server / TFS (on-premise, 2019+).

How It Works

A developer creates a Pull Request
A build pipeline with the Claude AI Code Review task triggers automatically
The task fetches the list of changed files from the PR
Each file's diff is sent to the Claude API for analysis
Claude returns structured findings (bug, security, performance, style, etc.)
Findings are posted as inline comments on the exact lines in the PR
A summary comment is posted on the PR with total counts
The build log also contains a full review summary

If the task runs again on the same PR (e.g., after a new push), it will not post duplicate comments — existing comments are detected and skipped.

Prerequisites

Anthropic API Key — Get one at console.anthropic.com
Azure DevOps / TFS 2019+ with Git repositories
Pipeline agent with Node.js 16+ (self-hosted or Microsoft-hosted)
Git installed on the agent (used for computing diffs)

Installation

Azure DevOps Services (Cloud)

Install the extension from the Visual Studio Marketplace
It will be available across all projects in your organization

Azure DevOps Server / TFS (On-Premise)

Download the .vsix file
Go to your TFS server: http://<your-server>:<port>/tfs/_gallery/manage
Click Upload extension and select the .vsix file
The extension will appear in the installed extensions list

Setup Guide (Step-by-Step)

Step 1: Create the Claude API Service Connection

This stores your Anthropic API key securely.

Go to Project Settings (bottom-left gear icon)
Under Pipelines, click Service connections
Click New service connection
Select Anthropic Claude API Connection
Fill in:
- API Base URL: https://api.anthropic.com (default, do not change unless using a proxy)
- API Key: Paste your Anthropic API key (starts with sk-ant-...)
- Service connection name: e.g., Claude-API (you will reference this name in the pipeline)
Click Save

Alternative: Instead of a service connection, you can pass the API key directly as a secret pipeline variable (see Alternative: Direct API Key below).

Step 2: Configure Build Service Permissions

The build service account needs permission to post comments on pull requests. Without this, the task will run but comments will NOT appear on the PR.

Go to Project Settings > Repositories
Select your repository (or "All repositories" for project-wide)
Go to the Security tab
Find the user: Project Collection Build Service (DefaultCollection)
- On some setups, it may be called <ProjectName> Build Service (<CollectionName>)
Set the following permission to Allow:
- Contribute to pull requests ✅

Older TFS (2019/2020): If you don't see "Repositories" in Project Settings, navigate to: your repository > Settings (gear icon) > Version Control > Security tab. Find the Build Service identity and set the permission there.

If you skip this step, you will see this error in the build log:
TF401027: You need the Git 'PullRequestContribute' permission to perform this action.

Step 3: Enable OAuth Token Access

The task needs the System.AccessToken to communicate with the Azure DevOps REST API.

YAML Pipeline

Add this at the pipeline or job level:

jobs:
  - job: CodeReview
    pool:
      name: 'Default'  # or 'Azure Pipelines' for cloud
    steps:
      - checkout: self
        fetchDepth: 0
      - task: ClaudeAICodeReview@1
        inputs:
          claudeServiceConnection: 'Claude-API'
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Classic Pipeline

In the pipeline editor, click on the Agent job (not a specific task)
In the right panel, check: Allow scripts to access the OAuth token ✅

Older TFS: On some TFS versions, this checkbox is on the Agent job level, not in the pipeline Options tab. Make sure to look for it on the Agent job.

If you skip this step, you will see this error:
System.AccessToken is not available. Enable "Allow scripts to access the OAuth token" in the pipeline settings.

Step 4: Set Up the Pipeline

Option A: YAML Pipeline

trigger: none  # We only want PR triggers, not push triggers

pr:
  branches:
    include:
      - main
      - develop

pool:
  name: 'Default'       # Your agent pool
  # vmImage: 'ubuntu-latest'  # Use this for Microsoft-hosted agents instead

steps:
  - checkout: self
    fetchDepth: 0        # IMPORTANT: Full history needed for git diff

  - task: ClaudeAICodeReview@1
    displayName: 'Claude AI Code Review'
    inputs:
      claudeServiceConnection: 'Claude-API'
      claudeModel: 'claude-sonnet-4-20250514'
      reviewScope: 'changedFiles'
      postPrComments: true
      failOnIssues: false
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Option B: Classic Pipeline

Create a new Build pipeline (use "Classic editor" if YAML is the default)
Select your repository and branch
Choose Empty job template
Agent job settings:
- Agent pool: Select your pool (e.g., Default)
- Check Allow scripts to access the OAuth token ✅
Click + to add a task, search for Claude AI Code Review, and add it
Configure the task inputs (service connection, model, etc.)
In Get sources settings:
- Set Fetch depth to 0 (unlimited) — this is critical for git diff to work

Important: fetchDepth: 0 (or "Fetch depth = 0" in Classic) is required. Without full git history, the task cannot compute diffs between branches.

Step 5: Set Up PR Trigger

The pipeline must be triggered when a Pull Request is created or updated.

Azure DevOps Services (Cloud)

YAML: Use the pr: trigger section (see YAML example above)
Classic: Go to Triggers tab > Pull request validation > Enable and select target branches

Azure DevOps Server / TFS (On-Premise)

On older TFS versions, PR-triggered builds are configured via Branch Policies, not pipeline triggers:

Go to Repos > Branches
Click the ... menu on your target branch (e.g., main) > Branch policies
Under Build Validation, click Add build policy
Configure:
- Build pipeline: Select the pipeline you created
- Trigger: Automatic
- Policy requirement: Optional (recommended) or Required
- Build expiration: After 0 hours (always run on new pushes)
Click Save

Now, every time a PR targets this branch, the build will trigger automatically.

Task Inputs Reference

Main Inputs

Input	Type	Required	Default	Description
Claude API Service Connection	Service Connection	No*	—	Service connection with your Anthropic API key
Claude API Key (alternative)	String	No*	—	Direct API key as secret variable, e.g., `$(CLAUDE_API_KEY)`
Claude Model	Pick List	Yes	Claude Sonnet 4	AI model: Sonnet 4 (recommended balance), Opus 4 (most capable, slowest), Haiku 4 (fastest, least detailed)
Review Scope	Pick List	Yes	Changed files only	Changed files only: sends only the diff. Full file context: sends the full file content alongside the diff for better context
Post comments to PR	Boolean	No	`true`	Post review findings as inline PR thread comments
Fail task on critical issues	Boolean	No	`false`	If Claude finds critical issues, the pipeline task will fail (blocks the PR if policy requires)

* Either Service Connection or API Key must be provided. Service Connection is recommended.

Advanced Inputs

Input	Type	Default	Description
Max Response Tokens	Number	`4096`	Maximum tokens in Claude's response per file. Increase if reviews are getting truncated
Custom Review Instructions	Multi-line text	—	Additional instructions for Claude, e.g., "Focus on security issues" or "Follow our coding standards: ..."

File Filtering Inputs

Input	Type	Default	Description
Include file patterns	Multi-line glob	`*/.ts`, `*/.js`, `*/.cs`, `*/.py`, `*/.java`, `*/.go`	Only review files matching these patterns (one per line)
Exclude file patterns	Multi-line glob	`*/.min.js`, `/package-lock.json`, `/yarn.lock`, `*/.generated.*`	Skip files matching these patterns (one per line)
Max file size (KB)	Number	`100`	Skip files larger than this size

Alternative: Direct API Key

If you prefer not to use a Service Connection, you can pass the API key directly:

Go to Pipelines > Library > Variable groups (or use pipeline variables)
Create a variable named CLAUDE_API_KEY and mark it as secret (lock icon)
In the task, leave "Service Connection" empty, and set Claude API Key to $(CLAUDE_API_KEY)

Warning: Always use secret variables for API keys. Never hardcode API keys in the pipeline YAML or task configuration.

What Claude Reviews

The AI analyzes each changed file for:

Category	Examples
Bug	Null reference, off-by-one errors, logic errors, missing return statements
Security	SQL injection, XSS, hardcoded secrets, insecure deserialization
Performance	N+1 queries, unnecessary allocations, inefficient algorithms
Maintainability	Dead code, duplicated logic, overly complex methods
Style	Naming conventions, code formatting, missing error handling
Best Practice	SOLID violations, anti-patterns, missing validation

Each finding includes:

Severity: Critical, Warning, Suggestion, or Nitpick
Line number: Exact line in the new version of the file
Message: Description of the issue
Suggested fix: Concrete code suggestion (when applicable)

Example: PR Comment Output

When the task runs, you'll see inline comments like this on the PR:

[CRITICAL] (security)

User input is passed directly to SQL query without sanitization. This is vulnerable to SQL injection.

Suggested fix:
var result = context.Users.Where(u => u.Id == userId).FirstOrDefault();
Generated by Claude AI Code Review

And a summary comment:

Claude AI Code Review Summary

Severity Count

Critical 1

Warning 2

Suggestion 3

Nitpick 0

Total 6

Files reviewed: 4

1 critical issue(s) found. Please review before merging.

Troubleshooting

"System.AccessToken is not available"

Cause: The pipeline is not configured to expose the OAuth token to tasks.

Fix:

YAML: Add env: { SYSTEM_ACCESSTOKEN: $(System.AccessToken) } to the task or job
Classic: Click on the Agent job > check Allow scripts to access the OAuth token
Older TFS: This setting may be on the Agent job level, not in the pipeline Options tab

"TF401027: You need the Git 'PullRequestContribute' permission"

Cause: The build service account does not have permission to post comments on pull requests.

Fix: Go to Project Settings > Repositories > Security > find Project Collection Build Service > set Contribute to pull requests to Allow. See Step 2 above.

"No PR detected; skipping review"

Cause: The pipeline is running but not in a pull request context.

Fix:

Make sure the build was triggered by a PR (not a manual run or push trigger)
For TFS on-premise, set up Branch Policies > Build Validation (see Step 5)
Check that Build.Reason is PullRequest in the build variables

"No files to review after filtering"

Cause: All changed files were filtered out by the include/exclude patterns or file size limit.

Fix:

Check the Include file patterns — make sure your file types are listed (e.g., **/*.cs for C# files)
Check the Exclude file patterns — make sure your files aren't accidentally excluded
Check Max file size — increase if your files are larger than 100KB

"git diff returned empty" or "Found 0 issue(s)"

Cause: The task couldn't compute a diff for the file. This usually happens when:

Fetch depth is not 0 — shallow clones don't have the target branch history
The target branch (origin/main) is not available locally

Fix:

YAML: Set fetchDepth: 0 in the checkout step
Classic: In Get sources, set Fetch depth to 0
Verify: In the build log, you should see git diff returned X chars for ... (not git diff returned empty)

"Claude API error" or "Request timeout"

Cause: The Claude API is unreachable or rate-limited.

Fix:

Verify your API key is valid at console.anthropic.com
Check if the agent has internet access (for on-premise agents behind a firewall)
If you're hitting rate limits (HTTP 429), the task retries automatically up to 3 times with exponential backoff
For on-premise environments with a proxy, configure the agent's proxy settings

Comments appear on wrong lines

Cause: The line numbers Claude reports are based on the diff context. In some edge cases, line mapping may be slightly off.

Fix:

Try changing Review Scope to "Full file context" — this gives Claude the full file alongside the diff, improving line number accuracy
This uses more API tokens but produces more accurate results

Task succeeds but with no comments posted

Cause: Claude found 0 issues (the code looks good), or all findings were duplicates from a previous run.

Check:

Look at the build log for Found X issue(s) in ...
If 0 issues were found, the code may genuinely be clean, or the diff was empty
If issues were found but not posted, check for the permission error above

Model Comparison

Model	Speed	Quality	Cost	Best For
Claude Sonnet 4	Fast	High	Medium	Everyday code reviews (recommended)
Claude Opus 4	Slow	Highest	High	Critical PRs, security-sensitive code
Claude Haiku 4	Very Fast	Good	Low	Quick reviews, large PRs with many files

Cost Estimation

Approximate API costs per file reviewed (varies by file size):

Model	~Cost per file	10-file PR
Haiku 4	~$0.002	~$0.02
Sonnet 4	~$0.01	~$0.10
Opus 4	~$0.05	~$0.50

These are rough estimates. Actual costs depend on the size of diffs and Claude's response length. Check Anthropic's pricing page for current rates.

Security

API keys are stored in Service Connections (encrypted, never visible in logs)
The task uses execFileSync (not shell execution) to prevent command injection
File paths are validated and sanitized
The task only reads code — it never modifies files or pushes changes
The System.AccessToken is used only for Azure DevOps API calls (read PR changes, post comments)

Limitations

Only Git repositories are supported (not TFVC)
Only Pull Request builds — the task skips if not triggered by a PR
Files larger than the configured max size (default 100KB) are skipped
Binary files are not reviewed
The AI may occasionally produce false positives — always review its suggestions critically
On-premise TFS requires outbound internet access to reach api.anthropic.com

Support

For issues or feature requests, please open an issue on GitHub.

Claude AI Code Review

Tsezari Mshvenieradze

Claude AI Code Review

How It Works

Prerequisites

Installation

Azure DevOps Services (Cloud)

Azure DevOps Server / TFS (On-Premise)

Setup Guide (Step-by-Step)

Step 1: Create the Claude API Service Connection

Step 2: Configure Build Service Permissions

Step 3: Enable OAuth Token Access

YAML Pipeline

Classic Pipeline

Step 4: Set Up the Pipeline

Option A: YAML Pipeline

Option B: Classic Pipeline

Step 5: Set Up PR Trigger

Azure DevOps Services (Cloud)

Azure DevOps Server / TFS (On-Premise)

Task Inputs Reference

Main Inputs

Advanced Inputs

File Filtering Inputs

Alternative: Direct API Key

What Claude Reviews

Example: PR Comment Output