Pull Request Inspection Assistant (PRIA) - OpenAI PR Review Bot for Azure DevOps

Automate pull request (PR) reviews in Azure DevOps using the PR Inspection Assistant (PRIA) and OpenAI. This bot analyzes code changes, offers suggestions, detects potential bugs, and ensures adherence to coding standards. Streamline code reviews with customizable criteria and natural language feedback, improving code quality and reducing review time.

Key Features

• Automated PR Reviews: Leverage OpenAI to analyze code changes in pull requests.
• Supports Azure OpenAI: Isolate your reviews using your own internal model deployments in Azure AI Foundry.
• Code Quality Suggestions: Detect potential issues and ensure best practices are followed.
• Customizable Review Criteria: Tailor the bot to specific code quality metrics.
• Azure DevOps Integration: Seamlessly integrates with existing DevOps pipelines.
• Natural Language Feedback: Provides human-readable, actionable feedback.

Use Cases

• Automate Routine PR Tasks: Speed up the code review process by automating common review tasks.
• Improve Code Quality: Receive consistent, detailed feedback to enhance code quality.
• Early Bug Detection: Help developers understand best practices and identify bugs early in the development cycle.

Process

Prerequisites

• An OpenAI API Key
• Build Administrators must be given "Contribute to pull requests" access. Check this Stack Overflow answer for guidance on setting up permissions.

If Using Azure OpenAI

• A generated API key for you OpenAI service
• Your service endpoint URL e.g., https://my-resource.azure.openai.com/
• A deployed model
  • By default the task will try to use a deployment called 'o3-mini'
  • Valid options are:
    • 'o3-mini'
    • 'o1-mini'
    • 'o1-preview'
    • 'gpt-4o'
    • 'gpt-4'
    • 'gpt-3.5-turbo'
• A designated API version. Defaults to '2024-10-21'

Learn more about Azure AI Foundry

Getting Started

1. Install the PRIA DevOps Extension

Install the PRIA DevOps extension from the Azure DevOps Marketplace.

2. Create a PRIA Code Review Pipeline

Create an Azure DevOps Pipeline using the following YAML snippet to set up the PRIA code review task:

OpenAI API

trigger: none

jobs:
  - job: CodeReview
    pool:
      vmImage: 'ubuntu-latest'
    steps:
      - checkout: self
        persistCredentials: true
      - task: PRIA@2
        inputs:
          api_key: '$(OpenAI_ApiKey)'

Azure OpenAI

trigger: none

jobs:
  - job: CodeReview
    pool:
      vmImage: 'ubuntu-latest'
    steps:
      - checkout: self
        persistCredentials: true
      - task: PRIA@2
        inputs:
          api_key: '$(OpenAI_ApiKey)'
          api_version: '2024-10-21'
          api_endpoint: 'https://my-foundry-project.openai.azure.com/'
          ai_model: 'gpt-4o'

3. Optional Task Inputs

Additional input options can be set to tailor how the code is reviewed.

4. Configure your Main Branch for Build Validation

Note that pr triggers do not work in Azure repos.

Cofigure Azure DevOps branch policies to use the PRIA Code Review Pipeline created above as a build validation pipeline.

Build & Publish

1. Install Prequisites
2. Run npm install -g typescript to install TypeScript
3. Install dependencies and build extension

# Navigate to src and install
$ cd pr-inspection-assistant/src
$ npm install

# Build typescript and package .vsix file
$ npm run package

Resources

• Marketplace Pipeline Extension
• Publisher Portal