Kibana PR Reviewer

A VSCode/Cursor extension for Kibana engineers to review pull requests efficiently.

Why?

Kibana has a large influx of new PRs. The introduction of AI assisted coding already makes the influx even higher.

Traditionally PRs are reviewed via the Github UI. While this is good for some use cases, it could be way better. At the time of writing, Github's UI has been largely stagnant for years.

Data needed for a good review

There is not an insignficant amount of data points that an engineer (or an LLM) needs to fully judge the quality of a PR.

A reviewer needs:

• the diffs
• the context in which the diffs are placed (the entire file)
• quick access to the existing files and folders surrounding the diffs
• type information
• information on test coverage
• how the proposed feature behaves functionally, at runtime

Therefore, strictly reviewing from the Github UI means a reviewer has not enough data to come to a good judgement. As such, this strategy leads to low quality reviews.

Friction when reviewing in Kibana

Some reviewers go further than a look at the code and actually check out the branch and see how the feature behaves. While some do this, many do not.

One of the reasons that contribute to reviewers not checking out PRs is due to the friction that is imposed on the reviewer in getting a Kibana PR ready to review.

A reviewer needs to:

• find the PR that she needs to review
• switch to the branch
• run yarn kbn bootstrap
• start es + kibana
• optionally load test data / fixtures
• understand the code
• understand the intended behavior
• judge the code in terms of coding standards
• judge the code in context of the product
• judge if the code covers both happy and unhappy paths

This, especially the getting started parts, is slow an unwieldy.

The speed of bootstrapping / starting Kibana we can't fix easily, but we can improve our tooling.

This is where this extension comes in.

Features

• Team-filtered PR queue — shows only PRs where your team is a requested reviewer

• One-click PR checkout — runs gh pr checkout <number> for you

• Quickly start ES, Kibana and populate with test data

• Analyze the diff using an LLM — provide a summary of every changed files so you quickly build context on what the changed files actually do. Also provide two sorting strategies for the order in which to review: top down and bottom up.

• Inline diff view — click any file to open a color-highlighted diff, read and reply to comments by authors and reviewers

• Approve, comment or request changes — Get the full discussion inside the IDE

Prerequisites

• GitHub CLI (gh) installed and authenticated (gh auth login)
• Authenticated against the elastic org for team detection

Getting Started

1. Build and install:

   cd dev_tools/kibana-pr-reviewer
   npm install
   npm run compile
   # Package as .vsix:
   npm run package
   # Then: Extensions panel → Install from VSIX...
   

2. Open the panel — click the PR Reviewer icon in the Activity Bar (left sidebar).

3. First run — the extension auto-detects which @elastic/team-* teams you belong to via the GitHub API. This is cached. Override via settings if needed.

Configuration

Open Settings > Kibana PR Reviewer:

Setting your LLM API key

Run > Kibana PR Reviewer: Set LLM API Key from the command palette. The key is stored in VS Code's encrypted SecretStorage.

How File Ordering Works

Files are ordered from most important to review first:

1. Core implementation — non-test .ts/.tsx source files
2. Entry points & APIs — index.ts, plugin.ts, route handlers
3. Tests — .test.ts, .spec.ts (review after you understand the change)
4. Configuration — package.json, kibana.jsonc, tsconfig.json
5. Documentation — *.md, *.mdx

With an LLM API key configured, the extension sends the PR title, description, and file list to the model and uses its judgment to refine the ordering, annotating each file with a group label (e.g. "core change", "supporting").

Development

npm install
npm run watch   # rebuild on change

# In VS Code: F5 to launch Extension Development Host