AI PR Review Agent

Overview How it works Features LLM Providers Project Structure Setup Configuration Usage CLI Reference Example Review Custom Rules Troubleshooting Benefits

AI Pull Request Review Agent

Automated GitHub pull request review agent that analyzes diffs using LLMs and posts structured review comments directly on the PR.

AI GitHub Code Review Automation

Overview

Code review is one of the most valuable yet time-consuming parts of development. This tool automates the first pass of PR reviews by analyzing code diffs and applying engineering rules.

  • Provider-agnostic LLM integration
  • CI/CD friendly CLI
  • Custom rule engine
  • Duplicate review prevention
  • Fully configurable through .env

How It Works

GitHub PR URL
│
▼
Fetch PR files via GitHub API
│
▼
Chunk large diffs into LLM-friendly segments
│
▼
Build prompt using engineering rules
│
▼
Send chunks to selected LLM provider
│
▼
Receive structured JSON review
│
▼
Filter and snap comments to valid diff lines
│
▼
Deduplicate previous reviews
│
▼
Post inline comments + summary to GitHub

Features

Feature Description
Multi-provider support OpenAI, Gemini, Claude, Groq, Grok, Bedrock
Structured JSON output Summary + inline review comments
Diff chunking Handles large PRs automatically
Comment snapping Fixes slightly inaccurate LLM line numbers
Review deduplication SHA-256 fingerprint prevents duplicate reviews
Draft review support Post GitHub draft reviews
Dry run Generate review without posting
Custom rules Load engineering rules via JSON

Supported LLM Providers

Provider Flag Env Var
Google Gemini gemini GEMINI_API_KEY
Groq groq GROQ_API_KEY
OpenAI openai OPENAI_API_KEY
Anthropic Claude claude ANTHROPIC_API_KEY
AWS Bedrock bedrock AWS_ACCESS_KEY_ID
xAI Grok grok GROK_API_KEY

Project Structure


ai-review-agent/
 ├ main.py
 ├ rules/
 ├ scripts/
 └ pr_review_agent/
     ├ github/
     ├ llm/
     ├ models/
     └ review/

Setup


git clone https://github.com/your-org/ai-review-agent.git
cd ai-review-agent

python -m venv .venv
source .venv/bin/activate

pip install -r requirements.txt

Configuration


GITHUB_TOKEN=your_token

GEMINI_API_KEY=
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GROQ_API_KEY=

LOG_LEVEL=INFO
HTTP_TIMEOUT_SECONDS=30

Usage


./scripts/run_review.sh \
 --pr_url https://github.com/org/repo/pull/42 \
 --provider groq

CLI Reference

Flag Description
--pr_url GitHub Pull Request URL
--provider LLM provider
--model Override default model
--dry_run Print review JSON only

Example Review Output


Wallet Service Code Review

Critical issues
- Missing validation
- Potential NPE

Major issues
- Missing transactional annotation

Inline comments
- 3 comments added

Custom Review Rules


{
 "categories":[
  {
   "name":"API Design",
   "rules":[
    "REST endpoints must use nouns",
    "pagination required"
   ]
  }
 ]
}

Troubleshooting

403 Forbidden
Your GitHub token must have Pull Requests Read and Write permission.
Gemini 503 Error
Switch to a stable model like gemini-2.0-flash.

Benefits

AI Pull Request Review Agent Documentation