Skip to content

README.md

Latest commit

 

History

History
165 lines (116 loc) · 5.35 KB

README.md

File metadata and controls

165 lines (116 loc) · 5.35 KB
Raw

ProReviewer Skill — Evidence-Based Paper Review

ProReviewer skill used by CLI agents to review scientific papers — a structured, evidence-based approach developed by the ProReviewer project.

What It Does

Instead of generating surface-level impressions, ProReviewer teaches the agent to:

  1. Extract and verify author claims — log what the authors assert, flag issues, and verify each claim against the paper's own evidence.
  2. Investigate systematically — raise questions about suspicious or missing content, then resolve every question through reading.
  3. Ground every review point — each strength, weakness, and question in the final review must cite specific evidence (equations, tables, figures, sections).
  4. Self-check quality — evaluate review points against five dimensions: analytical depth, grounding specificity, actionability, verifiability, and helpfulness.

Repository Contents

proreviewer/
├── SKILL.md              # Core skill: method overview, log operations, quality standards
├── review_cli.py         # CLI tool: PDF/LaTeX conversion + ReviewLog operations
├── reviewer_memory.py    # ReviewLog data structures and API
└── __init__.py           # Package init for module imports
agents/
└── openai.yaml           # Codex CLI agent configuration

Prerequisites

pip install pydantic pymupdf4llm

Installation

Claude Code

Global (available in all projects):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p ~/.claude/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer ~/.claude/skills/

Project-level (available only in this project):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p .claude/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer .claude/skills/

Codex CLI

Global (available in all projects):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p ~/.agents/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer ~/.agents/skills/

Project-level (available only in this project):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p .agents/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer .agents/skills/

Gemini CLI

Global (available in all projects):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p ~/.gemini/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer ~/.gemini/skills/

Project-level (available only in this project):

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git
mkdir -p .gemini/skills
cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer .gemini/skills/

One-liner (cross-platform project install)

Codex CLI and Gemini CLI both scan the shared .agents/skills/ directory at the project level:

git clone https://github.com/UKPLab/arxiv2026-ProReviewer.git && mkdir -p .agents/skills && cp -R arxiv2026-ProReviewer/ProReviewer-Skill/proreviewer .agents/skills/

Usage

Once installed, ask the assistant to review a paper. The prompt works with any supported input format:

# PDF
/proreviewer papers/submission.pdf

# Single LaTeX file
/proreviewer papers/main.tex

# LaTeX project directory (auto-finds main.tex)
/proreviewer papers/my-paper/

# Markdown
/proreviewer papers/draft.md

# With a specific conference scale
/proreviewer papers/submission.pdf --conference icml

Or use natural language:

Please review the paper at papers/submission.pdf for ICML

The agent will:

  1. Convert the paper (PDF, LaTeX, or Markdown) via review_cli.py init
  2. Read sections one at a time, operating the ReviewLog after each read
  3. Output a structured review with evidence-tagged strengths, weaknesses, questions, and an overall score

Supported Input Formats

Format Extension Notes
PDF .pdf Converted to markdown via pymupdf4llm (recommended) or pymupdf
LaTeX .tex \input{} and \include{} resolved recursively
LaTeX project directory Auto-finds main.tex, paper.tex, or manuscript.tex
Markdown .md Used as-is

Output

A review session produces three artifacts:

File Description
The review (printed to console) Structured review with Summary, Strengths, Weaknesses, Questions for Authors, and Overall Score. Every strength and weakness cites evidence tags (e.g., [C1, N3], [C5, Q2]).
review_log.json The ReviewLog — full investigation trail recording every claim extracted, question raised, and note recorded during reading. This is the evidence backing the review.
paper.md The converted paper in markdown. Generated by init from the input PDF, LaTeX, or Markdown source.

Supported Conference Scales

Conference Scale Source
iclr (default) {0, 2, 4, 6, 8, 10} ICLR 2026
icml 1–6 ICML 2026
neurips 1–6 NeurIPS 2026
acl 1–5 (0.5 steps) ACL Rolling Review
emnlp 1–5 (0.5 steps) EMNLP / ARR

Set the scale at init: python review_cli.py init paper.pdf --conference icml

Attribution

This skill is based on the ProReviewer method — an RL-trained, evidence-based peer-review agent for scientific papers.

License

MIT — see LICENSE.