Skip to content

fix(ci): make AI PR review bots concise and actionable#1601

Merged
imran-siddique merged 2 commits into
microsoft:mainfrom
imran-siddique:fix/concise-ai-reviews
Apr 29, 2026
Merged

fix(ci): make AI PR review bots concise and actionable#1601
imran-siddique merged 2 commits into
microsoft:mainfrom
imran-siddique:fix/concise-ai-reviews

Conversation

@imran-siddique
Copy link
Copy Markdown
Contributor

Summary

Makes AI PR review bot output concise and scannable instead of verbose essay-style.

Changes

ai-pr-review.yml:

  • Code-reviewer: enforces TL;DR + table format, 200 word limit, no essays/conclusions/suggestion lists
  • Test-generator: top 3-5 gaps only, 300 word limit, no exhaustive file-by-file analysis
  • Both agents: max-tokens reduced from 4000 to 2000
  • Verdict parser: recognizes 'Block' severity alongside 'Critical'

copilot-instructions.md:

  • Added Review Output Style section with the concise format standard
  • All future reviews (human and bot) follow: TL;DR, summary table, 1-line actions, no nits

Why

Bot reviews on PRs like #1594 produced 500+ word essays with multi-paragraph explanations, 8-item suggestion lists, and conclusion paragraphs. Contributors need scannable verdicts, not literature.

Type of Change

  • Maintenance (CI/CD, refactoring)

Imran Siddique and others added 2 commits April 29, 2026 16:28
- Rewrite code-reviewer custom instructions to enforce TL;DR + table format
- Reduce max-tokens from 4000 to 2000 for code-reviewer and test-generator
- Add explicit word limits (200 words code-reviewer, 300 words test-generator)
- Forbid essay-style output: no Conclusion sections, suggestion lists, or nits
- Test-generator now lists only top 3-5 gaps instead of exhaustive file-by-file analysis
- Update verdict parser to recognize 'Block' severity alongside 'Critical'
- Add Review Output Style section to copilot-instructions.md for persistence

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation scripts/ci/cd labels Apr 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

🤖 AI Agent: code-reviewer — View details

No issues found. Clean change.

@github-actions github-actions Bot added the size/XS Extra small PR (< 10 lines) label Apr 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

🤖 AI Agent: docs-sync-checker — Docs Sync

Docs Sync

  • copilot-instructions.md -- missing CHANGELOG entry for added "Review Output Style" section

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

🤖 AI Agent: breaking-change-detector — View details

No breaking changes detected.

@imran-siddique imran-siddique merged commit 0978dd5 into microsoft:main Apr 29, 2026
7 checks passed
@imran-siddique imran-siddique deleted the fix/concise-ai-reviews branch April 29, 2026 23:30
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

PR Review Summary

Check Status Details
🔍 Code Review ✅ Passed No issues found
🛡️ Security Scan ✅ Completed Analysis complete
🔄 Breaking Changes ⚠️ Warning See details
📝 Docs Sync ✅ Completed Analysis complete
🧪 Test Coverage ✅ Completed Analysis complete

Verdict: ⚠️ Ready for human review

@github-actions
Copy link
Copy Markdown

🤖 AI Agent: security-scanner — View details

No security issues found.

@github-actions
Copy link
Copy Markdown

🤖 AI Agent: test-generator — View details

Test coverage looks good. No gaps identified.

MohammadHaroonAbuomar pushed a commit to MohammadHaroonAbuomar/agt-acs that referenced this pull request Jun 1, 2026
- Rewrite code-reviewer custom instructions to enforce TL;DR + table format
- Reduce max-tokens from 4000 to 2000 for code-reviewer and test-generator
- Add explicit word limits (200 words code-reviewer, 300 words test-generator)
- Forbid essay-style output: no Conclusion sections, suggestion lists, or nits
- Test-generator now lists only top 3-5 gaps instead of exhaustive file-by-file analysis
- Update verdict parser to recognize 'Block' severity alongside 'Critical'
- Add Review Output Style section to copilot-instructions.md for persistence

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation scripts/ci/cd size/XS Extra small PR (< 10 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant