Key Findings
AI-written peer reviews outperformed top human experts in blind comparisons, according to a Harvard University working paper. In evaluations conducted by both a separate AI model and former journal editors, AI reviews won the large majority of matchups. Notably, AI-generated reviews contained approximately one-third the rate of incorrect claims compared to those written by humans.
Technical Details
Researchers compared human reviews from rapidPeer—a service where experts are paid well to provide 1.5 to 2 pages of feedback—against reviews generated by Anthropic’s Claude Opus 4.8. The dataset included 124 scholarly manuscripts and 134 human reviews from 2019 to 2026 across mathematical, statistical, and qualitative fields.
The AI was tested under two conditions: a “matched” condition using only the paper and reviewer instructions, and a “tuned” condition using a custom prompt and internet search access, averaging 12.4 searches per review. OpenAI’s GPT-5.5 served as the AI judge, ranking pairs across seven areas: theory and idea, methods, weaknesses, framing, communication, figures and tables, and sources. In a check providing the full manuscript, the AI review was preferred in at least 132 of 134 pairs.
Background & Context
Traditional peer review faces challenges including delays, recruitment difficulties, and human biases toward prestigious institutions. While previous 2026 studies yielded conflicting results—one showing human reviewers provided more varied points at BMJ Open, and another showing AI reviews were rated higher in a cardiology journal—this study specifically tested AI against experts performing at their best rather than unpaid referees.
Significance & Outlook
The results indicate that advanced AI can generate more thorough and constructive feedback than human experts operating under ideal conditions. However, the researchers note that widespread reliance on these tools could potentially make scientific thinking more uniform. The working paper has not yet been peer reviewed.
Get our weekly technology intelligence — free
Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.
Subscribe Free — Weekly Tech Intelligence
By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.
- Your email and selected fields are used only to deliver the newsletter.
- We never share your information with third parties.
- You can unsubscribe anytime via the link in each email.
See our Privacy Policy for details.
Takes about a minute · Unsubscribe anytime
