No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions

1. Introduction: The Automated Gatekeeper’s Vulnerability

As major academic conferences move toward integrating artificial intelligence into their peer-review infrastructure—notably AAAI 2026 trialing LLM-generated reviews and ICLR 2025 deploying review feedback agents—the security of these automated systems has become a primary concern for the research community. While most robustness research has focused on “explicit” attacks like prompt injection or hidden text designed to bypass system instructions, a study by Yang et al. explores a more subtle and policy-relevant failure mode: presentation-level manipulation.

The paper, titled “No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions,” investigates whether AI reviewers can be manipulated through legitimate academic writing practices. The authors establish a core thesis: a necessary condition for the safe deployment of AI review systems is “resistance to presentation-only review gaming.” This principle dictates that an AI reviewer’s evaluation should not systematically improve if the underlying scientific evidence remains unchanged. The findings suggest that current mainstream models fail to meet this standard.

2. The Methodology of Adversarial Repackaging

To test the robustness of AI reviewers, the authors developed “Adversarial Repackaging,” a closed-loop iterative system that uses an AI reviewer’s own feedback to optimize a paper’s presentation. To ensure that the scientific content remained fixed while the narrative was optimized, the authors partitioned the paper source into three specific editing zones:

Editing Zone	Content Type	Constraints
Free Zone	Abstract, Introduction, Related Work, Discussion, Conclusion	Narrative Framing: May be rewritten or reframed; narrative-level changes allowed.
Limited Zone	Method descriptions, result analysis	Technical Exposition: Rephrasing and reorganization allowed, but factual content must be preserved.
Fixed Zone	Experimental data, tables, figures, equations, proofs, numerical results	Scientific Evidence: Immutable. No changes permitted to the scientific evidence.

The system operates through a six-stage attack loop:

1. Profile: Extracting structured signals and recurring perceptions from AI reviews.
2. Plan: Mapping signals to specific presentation strategies (e.g., expanding discussion or reframing contributions).
3. Edit & Compile: Modifying the LaTeX source and compiling a new PDF.
4. Review: Generating independent reviews of the new version.
5. Evaluate: Comparing the new reviews against the baseline using a pairwise judge.
6. Update: Retaining the version only if it shows a confirmed improvement in review outcome.

3. Core Findings: Success Rates and Score Inflation

The authors tested this framework against three mainstream models: Claude Sonnet 4, Claude Sonnet 4.5, and GPT-5-mini. The results indicate a significant vulnerability to presentation-level optimization. Across the dataset, the attack achieved a 75.1% attack success rate, with a mean score gain of +1.21 out of 10. Statistical analysis confirmed that all score gains ($\Delta S$) were significant using the Wilcoxon signed-rank test ($p < 0.0001$).

These improvements were not the result of ordinary prose polishing. Most importantly, these gains were achieved despite the “Fixed Zone” remaining entirely untouched. The authors did not change a single data point, equation, or experimental result; the score inflation was driven entirely by how existing evidence was presented and framed to the model.

4. The Strength-Weakness Asymmetry: “Easier to Impress Than Convince”

A critical structural failure mode identified in the research is the asymmetry in how AI reviewers process positive versus negative feedback. The authors found that AI reviewers respond predictably to the amplification of strengths, but their responses to the resolution of weaknesses are highly unstable.

While perceived strengths improved in 86.1% of attack rounds, attempts to address criticisms had a 31.6% backfire rate. In these instances, the AI reviewer actually rendered harsher judgments after the authors attempted to resolve a stated weakness.

The paper also identifies a “Swamping Effect,” where the AI reviewer’s aggregate judgment is overwhelmed by positive signals. In 15.8% of rounds where the overall score increased, the AI reviewer simultaneously identified worsening weaknesses. This suggests that the models reward the salience of newly highlighted strengths more readily than they penalize—or even recognize the worsening of—technical gaps.

Key Takeaway: AI reviewers are significantly more sensitive to the presentation of strengths than to the actual resolution of identified weaknesses. They can be “swamped” into giving higher scores by narrative framing even when they concurrently recognize that the paper’s flaws have become more severe.

5. Limitation Laundering and the Effectiveness Gradient

The authors describe a phenomenon called “Limitation Laundering,” where unchanged scientific evidence is reinterpreted as a stronger contribution through narrative manipulation. Two primary mechanisms were observed:

• Reframing Scarcity as Design Intent: Converting a weakness (e.g., “only two datasets used”) into a stated design choice (e.g., “focusing on complementary data regimes”).
• Decoy Limitations: Proactively discussing minor limitations to steer the AI reviewer away from fundamental criticisms, “anchoring” the reviewer to points the author has already addressed.

Yang et al. note that these upgraded assessments often involve “Review Parroting,” where the AI reviewer adopts the author’s own self-positioning language and narrative framing as its own independent evaluation.

The study further identified a “Strategy Effectiveness Gradient,” revealing that AI reviewers are systematically sensitive to specific rhetorical signals rather than general writing quality. Based on the “accepted exposure rate”—the proportion of rounds where a strategy led to an improved version—the authors ranked the strategies by effectiveness:

1. Related Work Repositioning (49.3%): Anchoring the work more favorably within existing literature.
2. Discussion Expansion (44.9%): Expanding analytical depth to suggest higher significance.
3. Self-Deprecation Removal (40.0%): Deleting modest framing or admitted minor flaws.
4. Abstract Reframing (37.4%): Rewriting the high-level narrative.
5. Contribution List Enhancement (36.8%): Using structured language to upgrade novelty claims.
6. Table Formatting (29.8%): Improving the visual layout of results.
7. Local Text Polishing (27.8%): Minor syntactic and stylistic improvements.
8. Algorithm Boxes (26.5%): Formatting technical steps into formal boxes.

6. The Benchmarking Solution: A Rolling Dataset

To facilitate further research, the authors contributed a “contamination-free, rolling benchmark.” This dataset utilizes unpublished arXiv preprints paired with their LaTeX sources. Because the pipeline is automated, it can be re-executed to include the most recent submissions, ensuring the test data remains outside the training sets of the models being evaluated.

By using compiled PDFs and full official conference guidelines from ICLR, NeurIPS, and ICML, this methodology provides a more realistic evaluation of AI reviewer performance than previous studies that relied on simplified prompts or plain-text inputs.

7. Conclusion: Implications for Failure-First AI Safety

The findings of Yang et al. suggest that the deployment of AI reviewers carries risks beyond malicious instructions. The core vulnerability lies in the emergence of paper presentation as an “optimization surface” that can be gamed through legitimate academic writing. This creates a systemic shift where rhetorical skill may replace scientific merit in automated evaluation pipelines.

Key Insights for Researchers:

• Optimization of the Surface: AI reviewers can be systematically influenced by changes in framing, even when the underlying scientific merit is static.
• Structural Deficiency: Current models struggle to distinguish between the “appearance of resolution” (rhetorical framing) and “scientific resolution” (new data or proofs), leading to Review Parroting.
• Safety Standards: Robustness standards for AI in peer review must look beyond prompt injection to account for visible, legitimate writing strategies that distort evaluation outcomes.
• Incentive Distortion: As AI reviews are deployed at scale, there is a safety risk that authors will shift efforts away from research toward the “repackaging” of papers to satisfy the exploitable sensitivities of automated reviewers.

The authors conclude that without meeting the condition of resistance to presentation-only gaming, the integrity of the scientific record remains vulnerable to systematic distortion by automated gatekeepers.

Read the full paper on arXiv · PDF