MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correction

1. Introduction: The High Stakes of Clinical LLMs

As large language models (LLMs) move toward clinical decision-support settings, factual errors can pose safety risks even when the surrounding prose remains fluent. Ma et al. study this mismatch between linguistic coherence and clinical factuality; they do not evaluate MedGuards in a deployed clinical system.

In response to these vulnerabilities, the authors introduce MedGuards, a training-free, multi-agent framework designed to function as a medical safety guardrail. By reframing error detection and correction as a collaborative, in-context learning (ICL) task, MedGuards seeks to enhance the failure-resilience and auditability of LLM deployments in high-stakes clinical workflows.

2. The Problem: Why Current Quality Control Fails

Traditional quality control (QC) mechanisms for Natural Language Processing (NLP) are fundamentally misaligned with the requirements of clinical safety. Ma et al. identify a significant lack of structural understanding and out-of-distribution (OOD) robustness in existing automated checks. Furthermore, standard evaluation frameworks fail to prioritize the specific tokens that dictate patient outcomes.

The paper argues that traditional metrics—BLEU, ROUGE, and BERTScore—are insufficient for clinical validation due to the following structural flaws:

• Token Equality Assumption: Standard metrics treat all tokens with uniform weight, failing to distinguish between minor grammatical filler and clinically critical entities (e.g., dosage or drug names).
• Semantic Over-Reliance: High similarity scores can be maintained even when a dangerous factual swap occurs, such as substituting “Hepatitis A” for “Schistosoma mansoni.”
• Lack of Domain-Specific Sensitivity: Semantic overlap does not guarantee clinical correctness; changing a diagnosis can preserve much of the surrounding sentence while making the clinical content wrong.

3. MedGuards Architecture: A Three-Pillar Approach

The MedGuards framework utilizes three core principles to achieve variance reduction and improved error recovery without necessitating model retraining. The authors describe the functional logic of the system as a conditional pipeline where the overall task is decomposed as: $d = f_{Det}(x)$, $s = f_{Loc}(x | d)$, and $\hat{s} = f_{Corr}(s | x)$.

1. Chain-of-Thought (CoT) Decomposition: The framework explicitly articulates intermediate reasoning steps by breaking the process into three distinct stages: Detection (identifying the presence of an error), Localization (isolating the specific erroneous sentence), and Correction (generating a clinically valid revision conditioned on the localized sentence $s^*$ and the full context $x$).
2. Multi-Agent Self-Consistency: To mitigate the brittleness of individual model predictions, the paper proposes a redundant architecture where multiple agents process the same input independently. This ensemble approach treats agent disagreement not as noise, but as a trigger for higher-order deliberation.
3. ICL-Based Arbitration: When base agents conflict, a third “Arbitration Agent” resolves the disagreement. This agent is model-agnostic and utilizes In-Context Learning to synthesize a final decision based on the evidence provided by its peers.

4. The Mechanics of Confidence-Guided Arbitration

The arbitration process is central to the framework’s robustness. In instances where Agent 1 and Agent 2 disagree during the detection or localization phases, the Arbitration Agent ($Agent_{3}$) is invoked.

According to the authors, the arbitrator reaches a consensus by analyzing an input prompt that concatenates the original text with the reasoning traces and confidence scores (1–100) provided by the initial agents. Formally, the decision is reached via $d = Agent_{3}(x \parallel (d_1, r_1, c_1), (d_2, r_2, c_2))$, where $r$ represents the reasoning trace and $c$ the confidence score. These traces serve as “demonstrations” within the ICL framework, allowing the arbitrator to deliberate over conflicting rationales.

In the localization phase, the framework uses SequenceMatcher to map a generated sentence to the most similar source sentence by character-level similarity. This is the paper’s soft-matching procedure, not exact-string alignment: it accommodates paraphrased model output and outperforms hard index matching in the reported experiment.

5. Evaluating Accuracy: The Keyword-Prioritized Correction Score (KPCS)

To address the “Token Equality” failure mode, the authors introduced the Keyword-Prioritized Correction Score (KPCS). This domain-adapted metric prioritizes the presence of critical medical entities to ensure patient safety.

Step	Phase	Action
Step 1	Keyword Check ($K$)	Verifies the presence of critical clinical entities (diagnoses, medications) in the corrected output.
Step 2	Similarity Check ($M$)	Calculates the average of ROUGE-1, BERTScore, and BLEURT to assess linguistic fluency.
Step 3	Weighted Integration	Combines scores: $KPCS = \alpha \cdot K + (1-\alpha) \cdot M$.

The paper utilizes a default balance factor of $\alpha = 0.5$ to align the metric with human expert evaluations. This weighting ensures that a model which changes a diagnosis from “Schistosoma mansoni” to “Hepatitis A” is heavily penalized for the missing diagnostic keyword, despite having high general semantic similarity ($M$).

6. Experimental Results and Multilingual Performance

Ma et al. evaluated MedGuards across four datasets—MEDEC and MedErrBench—covering English, Arabic, and Chinese. The framework was tested using varied backbones, including Gemini 2.0 Flash, GPT-4o-mini, Doubao-1.5-thinking-pro, and Deepseek-V3.

• Benchmark Performance: MedGuards significantly outperformed established baselines. For example, using Doubao-1.5 as a backbone, the framework achieved 0.770 detection accuracy and 0.716 localization accuracy, surpassing knowlab_AIMed and Medifact.
• Architecture Gains: GPT-4o-mini demonstrated a 66.2% average improvement across all metrics when integrated with MedGuards compared to the standalone model.
• Matching Robustness: The authors highlight the “Soft vs. Hard Matching” results in localization. Soft matching (aligning generated text with the most similar reference sentence) achieved 0.61 accuracy for Gemini, vastly outperforming Hard matching (0.36), which suffered from the model’s inability to strictly match sentence indices.

7. Discussion: Failure Modes and Practical Deployment

The authors’ analysis of self-consistency rates reveals the complexity of medical error correction. Arbitration was required in 11.6% to 20.2% of detection cases and up to 37.8% for localization. The authors interpret this as evidence suggesting that localization is more ambiguous than detection; the disagreement rate alone does not establish that conclusion.

From a systems architecture perspective, MedGuards introduces a safety-performance trade-off. Integrating the framework increases latency (e.g., from 1.61s to 6.99s for Gemini 2.0 Flash) and token consumption. However, the paper suggests that the cost of multiple agents (noting Gemini 2.0 Flash input rates of $0.10 per 1M tokens) can be mitigated through context-caching, which is compatible with the multi-agent design where agents share the same base clinical note.

8. Conclusion: Key Takeaways for AI Safety

Ma et al. present several critical findings for safety researchers and clinicians focused on deploying resilient medical AI:

• Robustness via Redundancy: Disagreement between agents serves as a critical “safety signal,” triggering higher-order deliberation rather than defaulting to a single, potentially erroneous output.
• Inspectability Through Reasoning Traces: Decomposition exposes intermediate rationales that may aid inspection, but the paper does not establish that clinicians can reliably audit those traces in practice.
• Domain-Prioritized Metrics: Safety-critical deployments require metrics like KPCS that penalize the omission of “ground truth” clinical entities, as traditional NLP similarity metrics can mask dangerous factual errors.
• Proof of Concept, Not a Deployment Validation: The training-free design avoids model retraining, but the authors frame MedGuards as methodology-oriented proof of concept rather than a system ready for immediate clinical integration. They call for controlled clinical studies and human-in-the-loop clinician review before deployment.

The paper’s human evaluation covered 80 cases reviewed by two doctors; exact score agreement was 57.5%. That limited agreement reinforces the need for further validation rather than supporting autonomous clinical use.

Read the full paper on arXiv · PDF