Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learning

The High Stakes of AI Red Teaming

The rapid deployment of Large Language Models (LLMs) across every critical infrastructure—from automated code generation to proprietary corporate intelligence—has outpaced our ability to secure them. Currently, the primary defense against model misuse is human-led “red teaming.” While effective, these expert probes are prohibitively expensive, with costs typically ranging from $50,000 to$150,000 per exercise.

This financial bottleneck has forced a reliance on automated red teaming, but most current frameworks are fundamentally brittle. They remain “single-shot” tools that attempt to trigger a safety violation with one isolated prompt. This approach ignores the reality of modern adversarial behavior. Real-world attackers do not simply launch a single “fortress-cracking” query; they use iterative, multi-turn dialogues to “seep” through guardrails, negotiating compliance through persistent adaptation.

A new framework called GALA (Global and Local Learning Agent) is exposing these systemic failures. By emulating the inductive learning of a human adversary, GALA achieves over 90% attack success rates against state-of-the-art models, proving that even “aligned” models are vulnerable when an attacker can learn from every refusal.

The Multi-Turn Shift: Why Single-Shot Attacks Fail

Existing security benchmarks often view safety as a binary state, but from a “failure-first” perspective, guardrails that hold against isolated prompts often crumble under the pressure of persistent dialogue. Adversaries probe, build context, and refine their approach based on the model’s feedback.

The following table contrasts the limitations of traditional testing with GALA’s adaptive approach:

Feature	Single-Turn Attacks	Multi-Turn Attacks (GALA)
Adaptivity	Static; fails if the initial prompt is blocked.	Dynamic; computes “optimization directions” from feedback.
Attack Space	Limited to individual, isolated prompts.	Expansive; leverages sequences and strategic timing.
State Tracking	Not required (stateless).	Rigorous; tracks conversation progress in JSON belief states.
Adversarial Goal	Immediate, blunt exploitation.	Gradual escalation and deceptive roleplay.

In the multi-turn arena, a model’s refusal is not the end of the encounter; it is a signal that the attacker uses to refine their next move. When an adversary can iteratively probe a model, safety becomes a negotiation rather than a fixed boundary.

GALA’s Secret Weapon: Dual-Level Learning

GALA’s superiority stems from its move beyond “fixed strategy sets.” While earlier frameworks like GOAT or Crescendo rely on predefined tactics, GALA utilizes inductive learning across two specific dimensions:

• Global Tactic-wise Learning: The agent accumulates knowledge over time, generalizing which high-level strategies work best for specific misuse categories (e.g., malware vs. disinformation). It initializes with a robust Initial Knowledge Base including tactics such as Echoing, Hidden Intention Streamline, Injection, and Obfuscation.
• Local Prompt-wise Learning: This is GALA’s technical “killer app.” When a tactic is correct but the phrasing fails, the agent employs Automatic Text Differentiation. It treats the model’s refusal as a gradient signal, conceptually computing: $\frac{\partial \text{Evaluation}}{\partial \text{Prompt}} = \frac{\partial \text{Evaluation}}{\partial \text{Prediction}} \circ \frac{\partial \text{Prediction}}{\partial \text{Prompt}}$ By analyzing the refusal, GALA identifies specific “optimization directions” to bypass the filter without changing the underlying tactical goal.

The Workflow: How GALA Thinks and Adapts

GALA utilizes the Qwen2.5-72B-Instruct model as its primary “attacker” brain, chosen for its superior reasoning and ability to craft complex plans. The workflow operates in a distinct loop where strategic planning and belief updates happen in-turn, while the learning module functions as a post-trial process.

1. Planning Module: The agent selects a tactic and provides a rationale, identifying specific information points to extract.
2. Belief Update Module: This in-turn module tracks progress in a structured JSON “Belief State,” ensuring the agent maintains context.
3. Learning Module: A post-trial reflection phase where successful new tactics are recorded and failed implementations generate refinement suggestions for the next attempt.

Components of a GALA Belief State:

• Conversation Progress: A quantified progressScore (0 to 1) and a list of missingInformation points.
• Strategy State: A comprehensive record of tacticsTried to avoid repetitive, ineffective patterns.
• Response Analysis: Categorizes feedback as “partial compliance,” “direct refusal,” or “full compliance,” identifying specific triggered safety policies.

Empirical Results: Stress-Testing GPT-3.5 and Llama-3.1

GALA was stress-tested against the JailbreakBench gauntlet, a benchmark covering ten misuse categories. Within just five conversation turns, GALA achieved a near-total collapse of safety guardrails across both open and closed-source models.

Target Model	Attack Success Rate (ASR)	Diversity Score
GPT-3.5-Turbo-0125	91%	0.23
Llama-3.1-8b	87%	0.24
Llama-3.1-70B	92%	0.23

Significantly, GALA’s Diversity Score was 28% higher than GOAT’s (the leading baseline). This metric is critical; it is measured by calculating the semantic distance of entire attack trajectories using MiniLMv2 embeddings. This proves GALA isn’t just finding a “lucky” prompt; it is identifying a broad spectrum of vulnerability modes that human-crafted templates miss.

The Discovery of “New” Jailbreak Tactics

GALA’s most alarming capability is its ability to invent and formalize novel attack strategies not present in its initial training. It doesn’t just use its knowledge base; it expands it.

Direct Language Amplification: This tactic involves using direct and pointed language to amplify the harmful message, making it more convincing and impactful. It is often used in conjunction with other tactics to create a more persuasive and realistic outcome.

Private Candid Discussion: This tactic involves framing the harmful content as a private, candid discussion between trusted colleagues… allowing for the expression of harmful or biased opinions without the usual professional guardrails.

Conclusion: Reimagining AI Safety

The emergence of GALA signals a paradigm shift in AI safety. If an automated agent can learn to negotiate compliance and invent new tactics in real-time, our defensive strategies must become equally dynamic.

Key Takeaways for Developers:

• The Obsolescence of Static Safety Benchmarks: Single-turn testing is no longer a valid metric for model safety in production environments.
• The Necessity of Multi-Round Persistence: Models must be stress-tested for their ability to maintain safety across extended, adaptive dialogues.
• Automated Inductive Learning is the New Red Team: The scalability of GALA proves that manual templates cannot compete with the “differentiation-style” refinement of LLM-based attackers.
• Roleplay as an Alignment Bypass: Deceptive framing remains the most effective vector for bypassing RLHF-based alignment, particularly when escalated over multiple turns.

The future of AI safety lies not in building higher walls, but in developing defensive models that can detect and counter the adaptive, “seeping” nature of multi-turn adversaries. GALA has provided the blueprint for the next generation of attacks; the defensive community must now respond in kind.

Read the full paper on arXiv · PDF