Researchers at Google DeepMind have published AutoHarness (arXiv:2603.03329), a technique that uses Gemini-2.5-Flash to automatically synthesize code "harnesses" — runtime constraints that prevent LLM agents from taking illegal or prohibited actions in structured environments. The motivation is rooted in a concrete failure mode: in the Kaggle GameArena chess competition, 78% of Gemini-2.5-Flash losses stemmed not from poor strategy but from outright illegal moves, illustrating a persistent gap between a model's apparent rule comprehension and its actual action reliability. AutoHarness bypasses fine-tuning or manually written constraints, instead using the model's own code-generation capabilities to build its own guardrails.
The method frames harness synthesis as a search over program space, guided by Thompson sampling in a tree search. Gemini-2.5-Flash acts as a mutation operator, iteratively refining candidate harness code based on feedback from the game environment over a small number of rounds. The harness can take two forms: an action verifier that wraps the LLM in a rejection-sampling loop to filter out illegal moves before execution, or a code-as-policy variant that synthesizes an entire decision-making policy in code, removing the need to call an LLM at inference time altogether. Both approaches were evaluated across 145 TextArena games spanning single-player and two-player settings.
The action-verifier harness eliminated all illegal moves across all 145 TextArena games, allowing the smaller Gemini-2.5-Flash to outperform the larger Gemini-2.5-Pro. The code-as-policy variant went further, achieving higher average reward than both Gemini-2.5-Pro and GPT-5.2-High — OpenAI's high-compute reasoning configuration — on 16 single-player TextArena games, at lower cost since no LLM calls are needed at decision time. The paper is led by Xinghua Lou and co-authored by Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, and senior researcher Kevin P. Murphy, whose Bayesian background is reflected in the Thompson sampling at the core of the approach.
AutoHarness is the direct successor to the same team's "Code World Models for General Game Playing" (arXiv:2510.04542, October 2025), which used LLMs to translate game rules into executable Python for classical planning. The two papers trace a clear line: code not as an output format but as the <a href="/news/2026-03-14-percepta-ai-transformers-logarithmic-attention-inference">primary substrate for agent cognition</a>. Murphy, known for foundational work in probabilistic machine learning and a stated research focus on agents and decision-making under uncertainty, has described the direction as a long-term priority — though Google DeepMind has not made a formal statement on its investment in neurosymbolic approaches versus scale-based ones.