Researchers at Percepta AI have published a paper showing transformer neural networks can execute arbitrary programs internally using a novel attention mechanism that scales logarithmically with token count rather than quadratically. The work, led by Christos Tzamos and published March 11, 2026, introduces logarithmic attention — a mechanism that restricts computation to the convex hull of a 2D embedding space. Models using it can trace program execution, including text representations of registers and a call stack, while consuming compute that grows far more slowly than standard attention as context expands. The paper is an existence proof that vanilla transformers, given appropriately structured attention, can function as general computers without relying on external tool calls or symbolic co-processors.
For developers building reasoning-heavy agents, the mechanism opens a concrete path to speculative execution at inference time. A log-attention model proposes reasoning tokens or execution traces cheaply; a full-attention model verifies and accepts them — a pattern that mirrors dual-process cognitive architectures. That makes large-scale hypothesis generation and culling economically viable at inference time, directly targeting the cost bottlenecks that make chain-of-thought and tree-of-thought approaches expensive to run at scale.
Hacker News discussion surfaced two further implications. Interpretability researchers noted that if a meaningful portion of a model's computation occurs in this pseudo-symbolic regime — where internal state corresponds to legible registers and stack frames — auditing model behavior becomes considerably more tractable. The mechanism also offers a path to training data bootstrapping: rule-based or expert classifiers embedded into the computation graph via logarithmic attention can generate labeled synthetic data to fine-tune the model itself, lowering the barrier for resource-constrained organizations.
Tzamos brings an unusually rigorous theoretical background to this work. A 2017 Sprowls Award recipient for his MIT PhD under Constantinos Daskalakis — now Percepta's Chief Scientist — Tzamos previously held a faculty position at the University of Wisconsin-Madison and has authored more than 100 papers across STOC, FOCS, NeurIPS, and ICML. His prior work in halfspace geometry and efficient algorithms over structured input spaces is the foundation for the paper's convex-hull attention construction. Percepta, backed by General Catalyst and launched in October 2025, sits where theoretical computer science and frontier AI systems research share the most productive overlap — and this paper is evidence the combination produces more than positioning.