Stanford and Together AI: hybrid agents cut 62.5% to 72.9% of decoding costs, win benchmark

The prevailing assumption for LLM agents today is straightforward: the model must write every reasoning step as visible tokens before it calls a tool, runs code, or acts in the world. A new paper from Stanford and Together AI breaks that assumption. HybridAgent adds a small adapter that feeds the model's internal state (hidden states) back as the next input, so routine reasoning takes place in the latent vector space without writing a word. Only tool calls and key decisions switch to text, and even the silent steps can be decoded afterward for human readability.

On the GAIA benchmark, the 14B version of HybridAgent scored 52.4 points, compared with 43.7 for the same model trained as a text-only agent with ARPO. At the same time, runtime was 2.1 times faster than comparable text agents. The decoding savings of 62.5% to 72.9% come directly from dropping unnecessary tokens: instead of writing out a full chain of thought, the model "thinks silently" and speaks only when needed.

Under the hood, the adapter learns to map hidden states back into the model's input space without changing the original weights. This enables parameter-efficient training and preserves existing tool capabilities. When the agent needs to call an API, run Python, or present an answer to the user, it switches to full textual representation. The internal steps remain decodable for debugging and trust, but are not generated as tokens in real time.

For anyone running agents at scale, the implication is clear: fewer tokens means lower latency, lower GPU cost, higher throughput. The approach does not replace textual reasoning; it reserves it for the places where humans or external tools must read it. The agent's continuous, repetitive, "bureaucratic" reasoning stays latent. This is an architecture that fits how models actually work, rather than how we imagine they should work.