AI agents can buckle under memory overload — Box CEO Aaron Levie calls it “context rot.”
“The model will just get very confused and potentially focus on the wrong part of the information,” Levie told Business Insider.
Split big jobs into fleets of specialized sub-agents instead of relying on one super-agent, he added.
Aaron Levie told Business Insider there’s a catch to AI agents: If you feed them too much information, they start to lose the plot.
The CEO of the cloud-storage giant Box calls this problem “context rot.”
The more data you give the AI model, “it doesn’t necessarily lead to a better outcome,” Levie said on Tuesday. “The model will just get very confused and potentially focus on the wrong part of the information.”
As the task drags on, the model can lose track of what it’s supposed to focus on, leading to worse results, he added.
That flaw happens when agents are overloaded with massive amounts of data in their “context window” — the part of the process where models synthesize information before generating a response.
Instead of trusting one super-agent to handle everything, Levie said the smarter approach is to carve up the work and assign it to fleets of specialized sub-agents.
“You’re going to want to break apart the agents and the context that they have,” he said.
“You’ll have multiple agents, all with a set of goals and a set of contexts that are germane to their particular part of the workflow,” he added.
It’s counter-trend to the Silicon Valley dream of a single AGI overlord. Levie, who cofounded Box in 2005, said the sub-agent model is “definitely going to be the future of large-scale agent systems.”
The CEO also said the key to better AI performance is to give these models “the most accurate information and just the most precise data.”
“You have to be both very precise in your instructions and then you have to give the model an incredible amount of the right context to operate from, but too much context actually, it will do worse with,” he added.
AI agents are far from perfect
Silicon Valley has been buzzing about AI agents, with companies racing to use them for increasingly elaborate, multi-step tasks.
Regie AI’s “auto-pilot sales agents” prospect and follow up with buyers, Cognition AI’s Devin tackles complex engineering work, and Big Four professional services firm PwC has introduced “agent OS” to help different agents coordinate with one another.
But the reality is messy. In theory, agents can solve problems, execute tasks, and get smarter as they learn. In practice, the more steps they take, the more fragile the process becomes.
Researchers have warned that agent errors are prevalent and compound with each step they take.
“An error at any step can derail the entire task. The more steps involved, the higher the chance something goes wrong by the end,” Patronus AI, a startup that helps companies evaluate and optimize AI technology, wrote on its blog.
The startup built a statistical model that found that an agent with a 1% error rate per step can compound to a 63% chance of error by the 100th step.
Still, the company said that guardrails — such as filters, rules, and tools to identify and remove inaccurate content — can help mitigate error rates. Small improvements “can yield outsized reductions in error probability,” Patronus AI said.
Read the original article on Business Insider
