The Infinite
Team Problem

Meta recently launched Business Agent — an AI system designed to operate natively inside Instagram, Messenger, and WhatsApp, handling customer queries, guiding purchases, and resolving support tickets without human involvement. They're marketing it as an "infinite team": always on, infinitely scalable, never tired.

It's an impressive capability. It's also a near-perfect demonstration of the dynamic my dissertation was built around.

What My Research Actually Found

My work examined what happens when human operators collaborate with AI-assisted decision systems — specifically, how trust and understanding relate to each other under pressure. The finding I keep returning to is this: trust in AI systems and comprehension of their outputs are not the same thing, and they diverge in systematic, predictable ways.

Under low pressure, users engaged with AI outputs critically. They questioned anomalies. Their trust tracked their understanding. Under high pressure — time constraints, high volume, stress — that relationship collapsed. Trust increased while comprehension decreased. The less time people had to evaluate the AI, the more they relied on it.

I called this the understanding-trust gap. It isn't a character flaw. It's a structural feature of how humans interact with AI under cognitive load.

The Loop That Isn't There

Business Agent doesn't just widen this gap — it eliminates the feedback mechanism that might close it. In my research, even operators making miscalibrated trust decisions were still in the decision. They could observe outcomes. They could, over time, recalibrate. The loop was slow and imperfect, but it existed.

A fully autonomous customer-facing agent operates outside that loop. When it generates a subpar interaction — a wrong recommendation, a mishandled return, a confidence that doesn't match the facts — the customer experiences that failure. The brand absorbs it. But nobody in the organization necessarily sees it, learns from it, or updates their model of what the system is actually doing.

The article announcing Business Agent acknowledges this, almost in passing: "bad automated outputs actively damage consumer trust and corporate equity." That's accurate. What it doesn't reckon with is that the architecture being described makes those bad outputs systematically harder to detect and correct — because the human who would have noticed is no longer watching.

Scale Doesn't Solve This

The instinct is to treat this as a data problem: run enough interactions, monitor enough outcomes, and the system will self-correct. But the understanding-trust gap isn't primarily a performance problem. It's a relational one.

Trust between a customer and a brand is built through interactions that feel legible — where the person on the other end seems to understand what you actually need, and is accountable to getting it right. An agent that is statistically accurate but structurally unaccountable produces a different kind of relationship. Customers may not be able to articulate why an interaction felt wrong. They may simply not return.

This is the part that doesn't show up in accuracy benchmarks.

What Would Help

None of this means agentic commerce AI is inherently wrong. But the design question that matters isn't how autonomous can we make it — it's where does the human need to remain legibly in the loop, and for what kinds of decisions?

My MICA framework research suggests the answer isn't binary. The interfaces that performed best in my N=150 study weren't the ones that maximized automation or maximized human control — they were the ones that made it clearest to operators when to trust the AI and when to override it. The human doesn't need to be present for every interaction. They need to be present at the right moments, with enough understanding to make that presence meaningful.

An "infinite team" designed without that architecture isn't more capable. It's just harder to correct when it gets things wrong.