In February 2025, a 45-year-old man walked into a Queens emergency room coughing up blood. His name was Louie Quiros. He had been struggling to breathe for four days. His heart was beating fast. His chest X-ray showed nothing. His electrocardiogram was abnormal, but ambiguously so — the kind of reading that might indicate coronary disease, or might not. The ER doctors learned he had recently been in California during wildfire season. They sent him home with asthma medicine and an inhaler.

They were not negligent. The ECG was genuinely difficult to read. And as one cardiologist noted, ordering an echocardiogram for every abnormal ECG would be unsustainable — there are too many abnormal ECGs and not enough capacity to chase each one down. Clinical judgment, under pressure, means making reasonable calls about which signals to pursue. The doctors made one.

It happened to be wrong. But that is not the part of this story I want to stay with.

What the Algorithm Saw

Louie Quiros was lucky to be at that particular ER. NewYork-Presbyterian was running a clinical trial of an AI program called EchoNext, developed by Dr. Pierre Elias and his colleagues at Columbia University Irving Medical Center. EchoNext reads every ECG in the system within ten minutes of it being performed, scanning for patterns associated with structural heart damage — patterns that do not announce themselves to the human eye.

EchoNext flagged Quiros's ECG. The research team called him back a week later. An echocardiogram showed his heart was pumping at roughly ten percent of normal capacity. His mitral valve was leaking. Genetic testing revealed a rare hereditary disorder associated with sudden death. He received a heart transplant.

The case was published in Nature Medicine in June 2026. EchoNext has since received FDA approval and will be made available, free of charge, to any physician using OpenEvidence, a clinical decision support platform used by approximately half of U.S. doctors.

It is, by any measure, a remarkable result. And it is also, if you look at it from a certain angle, a near-perfect illustration of what makes this moment in AI adoption so consequential — and so precarious.

A Flag Without a Map

Here is what the EchoNext alert did not tell the doctors who received it: what it saw. Not in any form a clinician could independently evaluate. The system identified a pattern in the waveform data that its training associated with serious cardiac damage. It surfaced that finding as a flag. The physicians who acted on it — who called Quiros back, who ordered the echo, who ultimately saved his life — did so on the basis of that flag alone.

This is not a criticism of EchoNext, or of the clinicians. It is a description of the moment. The algorithm detected something real. The humans responded appropriately. The outcome was good. But the chain of events rested on a form of trust that is worth naming precisely: trust in a signal that could not be verified by the people acting on it.

"The outcome was good. But the chain of events rested on a form of trust that is worth naming precisely."

In my research on AI-assisted decision-making, I have found that this is not an unusual structure — it is increasingly the default one. AI systems trained on vast datasets can surface patterns that exceed what any individual human could detect unaided. That is the value proposition, and in cases like Quiros's, it is real. But the detection and the understanding are separated. The system sees something. The human acts. The gap between those two events is where trust lives, and it is rarely examined with the care it deserves.

What Pressure Does to Trust

There is a well-documented pattern in how people respond to AI outputs under cognitive load. When decisions need to be made quickly, the mind looks for the fastest available signal that reduces uncertainty. A confident AI flag is exactly that kind of signal. It feels authoritative. It collapses ambiguity. It invites action without requiring the slower, more effortful work of actually understanding what the system found or why. Comprehension and trust are not the same cognitive process, and pressure affects them differently.

Emergency medicine is perhaps the highest-pressure environment in which AI tools are now being deployed. That is also what makes the Quiros case instructive, not just as a success story but as a data point. EchoNext worked. The flag was correct. The doctors followed it. But the conditions that made this outcome possible — a research context, a clinical trial with follow-up protocols, a team already primed to take the alert seriously — are not the conditions under which most clinical AI deployments will operate.

Scale Changes the Stakes

EchoNext's availability through OpenEvidence means it will now reach physicians practicing far outside major academic medical centers, in settings without research infrastructure, without specialists on hand to contextualize a flag, and without institutional familiarity with how the model behaves when it is wrong. That reach is part of the point. The physician who treats patients in a rural clinic, or a community ER, or a primary care practice where cardiology consultation is a referral away, is exactly who this tool is meant to serve.

But scale also means the distribution of outcomes will widen. EchoNext will catch things that would otherwise be missed. It will also, at some rate, flag things that do not require urgent follow-up. Clinicians who encounter those false positives will form impressions about the system's reliability. Some will recalibrate appropriately. Others will over-trust subsequent alerts, because the last one turned out fine. Others will begin to discount the tool, because it sent them chasing something that wasn't there.

None of that is speculation. It is what happens when any decision support system is distributed at scale, to users with varying levels of context about how it works, without corresponding investment in the conditions that make calibrated trust possible.

"The flag arrived. The doctors couldn't see what it saw. They acted anyway. That is the structure of most AI-assisted decisions now."

The Question Behind the Case

Louie Quiros is alive. That is not a small thing, and nothing in this essay is meant to diminish it. EchoNext did something that cardiologists sitting in quiet offices, given time and every resource, might also have done. What it did was do it faster, and in an environment where those conditions did not exist.

The question this case raises is not whether AI should be used in clinical settings. It clearly should. The question is what kind of trust architecture we are building around these tools as we scale them. Whether we are investing in the conditions — the protocols, the feedback loops, the interface design, the practitioner training — that allow clinicians to know when to follow a flag and when to interrogate it. Or whether we are distributing powerful tools into high-stakes environments and assuming that a good outcome story is the same as a validated trust framework.

The signal was real this time. The noise is still there. And the gap between them is where the work actually lives.

Source

Gina Kolata, "Doctors Thought It Was Asthma. A.I. Flagged a Serious Heart Problem." The New York Times, June 22, 2026. nytimes.com

DH

Debra Hogue, PhD

Computer Scientist · Human-AI Collaboration Researcher · Oklahoma