Doctor at computer with AI screen showing 'I DON'T KNOW' message on medical diagnosis

When AI Can’t Say “I Don’t Know”

Current AI systems don’t truly “know what they don’t know.” They generate text that seems likely based on patterns in data, not on real understanding. Even advanced systems can give confident recommendations without showing how unsure they are, or without pointing out that the research they are using may not match a specific patient. Researchers…

2–3 minutes
Home » News » When AI Can’t Say “I Don’t Know”

When a new doctor is unsure about a lab result, they are expected to say “I don’t know,” ask for help, and look more closely at your chart. This honesty is part of keeping patients safe. It gives the care team a chance to slow down, review the medicines, and consult another specialist if needed.

Artificial intelligence (AI) tools are being tested to help doctors make decisions. These programs scan large amounts of medical information very quickly. But right now, many of them are designed to always give an answer, even when the information is incomplete or conflicting. That means an AI system might sound very sure of itself while actually being wrong, and it may not warn the doctor that there is uncertainty or missing data.

Doctors deal with many kinds of uncertainty in medicine: sometimes we don’t yet have the facts, sometimes the diagnosis is not clear, sometimes the outcome is hard to predict, and sometimes people’s goals and values differ. Learning when to pause and say “I don’t know” is an important skill. It helps prevent quick, incorrect decisions and encourages a more careful look at the situation.

Current AI systems don’t truly “know what they don’t know.” They generate text that seems likely based on patterns in data, not on real understanding. Even advanced systems can give confident recommendations without showing how unsure they are, or without pointing out that the research they are using may not match a specific patient. Researchers worry that this could be especially risky in high‑stakes situations, such as cancer treatment decisions.

Experts are calling for AI tools in health care to be built and tested in a new way. They suggest that, just as doctors are trained to recognize and talk about uncertainty, AI systems should also be designed to:

  • Flag when information is missing or conflicting.
  • Admit when a question is outside their area.
  • Offer more than one possible explanation or plan when there is no single clear answer.
  • Ask for human review instead of giving a firm recommendation when it is not confident.

These ideas could be built into rules for how AI tools are developed, tested, and approved for medical use. Health systems could track how often an AI system expresses uncertainty and how that compares with real‑world accuracy and patient outcomes. In very serious situations, like treatment choices for cancer, we may want AI tools to be especially cautious and more likely to say “I’m not sure” and ask a clinician to step in.

For patients, the key message is that AI should never replace your health care team. Instead, it should support your clinicians by helping them notice patterns and questions they might otherwise miss — and by clearly signaling when it is unsure. The safest systems will be those that, like experienced doctors, are willing to pause, admit uncertainty, and ask for help rather than guess.

Reference:

Sikora A, Celi LA, Abdulnour RE. Can AI Say “I Don’t Know”? N Engl J Med. 2026 May 14;394(19):1873-1875. doi: 10.1056/NEJMp2517624. Epub 2026 May 9.

You May Be Interested In