For triage and common conditions, modern medical AI performs well - published research shows strong results on history-taking and generating the right likely-cause list. But accuracy depends entirely on what you ask it to do: pattern-matching your symptoms, yes; examining you, running tests, or handling emergencies, no.
Where the evidence is genuinely good: studies of LLM-based clinical systems show performance at or above average clinician level on structured diagnostic reasoning with text-based histories, and excellent performance on triage (deciding how urgent something is) - which is why health systems are adopting AI triage. Where it breaks down: rare conditions (AI is trained on common patterns), anything needing a physical exam, lab values or imaging (it can only work from what you tell it), medication interaction edge cases in complex patients, and emergencies - an AI chat is never the right first move for chest pain or stroke symptoms. The honest architecture: AI handles the 90% that is pattern and conversation, and hands off to licensed clinicians for the rest. Any service that claims the AI alone replaces doctors is overstating it; any that claims AI adds nothing is ignoring the evidence. Emergencies skip this list: chest pain, trouble breathing, stroke signs, severe bleeding, or thoughts of self-harm mean call 911, or 988 for a mental health crisis in the US.
What to do
Common questions
What does the research actually say about AI diagnostic accuracy?
Published studies (including work in Nature and JAMA-family journals) show leading LLMs scoring at or above average physician level on standardized diagnostic reasoning cases and history-taking quality - when the case is presented as text. The same studies show real weaknesses: performance drops on rare conditions, on cases with missing information, and the systems can be confidently wrong, which is why the credible deployments pair AI with clinician oversight rather than replacing it.
Can an AI doctor misdiagnose me?
Yes - and so can a human doctor; misdiagnosis rates in primary care are well documented. The AI-specific risks: it works only from what you type (leave out a symptom, skew the answer), it's weaker on rare conditions, and it can't notice what an exam would (your color, your breathing, the tenderness when pressed). The mitigation is the same in both worlds: for anything serious, persistent, or worsening, get in-person care and testing.
Is an AI doctor accurate enough for triage?
Triage is where AI is arguably most reliable - 'how urgent is this and what kind of care does it need' is a pattern problem with clear right answers, and triage performance is the strongest part of the research literature. It's also the lowest-stakes use: a good AI that tells you 'this can wait until morning' or 'this needs the ER tonight' has done the most valuable thing a first contact can do.
When should I absolutely not rely on an AI answer?
Emergencies - chest pain, stroke signs, trouble breathing, severe allergic reactions, thoughts of self-harm (call 911, or 988 for a crisis). Beyond those: symptoms that are severe, rapidly worsening, or persistent despite the AI's suggested care; anything in an infant under 3 months with fever; and decisions about stopping prescribed medications. Emergencies skip this list: chest pain, trouble breathing, stroke signs, severe bleeding, or thoughts of self-harm mean call 911, or 988 for a mental health crisis in the US.