AI Might Be Coming To An Urgent Care Clinic Near You, New Study Suggests

Researchers announced the completion of a groundbreaking study where an artificial intelligence chatbot assisted primary care physicians by interacting with patients at a clinic.

An eight-month clinical study following a chatbot engaging with patients published its findings Thursday in the medical journal The Lancet, the New York Times reported. The study was conducted at Beth Israel Deaconess Medical Center (BIDMC) in Boston.

The chatbot, Articulate Medical Intelligence Explorer — known as AMIE — first launched in 2024 when at team of Google Researchers developed the AI system using large language models (LLMs) optimized for diagnostic reasoning and conversations, according to a post by Google Research.

“They compared it to a high-performing medical student or resident,” Peter Brodeur, a resident physician at BIDMC told the Times of the chatbot.

After two years worth of working out the kinks in programming, and conducting in-depth clinical trials at BIDMC’s primary care clinic with supervising physicians who monitored the bots interactions with patients, the researchers discovered roughly 100 patients consulted the AMIE chatbot ahead of going to urgent care visits, the outlet reported.

The clinicians conducting the study also found that AMIE’s summaries helped them prepare for visits in 75% of cases, influenced their approach to care in more than half, and provided differential diagnoses that matched their final diagnoses 90 percent of the time, the Times reported.

However, the study also found that the bot made stuff up — but reportedly only once during hundreds of conversations, the Times reported.

“None of our patients trusted it until they realized that it was handing-off their doctor,” Adam Rodman, a BIDMC internist and medical historian who helped conduct the trial, told the Times.

“Recent progress in large language models (LLMs) outside the medical domain has shown that they can plan, reason, and use relevant context to hold rich conversations,” researchers said in their initial 2024 report on Google Research — which was later published in the scientific journal Nature. “However, there are many aspects of good diagnostic dialogue that are unique to the medical domain. An effective clinician takes a complete ‘clinical history’ and asks intelligent questions that help to derive a differential diagnosis. without providing misleading information.”