Medical writing: caution warranted if using ChatGPT

MONTREAL – Researchers from CHU Sainte-Justine and the Montreal Children’s Hospital recently asked ChatGPT 20 medical questions. The chatbot provided answers of limited quality, including factual errors and fabricated references. The researchers highlighted the results of their study in Mayo Clinic Proceedings: Digital Health.

“These results are alarming given that trust is a pillar of scientific communication. ChatGPT users should pay particular attention to the references provided before integrating them into medical manuscripts,” says Dr. Jocelyn Gravel, lead author of the study and emergency physician at CHU Sainte-Justine.

The researchers drew their questions from existing studies and asked ChatGPT to support its answers with references. They then asked the authors of the articles from which the questions were taken to rate the software’s answers on a scale from 0 to 100 percent.

Out of 20 authors, 17 agreed to review the answers of ChatGPT. They judged them to be of questionable quality (median score of 60 percent). They also found major (five) and minor (seven) factual errors.

For example, the software suggested administering an anti-inflammatory drug by injection, when it should be swallowed. ChatGPT also overestimated the global burden of mortality associated with Shigella infections by a factor of ten.

Of the references provided, 69 percent were fabricated, yet looked real. Most of the false citations (95 percent) used the names of authors who had already published articles on a related subject or came from recognized organizations such as the Centers for Disease Control and Prevention or the Food and Drug Administration. The references all bore a title related to the subject of the question and used the names of known journals or websites.

Even some of the real references contained errors (eight out of 18).

When asked about the accuracy of the references provided, ChatGPT gave varying answers. In one case, it claimed that “references are available in Pubmed” and provided a web link. This link referred to other publications unrelated to the question.

At another point, the software replied, “I strive to provide the most accurate and up-to-date information available to me, but errors or inaccuracies can occur.”

“The importance of proper referencing in science is undeniable. The quality and breadth of the references provided in authentic studies demonstrate that the researchers have performed a complete literature review and are knowledgeable about the topic. This process enables the integration of findings in the context of previous work, a fundamental aspect of medical research advancement. Failing to provide references is one thing but creating fake references would be considered fraudulent for researchers,” says Dr. Esli Osmanlliu (pictured), emergency physician at the Montreal Children’s Hospital and scientist with the Child Health and Human Development Program at the Research Institute of the McGill University Health Centre.

“Researchers using ChatGPT may be misled by false information because clear, seemingly coherent and stylistically appealing references can conceal poor content quality.”

This is the first study to assess the quality and accuracy of references provided by ChatGPT, the researchers point out.

The study Learning to fake it: Limited responses and fabricated references provided by ChatGPT for Medical Questions was conducted by Jocelyn Gravel, Madeleine D’Amours-Gravel and Esli Osmanlliu.

About CHU Sainte-Justine
The CHU Sainte-Justine Research Centre is a leading mother-child research institution affiliated with the Université de Montréal. It brings together 295 research investigators, including over 163 clinician-scientists, as well as more than 580 graduate and postgraduate students focused on finding innovative prevention means, faster and less invasive treatments, as well as personalized approaches to medicine. The centre is an integral part of CHU Sainte-Justine, which is the largest mother-child centre in Canada. Please visit recherche.chusj.org.

About the Montreal Children’s Hospital
Established in 1904, the Montreal Children’s Hospital (MCH) is Quebec’s oldest children’s hospital and the pediatric hospital of the McGill University Health Centre (MUHC). A tertiary and quaternary care teaching and research facility, treating newborns, children and adolescents up to age 18, it serves 63 percent of the geographic population of Quebec. The MCH is designated to offer services to patients in English. With its pediatric care and research facilities adjacent to the adult facility on the Glen site, the Children’s is in a unique position to offer services and research across the lifespan. The Centre for Innovative Medicine – the only clinical research centre in a hospital setting in North America – allows its researchers to conduct clinical trials on the Hospital site. The Children’s is a leader in providing a broad spectrum of highly specialized care to young patients and families from all across Quebec. The hospital is a provincially designated trauma centre and is recognized for its wealth of expertise in cardiology and cardiac surgery, emergency care, neurology and neurosurgery. Please visit thechildren.com.

About the Research Institute of the MUHC
The Research Institute of the McGill University Health Centre (RI-MUHC) is a world-renowned biomedical and healthcare research centre. The Institute, which is affiliated with the Faculty of Medicine of McGill University, is the research arm of the McGill University Health Centre (MUHC) – an academic health centre located in Montreal, Canada, that has a mandate to focus on complex care within its community. The RI-MUHC supports over 420 researchers and close to 1,200 research trainees devoted to a broad spectrum of fundamental, clinical and health outcomes research at the Glen and the Montreal General Hospital sites of the MUHC. Its research facilities offer a dynamic multidisciplinary environment that fosters collaboration and leverages discovery aimed at improving the health of individual patients across their lifespan. The RI-MUHC is supported in part by the Fonds de recherche du Québec – Santé (FRQS). Please visit www.rimuhc.ca.