Canadian Healthcare Technology Logo
  • Issues
    • Current Print Issue
    • Print Archive
  • Advertise
    • Publishing Schedule
    • Circulation
    • Unit Sizes and Rates
    • Mechanical Requirements
    • Electronic Advertising
    • White Papers
  • Subscribe
    • Print Edition
    • e-Messenger
    • White Papers
  • Events
  • Vendors
  • About Us

Enovacom (replay)

Enovacom (replay)

Sectra 2021 Klas award

Sectra 2021 Klas award

Artificial intelligence

Researchers call for more transparency in AI

October 21, 2020


Benjamin Haibe-KairnsTORONTO – International scientists are challenging their colleagues to make Artificial Intelligence (AI) research more open and reproducible to accelerate the impact of their findings for cancer patients. In particular, they’re targeting studies that claim AI systems are better at diagnosing disease than physicians, but which fail to offer enough transparency to test and reproduce the results.

In an article published in Nature on Oct. 14, scientists at Princess Margaret Cancer Centre, University of Toronto, Stanford University, Johns Hopkins University, Harvard University School of Public Health, Massachusetts Institute of Technology, and others, challenge scientific journals to hold computational researchers to higher standards of transparency, and call for their colleagues to share their code, models and computational environments in publications.

“Scientific progress depends on the ability of researchers to scrutinize the results of a study and reproduce the main finding to learn from,” says Dr. Benjamin Haibe-Kains (pictured), senior scientist at Princess Margaret Cancer Centre and first author of the article.

“But in computational research, it’s not yet a widespread criterion for the details of an AI study to be fully accessible. This is detrimental to our progress.”

The authors voiced their concern about the lack of transparency and reproducibility in AI research after a Google Health study by McKinney et al., published in a prominent scientific journal in January 2020, claimed an AI system could outperform human radiologists in both robustness and speed for breast cancer screening.

The study made waves in the scientific community and created a buzz with the public, with headlines appearing in BBC News, CBC and CNBC.

A closer examination raised some concerns: the study lacked a sufficient description of the methods used, including their code and models. The lack of transparency prohibited researchers from learning exactly how the model works and how they could apply it to their own institutions.

“On paper and in theory, the McKinney et al. study is beautiful,” says Dr. Haibe-Kains, “But if we can’t learn from it then it has little to no scientific value.”

According to Dr. Haibe-Kains, who is jointly appointed as associate professor in Medical Biophysics at the University of Toronto and affiliate at the Vector Institute for Artificial Intelligence, this is just one example of a problematic pattern in computational research.

“Researchers are more incentivized to publish their finding rather than spend time and resources ensuring their study can be replicated,” explains Dr. Haibe-Kains. “Journals are vulnerable to the ‘hype’ of AI and may lower the standards for accepting papers that don’t include all the materials required to make the study reproducible – often in contradiction to their own guidelines.”

This can actually slow down the translation of AI models into clinical settings. Researchers are not able to learn how the model works and replicate it in a thoughtful way. In some cases, it could lead to unwarranted clinical trials, because a model that works on one group of patients or in one institution, may not be appropriate for another.

In the article titled Transparency and reproducibility in artificial intelligence, the authors offer numerous frameworks and platforms that allow safe and effective sharing to uphold the three pillars of open science to make AI research more transparent and reproducible: sharing data, sharing computer code and sharing predictive models.

“We have high hopes for the utility of AI for our cancer patients,” says Dr. Haibe-Kains. “Sharing and building upon our discoveries – that’s real scientific impact.”

PreviousNext

WP 900×150

WP 900x150

News and Trends

  • Using Lean to reduce wait times for COVID-19 tests
  • Ultrasound techs devise face-shield with better protection
  • Lakeridge Health quickly pivots to virtual care during COVID
  • Partnership brings virtual care to Indigenous communities
  • e-VOLVE modernizes electronic records at SW Ontario hospitals
More from the Print Edition

Subscribe

Subscribe

Free of charge to Canadian hospital managers and executives in nursing homes and home-care organizations. Learn More

Follow us on Social Media!

Follow us on Social Media!

C60 BeyondMyths

C60 BeyondMyths

Spok

Spok

Infoway (Feb)

Infoway (Feb)

MIIT 2021

MIIT 2021

Advertise with us

Advertise with us

Cisco SVT

Cisco SVT

CDW Lenovo T14

CDW Lenovo T14

Ampronix

Ampronix

Dapasoft CVC

Dapasoft CVC

Mohawk College (2)

Mohawk College (2)

CABHI Summit 2021

CABHI Summit 2021

WP 900×150

WP 900x150

Advertise with us

Advertise with us

Cisco SVT

Cisco SVT

CDW Lenovo T14

CDW Lenovo T14

Ampronix

Ampronix

Dapasoft CVC

Dapasoft CVC

Mohawk College (2)

Mohawk College (2)

CABHI Summit 2021

CABHI Summit 2021

Contact Us

Canadian Healthcare Technology
1118 Centre Street, Suite 207
Thornhill, Ontario, Canada L4J 7R9
Tel: 905-709-2330
Fax: 905-709-2258
info2@canhealth.com

  • Quick Links
    • Current Print Issue
    • Print Archive
    • Events
    • Vendors
    • About Us
  • Advertise
    • Publishing Schedule
    • Circulation
    • Unit Sizes and Rates
    • Mechanical Requirements
    • Electronic Advertising
    • White Papers
  • Subscribe
    • Print Edition
    • e-Messenger
    • White Papers
  • Resources
    • White Papers
    • Writers’ Guidelines
    • Privacy Policy
  • Topics
    • Administrative Solutions
    • Clinical Solutions
    • Companies
    • Continuing Care
    • Diagnostics
    • Education & Training
  •  
    • Electronic Records
    • Government & Policy
    • Infrastructure
    • Innovation
    • People
    • Privacy and Security

© 2021 Canadian Healthcare Technology

The content of Canadian Healthcare Technology is subject to copyright. Reproduction in whole or in part without prior written permission is strictly prohibited. Send all requests for permission to Jerry Zeidenberg, Publisher.

Search Site

Error: Enter a search term

  • Issues
    • Current Print Issue
    • Print Archive
  • Advertise
    • Publishing Schedule
    • Circulation
    • Unit Sizes and Rates
    • Mechanical Requirements
    • Electronic Advertising
    • White Papers
  • Subscribe
    • Print Edition
    • e-Messenger
    • White Papers
  • Events
  • Vendors
  • About Us