St. Michael’s devises high-powered forecasting tool

LAS VEGAS – St. Michael’s Hospital, in Toronto, has created a business intelligence (BI) and forecasting tool that enables the acute-care centre to answer questions in just two hours, down from the three to five days it took previously.

“Now it takes us longer to get a good question than to answer it,” quipped Jeremy Petch, PhD, Project Direct, Quality and Analytics, who gave a presentation on how St. Mike’s built the AI-powered system.

Dr. Petch was a speaker at the annual Healthcare Information Management and Systems Society (HIMSS) meeting, which attracted over 40,000 health I.T. professionals to sunny Las Vegas. It was held in March.

St. Mike’s “rapid response” tool, developed in partnership with the hospital’s Decision Support team, enables managers in the hospital to forecast, for example, expected volumes in the Emergency Department. And it’s doing it within a 6 percent error rate. With 200 or so visits per day to the ER, the tool is off by only 12 visits, at most, which still gives managers a pretty accurate idea of what to expect.

By knowing what to expect in the way of patient volumes, hospitals also get a better idea of their staffing needs.

It can be deployed, moreover, to forecast whether various programs will meet their quarterly or annual targets.

And it can also be used for retrospective studies – to determine whether a change implemented by a department actually worked.

Moreover, a group at the 463-bed hospital is now using the system to develop an Early Warning System for the intensive care unit. It will determine which patients on medical floors are about to crash, so they can be helped before they need to be moved to the ICU.

For years, as Dr. Petch described, it was difficult to answer various questions about past performance and to forecast the future, as the information needed was housed in different silos. As well, the hospital lacked a powerful analytics tool.

“Any analyses that were done were purely descriptive,” said Dr. Petch. “There wasn’t much rigor behind the analysis.”

However, a group of managers in the Quality, Performance, Information Technology and Information Management portfolio, and scientists at the hospital, wanted to change this.

“A few of us had a vision to transform St. Michael’s into a data-driven organization,” he said. The goal was to support high-quality patient care and to increase operational efficiency.

In 2016, they created the Centre for Healthcare Analytics and Training (CHART), based at St. Michael’s, which is part of a new network with St. Joseph’s Health Care Centre and Providence Healthcare. And they have started to recruit data scientists with expertise in machine learning, neural networks, biostatistics, simulation modelling and industrial engineering.

Dr. Petch commented that hospitals have to compete with companies like Google for AI-experts and other data scientists, but in Toronto they had something attractive to offer. As a teaching hospital and affiliate of the University of Toronto, St. Michael’s was able to facilitate professorships at the university for highly qualified individuals.

This attracted high-calibre talent, as many data scientists also want to make an impact as teachers and academic researchers.

“That’s something that Google and other companies can’t do,” commented Dr. Petch. “We can give them a good salary, plus an academic career.”

Finding talented junior staff is also a challenge. Many younger people, in search of fame and fortune, take quickie courses in Deep Learning or neural networks, and then dub themselves data scientists.

“There’s a proliferation of online courses available,” said Dr. Petch. “People are taking them, but then you discover they can’t write a line of code.”

To weed out the better candidates, CHART invited applicants to come in to do three or four hours of coding. “75 percent don’t even show up,” he said. “They know they can’t do it.”

By assessing the skills of those who do show up, CHART has been able to recruit a talented team.

On a lighter note, he commented that St. Mike’s has created a looser “culture” for its CHART think-tank, as computer professionals like a more free-wheeling atmosphere.

To solve the problem of disparate sources of information, with data separated into silos throughout the hospital, the team spearheaded the creation of a data warehouse.

Dr. Petch said there was a decision to be made about whether to invest in a data warehouse or a data lake. Data warehouses contain information of a more structured variety, while data lakes contain “raw data”, in all and any formats.

As the information in a data warehouse is more usable to researchers, and can be used more quickly to answer questions and solve problems, the team decided to go with this option.

To build the BI tool itself, the CHART team used R software – a language and environment for statistical computing and graphics. Not only is it high quality, but it also happens to be freeware – a big advantage for cash-strapped hospitals.

Today, the forecasting tool enables St. Michael’s Hospital to answer questions faster and more accurately than before. It is also reducing the staff time needed for forecasting and analytics.

Dr. Petch noted that in the ED, it could previously take 1.5 person days to answer a particular question; now, that can be done in a couple of hours.

Of course, not everything is perfect. He observed that data quality is still an issue, and that often, the problem occurs right at the point of entry – when clinicians are entering their data into various systems.

Dr. Petch said some organizations have been successful at improving their data quality, and singled out the Centre for Addiction and Mental Health (CAMH) in Toronto, as a leader in this area.

For its own part, in building the tool, Dr. Petch said that St. Michael’s learned a great deal from others. In particular, he acknowledged InterMountain Healthcare, in the United States. “They were mentors to us,” he said.