Physicians and game developers to create VR for healthcare

David Wyand’s introduction to bringing lifelike, immersive reality to the medical field came through a chance meeting.

In the fall of 2016, at a virtual reality meetup, the lead-developer with Toronto-based Quantum Capture encountered Dr. Clyde Matava and Dr. Fahad Alam, two Toronto-based anesthesiologists. The two had co-founded the Collaborative Human Immersive Interaction Laboratory (CHISIL), a community of medical professionals from across Canada devising VR systems to simulate patient experiences and training scenarios.

The veteran of game engine design with now-defunct Garage Games became one of the first developers to work on a CHISIL project. It was new territory for Wyand, and for Quantum.

Quantum is known as the virtual people people, providing hyper-realistic, 3D, full-body scans for VR and augmented reality content developers, largely gaming companies. “None of us had ever done a medical simulation before,” Wyand says. “It ended up being more complicated than some.”

Quantum worked with CHISIL to develop an immersive simulation of a fibre-optic bronchoscopy procedure. The trainee selects appropriate monitors and equipment while operating the bronchoscope handle, scoping the throat of an imaginary patient with tracheal damage.

Meanwhile, there’s a virtual collar to be removed, chatter and instructions from other doctors to be processed, and, at the end of the day, a score reflecting performance.

Dr. Matava, an assistant professor of anesthesiology at the University of Toronto and staff member at Sick Kids Hospital, and Dr. Alam, a staff anesthesiologist at Sunnybrook Health Sciences Centre in Toronto and director of research at the Sunnybrook Simulation Centre, founded CHISIL in 2014.

CHISIL has been lauded for its ChildLife Virtual Reality application, a 3D headset that simulates the preoperative experience in a 360-degree interactive environment. More than 200 patients – mostly children at Sick Kids and adults with anxiety issues – have used the headset, finding it prepares them better and calms their nerves more than a PowerPoint presentation or drawings.

Anxiety can affect all stages of surgery, requiring more anesthetic and extending hospital stays. (The application is available for download by Apple users on iTunes.)

“We’re at the forefront of bringing virtual reality to healthcare,” says Dr. Alam. The mission of CHISIL is to develop VR for patient and medical education, as well as clinical applications: assessing, diagnosing and treating patients. It’s a community of “the right people in the right places,” with common interests for common outcomes, that can move the technology from conception through validation, Dr. Matava says.

There is some resistance in the field. Clinicians are very wary of consumer technology, according to Matava. “Virtual reality is something that stays at home,” he says. CHISIL is trying to move VR from a gaming experience to something of clinical value.

There’s a learning curve attached. They’ve had to learn about the technologies and the many skill sets involved in creating an immersive experience: not just graphic design, game design and coding, but directing, set design, acting and scripting.

“It really does take a village to build virtual reality,” says Dr. Matava.

Quantum photographed Sunnybrook’s trauma bay, modeled a patient’s mouth and trachea, and replicated every device – what it looked like, what it sounded like. The game development process calls for a game document, which details player options and consequences; the team worked with Dr. Alam and Dr. Matava on a flow chart for the procedure.

It wasn’t easy creating all of this, as the doctors were conducting their day-to-day clinical duties and were pressed for time. The need for close collaboration, combined with the doctors’ busy schedules, demanded flexibility from the Quantum team.

“We had to make sure when they were available, we were available,” Wyand says. That meant lots of nights and weekends.

And it was necessary to add different levels of sophistication and difficulty. For example, Drs. Alam and Matava breezed through the simulation in early iterations, so it was necessary to toughen it up for more experienced users.

In the end, the project was a success – it resulted in a useful immersive reality training tool for clinicians.

Meanwhile, another member of CHISIL is working on how to use virtual reality to explore the connections between anesthetic use and cognitive deficits.

Dr. Sinziana Avramescu is the senior investigator for virtual reality and cognition with CHISIL. She is working on how immersive reality can help predict, diagnose, and eventually treat post-operative delirium (POD).

POD is often associated with general anesthetic use, and is a persistent threat. Its incidence varies from nine to 87 percent depending on the age of the patient and the severity of the surgery, according to a 2011 paper from the Washington University School of Medicine in St. Louis. POD is associated with increased length of hospital stay and a cost of $164 billion a year to the healthcare system in the U.S. alone. More frightening is the 10-20 percent increase in mortality for every 48 hours of delirium, and a 10-fold increase in the risk of dementia.

“That’s quite scary,” says Dr. Avramescu. “Having surgery, most of the time, isn’t optional.” Dr. Avramescu, a clinician investigator and assistant professor at the University of Toronto, says rapid diagnosis is key to ameliorating the negative outcomes of POD.

Identifying patients likely to experience POD could help. While much is unknown about the risk factors, a previous history of cognitive problems appears to predispose patients, according to a New England Journal of Medicine article by Dr. Sharon Inouye.

Patients with mild cognitive impairment can often disguise symptoms in order to function in the community, Dr. Avramescu says; they can’t hide spatial impairment from a 3D simulation.

By engaging the patient in a fully immersive experience to test spatial cognition – a maze, in this case – early signs of dementia can be detected without the need for an expert interviewer and patients screened for more in-depth investigation.

Dr. Avramescu’s mission is ongoing. The team is at the storyboarding stage of the spatial awareness project. After initial development comes a screening test on healthy volunteers. Further down the road, she hopes the technology can be used for treating POD, not just diagnosing it. “That’s a huge thing to do, to help them get better, not just telling them they have a problem,” she says.m,” she says.