A trio of bright and innovative medical physicists at The Ottawa Hospital’s new realizeLAB, for virtual reality solutions, have created a VR platform that enables surgeons and other clinicians to don headsets and seemingly enter organs like the colon for a better look at the anatomy.
By manipulating handheld controllers, they can also slice and pull apart 3D renderings of the thorax and skull – compiled from CT and MRI scans of their patients – that appear to be floating in space in just front of them.
Drs. Justin Sutherland, Dan La Russa, and Jason Belec put the system together, which is powered by their own software and off-the-shelf hardware. The Vive headsets are made by HTC, and other vendors, and the team cobbled together a number of other consumer technologies.
“It runs on a gaming computer,” said Dr. Sutherland. “All together, it cost less than $5,000 to put together.”
Amazing then, the enormity of capabilities the system has to offer for such a low cost.
“Our clinicians are very interested in this because it saves time, money and patient lives,” asserted Dr. Frank Rybicki, chair of the department of radiology at the Ottawa Hospital, and mentor of the VR project at the realizeLAB. He also heads the 3D printing project at the hospital – one of the most advanced in the country.
Dr. Rybicki explained the VR system enables surgeons, for example, to look at the position of a brain tumour and its relationship to adjacent structures, before going into the operating room to conduct the surgery.
Moreover, by manipulating the VR tools – two handsets that are like Wii controllers – you can turn the object, remove blood vessels or other tissue, and view the tumour from various angles.
Sutherland, La Russa and Rybicki demonstrated the new solution at the Radiological Society of North America (RSNA) conference in Chicago last November. They gave visitors a chance to test-drive a demo version of the system, while wearing the headset and using the handheld controllers.
It’s a mind-altering experience, to be sure, to wave your hand and see the bones of a patient disappear before your eyes, revealing only the soft tissue organs.
And it all seems so easy, like playing a video game. Put on the glasses, and a number of semi-transparent orbs are circling before your eyes. Simply grab one with your handheld controller, and drag it into a bigger circle, and you’ve activated that app – it may be the patient’s colon, skull or chest.
Wave again, and you remove bone or software tissue, or you can zero in on areas of interest.
“It gives you an excellent understanding of the tumour before going into the OR, of the spatial relationships,” said Dr. Rybicki. “You’ll never get this on a 2D monitor.”
With 3D virtual reality, “You can see 20 different views,” said Dr. Rybicki. “You can visualize things in this way with 3D printing, too, but you wouldn’t want to make 20 different models.”
He explained that 3D printing is a better option for procedures like stent grafts, where there are not so many options and variables for the surgery, but you still need a good view of the anatomy. Sometimes, as well, you want to be able to hold the anatomy in your hands, which 3D printing allows you to do.
Dr. Dan La Russa noted that many of the data sets for VR come from the models generated by the 3D printing lab, which makes use of CT scans and other exams. “There’s a complementarity, and we can use their data, and they can use ours.”
The realizeLAB, which has been up and running for about a year, has recently been assisting radiation oncologists as they plan procedures like brachytherapy and the positioning of radiactive seeds.
“They are the biggest users of VR, so far,” said Dr. Rybicki. “But we’re now explaining virtual reality to others.”
Other clinicians, including surgeons, cardiologists and orthopedic surgeons have all expressed an interest in the platform, as it gives them a much better idea of how to plan the most effective surgery or procedure.
The hospital’s 3D printing lab now has eight employees, and Dr. Rybicki is confident that the VR lab, which now has three PhDs splitting part of their time there, will expand to five employees in the next year or two.
Dr. Sutherland commented that a lot of discussions have been going on with surgeons, radiologists and radiation oncologists. “We’re working with them to develop the tools they’d like to see.
“It’s a new domain,” he added. “We expect to be creating applications that we never imagined.”
Indeed, the field of VR is brand new, and many concepts are just being developed and rolled out as practical technologies. “Anything you can coherently explain, we can build,” said Dr. Sutherland.
Having the realizeLAB right in the hospital is a definite advantage, as it puts researchers like himself, La Russa and Belec in the midst of clinicians working with patients. Together, they can fine-tune solutions that solve real-world problems.
One radiation oncologist, for example, has asked for an app that allows him to stand inside the blood vessels running through the brain, giving him a better view of the anatomy of the issues he is dealing with.
Dr. Rybicki commented that not only is VR a big help to clinicians inside the Ottawa Hospital, but it could be used as a telehealth platform. Images could be taken of patients at a remote location, and experts at the hospital could wear the VR goggles to obtain a better look at aneurysms, tumors and other problems.
They could then advise local doctors on the best ways of treating the lesion or neoplasm. “You could do a consult with someone in Nunavut,” he said, effectively raising the level of care given to patients in remote regions.
Moreover, it’s a group activity. “You can collaborate with clinicians in a remote location, and you’ll see others [in your field of vision] as avatars,” said Dr. La Russa. Your collaborators appear as heads on-screen, and you can interact simultaneously with patient objects while conducting an exam and consultation.
“It’s easy to have a discussion about the images,” said La Russa.