Of the many fields where immersive virtuality (iV) is making its mark, some of the most innovative work is being done in the world of science and medicine.
In medicine, for example, life and death can depend on the speed, accuracy and safety of a procedure. Virtual reality (VR) and augmented reality (AR) give physicians the means to plan and practice difficult surgeries on virtual patients risk-free. Digital diagnostics such as CT scans and MRIs can even be converted into custom VR models of a specific patient’s physiology.
HTC Vive was working on just such applications when it teamed with Surgical Theater.
“The original focus was on children’s brain tumors, where there’s so little space to operate,” said Hervé Fontaine, vice president of B2B Virtual Reality at HTC Vive. “Surgical Theater can generate a larger-than-life VR environment so the surgeon can actually walk into it and look at the tumor from every angle and plan the best way to approach it. This has significantly improved the success rate.”
Surgical training also benefits from VR. “Your training environment must be very accurate,” said David Weinstein, director of professional VR at NVIDIA, a leader in computer graphics. “Your digital model of the patient has to behave, from a physics point of view, as a real patient’s would. There’s a very complicated object-tracking algorithm that needs to know when two objects get close to each other and how they react when they touch, such as a scalpel pressing on flesh, so we’re developing libraries to support haptics in both audio and video.”
CREATING ACCURATE PHYSIOLOGY IN VR
Pete Johnson is vice president of Strategic Business Development at zSpace, which makes customized workstations for viewing and manipulating 3D medical models in VR.
“With the Living Heart project, for example, we’ve worked with Dassault Systèmes (publisher of Compass) and many scientific experts worldwide so that you can observe a precise model of the heart beating, see how it actually works inside the body,” Johnson said. “You can’t do that inside a cadaver. That model includes full mechanical and fluid dynamics, so it can be used for everything from patient instruction to pre-surgery planning to medical research.”
The unique design of zSpace workstations encourages and enables interaction among users, an important advantage in science and medicine, Johnson said.
“Head-mounted displays (HMDs) are isolating. You’re not interacting with others. Our model operates as a science workbench that enables a high degree of collaboration and engagement between users, whether they’re students in a classroom or researchers in a lab.”
VIRTUAL PHYSIOLOGICAL HUMAN
At the University of Sheffield’s Insigneo Institute for in silico medicine, researchers are helping to develop 3D models of the fundamental systems of the human body, part of a European Union-funded project called Virtual Physiological Human (VPH).
“The VPH vision involves developing simulations and models of the human body that can be used individually and together to address complex research problems – to improve treatment for diseases of the cardiovascular system, for example, or the muscular system, or vision or digestion,” said Andrew Narracott, a lecturer at the university in medical physics. “There’s a strong focus on translating these findings into clinical settings so that doctors can use them in routine practice.”
As in surgery, the “ability to try lots of different scenarios that would be too risky to try in the real world is a major benefit,” Narracott said. “For example, you wouldn’t experiment with implanting different versions of a medical device into an actual patient, but you can test different implants in a virtual model of a patient’s anatomy to evaluate the best option. Yes, you can do that on a computer screen, but VR provides new ways of interacting with the results of these simulations.”
VR also has the potential to improve communication between doctors collaborating to treat a patient, said John W. Fenner, also a lecturer in medical physics at the University of Sheffield.
“A lot of medicine is about communication,” Fenner said. “Surgeons communicating with radiologists to introduce a surgical procedure, or doctors communicating with patients so they understand the treatment they will undergo. VR communicates this information in a more understandable way than words or diagrams or even videos.”
DISCOVERING THE SECRETS OF CELLS IN VR
While medical researchers work on applying VR to treatment scenarios, theoretical scientists are using the technology to understand the most fundamental principles of how biology assembles an organism.
“If you talk about a cell, everyone can agree on what they do, but how they do it . . . this is still largely a mystery,” said Reza Sadeghi, a managing director of Dassault Systèmes’ BIOVIA brand. “You can’t put a camera inside a cell, but we now understand enough to create simulations. And we have enough computing power to create algorithms that translate equations into experience.”
Together, VR and advanced computing will change the way scientific research gets done, Sadeghi predicts.
“When we can actually see what is happening, we have a much better opportunity to fill in the blanks in our knowledge,” he said. “The simulations aren’t entirely there but we’re close, and we now have the platform to bring together all of the specialties in a truly collaborative environment that will move scientific discovery forward in an exciting new way.”
Watch Surgical Theater in action :
http://3ds.one/SurgicalVR
