Virtual reality promises to be great for gamers, but it also has potential to change the world in other ways. Shauna Heller recognized that potential, so she quit her job at Oculus VR and became a consultant — working to bring non-VR applications to life. One of the projects she took on was to create a VR simulation that teaches doctors how to treat infants in emergencies.
Heller pulled together a team of developers and technologists and matched them with surgeons at the Children’s Hospital in Los Angeles (CHLA). Together, they made a VR simulation that helps doctors in training save infants who have suffered some kind of seizure or are in anaphylactic shock.
The team worked with the consumer-grade Oculus Rift + Touch on the project, which began in early 2016 and went through two prototyping developments. They delivered a working model in early 2017, and the development and medical teams will continue to monitor and enhance the virtual world as more users learn with it. About 18 developers worked on the project.
“This is a really important part of VR,” said Heller, executive director of the project and former developer relations liaison for non-game projects at Oculus. “It takes a game technology and enables it to be used for training in the medical space, where children’s lives are on the line.”
Facebook’s Oculus VR division, maker of the Oculus Rift VR headset, funded the project. It is one of a number of projects the company calls “VR for good,” and, in the long run, such projects could bring the real value of VR home to society.
Heller was one of the original members of the Oculus team. But she wasn’t as passionate about games as she was about other uses for VR. As a consultant, she got a tip from Oculus about an idea to improve health care education through the immersive power of VR. Heller enlisted the help of AiSolve and VR developer BioflightVR. Both companies had done medical simulations before, so they knew how to talk to the doctors in a a hospital setting.
Above: Shauna Heller is a VR consultant for non-game applications.
BioflightVR was founded by Hollywood visual effects wizards from the CBS show CSI who wanted to put their skills to work on a project with greater social impact. Fortunately, they were able to understand the medical terms the doctors used, Heller said.
“BioflightVR was tasked with helping realize a grand vision — what the future of medical training could look like,” said Rik Shorten, chief creative officer and cofounder. “I can’t think of a better use for our team’s talents then helping doctors and residents learn and train using these dynamic new technologies.”
United Kingdom-based AiSolve is an immersive training and simulation company that developed a proprietary A.I. framework to create VR content. Together, the two companies worked closely with the hospital, even to the point of scanning in the images of real nurses.
Above: Nurses were captured for 3D animations.
The doctors conceived the scenarios. A team of programmers at AiSolve took the medical environment and created an AI-powered virtual world in which students can make decisions and progress or re-evaluate their decisions based upon responses from the virtual patient, virtual medical staff, and other program parameters.
Pediatric emergencies are rare, but by their nature, they are intense. The time pressure is extreme, measured in seconds or minutes. Young surgeons can’t practice these scenarios in real life as often as would be necessary to gain expertise. They usually have to work with mannequins, and the cost can run $430,000 a year for a single hospital to train surgeons on standard scenarios, according to Todd Chang, doctor at CHLA. The cost of the VR system is less than that amount, Heller said.
Above: Standing back row/ left to right: Todd Chang (CHLA), Tom Dolby (AiSolve), Doctor Josh Sherman (CHLA), Randy Osborn (BioflightVR), Rik Shorten (BioflightVR). Standing front row/ left to right: Devi Kolli (AiSolve), Shauna Heller (Clay Park VR). Kneeling: David Bryant (BioflightVR).
“This sort of cutting-edge medical training is where VR shines,” said AiSolve CEO Devi Kolli, in a statement “Working with the project team and CHLA doctors, we harnessed our AI-powered VR simulation tech to closely replicate real-life scenarios in a way simply not possible before. Plus, the AI features let students customize their learning, which strengthens their skills in the long run.”
The VR simulation not only reproduces the scenarios, it captures them in a controlled training experience, where performance can be measured and adjustments made.
The simulation reproduces the emergency situation. Paramedics rattle off symptoms. Nurses and technicians turn to the participating doctor to make a decision. Distraught parents pray for their child’s survival. It’s like a real Hollywood drama, but visceral and interactive. It prepares the surgeon for a real-life emergency without putting real humans’ lives on the line.
“We’ve been testing the validity of VR as a training and assessment tool for medical resuscitations, and the preliminary results are strong,” said Josh Sherman, CHLA doctor, in a statement. “Even physicians with little video game experience caught on quickly and gave us great feedback. We’re also seeing a similar stress response in the VR experiences compared to real-life ER situations. Our trainees know it’s a simulation, but it feels real — and that makes all the difference.”
Above: VR hospital
By recreating real cases in VR, students and doctors can repeatedly go over tough scenarios until they get the diagnosis and treatment right. CHLA is actively using two training modules with medical students, residents, and doctors. The surgeons become faster over time. Based on the results, Oculus said that it is hoping to ramp up support for medical professionals.
AiSolve CEO Devi Kolli said that the objective at Children’s Hospital Los Angeles was to work with medical experts to ensure technical and environmental accuracy and that the virtual training has multiple applications across industry sectors.
“Experiential learning is among the best way to practice pediatric emergencies,” Chang said. “We had a rather aggressive timetable, and the VR simulation literally improved week by week. VR allows for the first-time experiential learning where not all the people are in the same room at the same time. It is far more flexible, and students can perform the training far more often.”
Both AiSolve and BioflightVR have moved on to related projects.
