Surrounded by darkness, the looming 20 foot skull is so close I can touch it. With the click of a mouse, the 3D model of the human head and neck pivots and I’m inside the eye socket examining this complex system from the inside out. It’s A viewpoint typically reserved for surgeons on the operating table; I’m amazed by the scale and detail of the mechanism that gives us the miracle of sight.
Above: Creative and medical teams examine the 3D Neck and Head project in SimVis’s ‘Lab 01’ immersive media space.
The finished product of three years of work and almost 1 million pounds, the 3D Head and Neck anatomy project has revolutionized the way we study anatomy. Glasgow School of Art’s School of Simulation and Visualization's virtual reality model has been so well received by the medical community that the school was awarded funding to complete male and female versions of the entire human body. These complete anatomical models, dubbed the ‘3D Definitive Human’ project, will be launched in early 2018.
Best thing since Gray’s Anatomy?
The 3D VR model of the head and neck has been rapidly adopted by universities and training programs across the U.K. with thousands of interactions annually. According to Dr. Malcolm Skingle, CBE DSc PhD, Director and Academic Liaison for GlaxoSmithKline, “I think that the 3D Definitive Human has the potential to be the best thing available since Gray’s Anatomy to train the next cadre of medics and biological scientists.”
The School of Simulation and Visualization, or SimVis, partnered with leading hospitals and universities in order to complete the most accurate and detailed model of the head and neck ever constructed. The ultra-high resolution 3D virtual model is fully interactive and can be consumed across a variety of environments and platforms. Students can interact with the model in a group setting, similar to an IMAX theater, or study the model solo on their tablet, laptop or mobile device. The 3D model includes accurate interactive visualizations of all the anatomical systems including the musculo-skeletal system, circulatory system, nervous system and digestive system.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
The model also includes haptic capabilities providing feedback through hand controllers that can simulate routine procedures, such as giving an injection. In the past, medical students would practice giving an injection on crude, plastic dummies or on each other. Today, students can perfect this skill safely and expertly without ever touching a patient.
Above: The 3D Neck and Head project include haptics or touch technology that simulates administering an anesthetic injection to a patient
Blending of art and science
In order to create the medically verified model, the SimVis team worked alongside a clinical verification team of surgeons and anatomy professors including Professor Anna Lysakowski, a board director of the American Association of Anatomists.
Above: SimVis 3D modeller works to create texture maps for an anatomical model
The creation process brought together computer scientists, 3D modellers, artists, doctors, anatomists, researchers and professors.. This blending of the creative arts and sciences is what sets SimVis apart and allowed for such stunning results.
The workflow begins with actual dissections of human cadavers that are then photographed using advanced photogrammetry. This technique includes taking thousands of high-resolution photos from multiple angles and stitching them together to help form the precise 3D model. Modelling software, such as 3ds Max or Maya, is then used to construct the shape of the model virtually and overlay the exact look and feel through a process known as “texture mapping.”
Once this has been completed, the user interface is constructed and the content is formatted for various platforms with the help of the gaming engine, Unity. Rigorous tests are then conducted in order to validate the model’s accuracy.
Above: SimVis designer shows how real cadavers are photographed at various points and then stitched into the shape underlying the 3D virtual model
The road ahead
Capitalizing on the momentum of head and neck projects success, SimVis is preparing to launch the completed “3D Definitive Human” project early next year. This second-stage project is the culmination of a three year partnership between SimVis, the Scottish Funding Council, NHS Education for Scotland, The Royal College of Surgeons of Edinburgh, the Fulbright Commission, and selected colleges and universities in the region. The end result is a highly accurate, 3D model of the entire male and female human body.
Above: SimVis Director, Paul Chapman, on stage sharing a preview of The Definitive Human project launching in early 2018
Based upon the positive results and improved learning outcomes from the head and neck project, the implications for the “3D Definitive Human” project are limitless. According to SimVis director Paul Chapman, “Our models will now be augmented with pathophysiology, for example skin cancer, which could result in better diagnosis, improved training and education for medical professionals and will ultimately lead to much better healthcare for patients.”
SimVis is not alone in seeing the potential that immersive technology, such as virtual reality, has in shaping the future of health care. A US based start-up, Surgical Theater, uses virtual reality to create a model of the patient’s anatomy to help both doctors and patients better understand complex procedures. Microsoft has also been exploring medical case uses for their augmented reality headset the Hololens. One pilot project, dubbed “Project Serra,” experimented with using the Hololens to project MRI data, real-time on a patient being prepped for surgery. This would then allow for more precise targeting of tumors within the body.
VR projects like the “3D Definitive Human” are a great stride forward but we’re still in the nascent stages of truly leveraging the power of immersive technologies to help improve health care. Combining highly accurate, VR models with actual patient data, the emerging capabilities of AR and AI, and new concepts such as “digital twins” will result in life altering outcomes. Chapman sums it up best, “We can only begin to imagine what comes next.”
Lisa Peyton is an immersive media strategist and media psychologist focusing on the business applications of new technologies.
