jwworth@informatik.umu.se
Outline the state-of-the-art
Make recommendations
Stimulate discussion and collaborative action.
3 target areas: surgery in general, neurosurgery, mental health
and 3 types of application: training, planning, treatment
(excluded: visualisation of records, architectural planning, robotics)
Simulated behaviours of simulated realities:
increasingly as a result of interactions
"Realisation" in modalities other than vision:
sound, force, touch, smell
The current status, pre-VR: 2D data slices:
Visible Human Slice Stacks - CT scans
Models have to be accurate for purpose:
training, planning, performing, therapy....
Tomography
CT (CAT)
MRI
Ultrasound
Physiological Imaging (PET, SPECT)
Range finders...
But, extracting surface is not easy, and there are no insides!
Immersive Body Parts - EVL eye - Surface model with texture mapping
Boston Dynamics Reach-In Suture Trainer with texture mapping
needs alot of processing power, but is worth it!
e.g. using atlases (e.g. VH),
Identifying the objects - Segmentation, labelling
Segmented and labelled, from VH data by KRDL, Singapore
Talairach and Tournoux Atlas (left) and Schaltenbrand and Wahren Atlas (right)
e.g. registration of multiple sources
When more than one image modality involved: Registration - Voxelman
using physically-based modelling
How do objects behave?
Catheter simulation (da Vinci) from KRDL
A big advantage of object and behaviour simulation is that it allows prediction of outcomes, not just planning the interventions. For example, a face can be visualised after reconstructive plastic surgery to assess the appeal of the results.
Craniofacial surgery simulation from Erlangen Institute, Germany
Mechanically Tethered Displays: The BOOM, University of Illinois
(avoids hands in the way)
Reflection-based display - The Virtual Workbench, KRDL, Singapore
Reflection-based display - Boston Dynamics
Currently cumbersome and coarse,
Used in endoscopy simulators
Endoscopy, Open Surgery
Collaborating: Telemedical Training, Planning, Performing
Training in OT increases risk and produces longer operations
New surgical procedures require training by other doctors,
usually busy with their own clinical work
Difficult to train physicians in rural areas in new procedures
Training opportunities for surgeons are on a case-by-case basis
Animal experiments expensive, anatomy is different
Training Simulators: Practice difficult procedures under computer control
Training can be done anytime
Reduction of operative risk with new technology
Improves surgical morbidity and mortality
Transfer of skills from simulation to actual patient?
Faithfulness hard to achieve:
how good is a training system if is not like the real thing?
Certification?
Force feedback, modelling of soft tissue, sound feedback
Karlsruhe Endoscopic Surgery Trainer
The pictures above illustrate both the value of simulators for training procedures, but also their current weaknesses in terms of realism. To realistically simulate an operation, the method of interaction should be the same as in the real case (as with flight simulators). When this is not the case, the VR can serve as an anatomy educational system rather than a training simulation.
Gatech: Endoscopic Surgical Simulator
These systems focus on training the surgeon in the use of particular medical devices, rather than on training a better awareness of general or specific patient anatomy.
(stereotaxis)
Multiple data sources can be fused
Reduces risk, increases accuracy and speed
University of Virginia "Props" Interface used in pre-operative planning
In pre-operative planning the interaction method need not be realistic and generally is not. The main focus is on exploring the patient data as fully as possible, and evaluating possible intervention procedures against that data, not in reproducing the actual operation. The University of Virginia "Props" interface illustrates this. A doll's head is used in the interaction with the dataset, without any suggestion that the surgeon will ever interact with a patient's head in quite this way.
KRDL VIVIAN: the Virtual Workbench used for stereotactic tumour neurosurgery planning
Of course, the simulation must be accurate. Given this, techniques developed for plannng can sometimes be applied to the prediction of outcomes of interventions, as in bone replacements or reconstructive plastic surgery. Such simulations can also help in training, and in communications between doctors and patients (and their families).
Augmented Reality
University of North Carolina - Ultrasound data seen in HUD superimposed on real world
(degradation of direct view)
Combined neurosurgery planning and augmented reality from Harvard Medical School
Robotics more useful for precision and routine than remotely
(network delays, etc.)
Arguably, the most successful application area to date
(e.g. strokes, Alzheimer's disease)
(e.g. disability coping, physical weakness)
(e.g. acrophobia, anorexia)
(e.g. burn treatment, bed confinement)
Acrophobia project - University of Michigan
VR is sufficently realistic to elicit fear
But encounter (and fear) can be controlled
Avoids over-stimulation
Also highly effective for biofeedback
Automated, standardised tests
Can capture data unavailable through traditional tests
Detour: Brain Deconstruction Ahead by Rita Addison
Scene from Osmose by Char Davies
although products can be incorportated or enhanced
between medics, programmers and HCI/HF people
make a big difference, save money, and make us famous
Techniques:
Application Areas:
| Application Area | Type of Outcome | Scope of Research | Primary Use |
| Surgery in General | Specialised Devices (endoscopy) | Expanded Modalities | Training |
| +
Soft Tissues and Interactive Behaviours + |
|||
| Neurosurgery | "Reach-In" Realities | Increased Fidelity of Complex Structures | Planning |
| Mental/Physical Health and Rehabilitation | Immersive "Holotropic" Realities | More Imaginative Realisations | Treatment |
Text (c) John Waterworth, Informatik, June1998
