Future imperfect
Virtual reality is changing the face of medical
education and surgical practices. Sukhmet
Panesar, Anish
Shah, and Iain Mckay-Davies look to
the future
The year is 2020 and Mrs
Barrymore lies on the operating table, awaiting her laparoscopic
cholecystectomy. She was initially concerned that Mr Stewart is only in his
first year of specialist training, but he reminds her that he has performed
over 50 virtual reality (VR) laparoscopic cholecystectomies on the
department’s VR simulator, with a success rate of over 95%. The
patient is not particularly clear on the meaning of all this; VR vaguely
reminds her of a movie, The Matrix, which she saw two decades ago.
From cockpit to theatre
Virtual reality (VR) is a way for humans to visualise,
manipulate, and interact with extremely complex data. Originally born out
of the need to familiarise pilots with an aeroplane’s instrumentation
in the 1930s, as a form of mechanical flight simulation, virtual reality
has followed a path of rapid technological development, with the major
advances in digital simulation taking place since the 1980s. These have
included everything from space flight simulators designed by NASA, to Fish
Tank VR (a small virtual environment displayed on a computer monitor that
tracks your head movements and shifts the angle of view and perspective
accordingly).
However, the idea of applying it to medicine was
developed about 15 years ago, by groups at the University of North Carolina
and the US Department of Defense, who envisaged surgeons rehearsing
complicated procedures using VR headsets.1
ERIC RISBERG/AP
A laparoscopic impulse device coupled with a
virtual surgery simulator - now that's what I call science
Unlike pure three dimensional visualisation, VR
employs haptics. Robotic arms convey the relative sensations of pressure
and force back to the instruments in the user’s hands (also known as
force feedback).
Every VR device is unique, but in general they all
require a method for inputting the user’s actions, for processing
this information, and for feeding the results of the user’s
interaction within the virtual environment back to the user. Devices for
learning the basics of how to control laparoscopic instruments provide good
examples. The VR equipment transmits information gathered from position
sensors located on or around the handheld surgical instruments, to a
computer that transduces this information to an image of the instruments in
a computer generated artificial environment, usually displayed on a
computer monitor.
The VR environments are currently programmed in low
level dedicated programming languages, such as the Virtual Reality
Modelling Language (VRML), and run on powerful computers. The computers
model how the graphical environments react to the user’s movements
based on either non-physics based models (for example, free form
deformation), or on physics based models (for example, mass spring
deformations). For example, if the latter is used, graphical environments
respond more realistically when tissue is deformed by a (virtual)
laparoscopic tool.
The image wire frames are surface rendered or texture
mapped in real time and displayed either on a monitor, a head mounted
display (to enhance three dimensional viewing), or, for maximum immersion,
within a “cave” (a bit like an iMax cinema).
Remote surgery
Endoscopic surgery coupled with VR is one of the most
rapidly developing areas. Endoscopic surgery is easier to simulate as
surgeons already work with an unnatural view of the patient by looking at
the monitor in front of them. In endoscopic surgery, access is already
limited, tactile feedback restricted, and freedom of movement of
instruments limited. Endoscopic surgery coupled with VR allows for
operations to be performed remotely: in February this year, a woman in
Ontario had laparoscopic antireflux surgery performed by a surgeon who was
almost 400 km away, using telerobotic technology, which will eventually
allow complex specialist surgery to be performed in many of the remote
communities in Canada.2 Another fascinating area is that of virtual endoscopy. Data
from conventional magnetic resonance imaging or computed tomography
machines are combined into a virtual data model that can be explored by the
surgeon as if an endoscope were inserted in the patient. Virtual
colonoscopy has been shown to have a sensitivity of 90% and specificity of
94.6% in detecting lesions that are 10 mm or larger.3
VR is also being used in neurosurgery. It has been
tried on one or two patients per week since 1999 and entails co-registering
an image of the specific part of the brain that the surgeon is interested
in and overlaying that on the patient. It is used preoperatively for
planning and teaching, and intraoperatively for guidance, as shown in the
figure.4
Several other potential uses for VR, in procedural training and competency
assessment, have also been proposed or trialled—including
hysteroscopy, amniocentesis, arthroscopy, epidural injection, and
bronchoscopy5
Surgical education and training is probably the single
most important application for VR in medicine—an area where you can
only imagine the endless advantages. In the aviation industry, trainees
hone their skills without the attendant dangers of flying, and studies have
shown that two hours on the VR simulator are equivalent to one hour in the
air. Evidence about the efficacy of such training measures in medicine has
accumulated: endoscopic simulators such as the Minimally Invasive Surgical
Trainer
-Virtual Reality (MIST-VR) and LapSim have been shown in
randomised trials to improve laparoscopic skills in surgical trainees
and medical students, respectively.6 7
Recently, basic virtual reality simulations have
become available over the internet. Originally developed by Manchester
University, they can be tried out at www.hoise.com/vmwc/projects/webset/articles/websetHome.html. These allow you to practise various practical skills, and
the addition of a haptic mouse enhances the sense of reality. These systems
are in the early stages of development, the full extent of which will
realise the e-classroom, with e-mentors from sites around the world
allowing the sharing of learning experiences.
A virtual walk
Potential for VR in this field is good because the
demands of interaction and detailed visualisation are less stringent than
for surgery. These systems simulate a physical environment; they can be
used as an adjunct in the treatment of phobias, eating disorders,
Parkinson’s disease, and for the rehabilitation of stroke patients. A
remarkable system from Japan, the “Bedside Wellness” system,
allows bedridden patients to take a virtual forest walk while lying on
their backs in bed. An array of three video screens presents the unfolding
view of the forest as the patient gently steps on two foot pedals. To add
even greater detail, there is three dimensional sound of birds, streams,
and wind in the trees, and a slot below the central screen delivers a
gentle breeze scented with pine to the “walking” patient.8
Back to reality
VR techniques currently lack odour and sound feedback.
Also, force feedback is only crudely available. There has been some
progress with the use of Phantom haptic feedback devices, which provide a
degree of tactile feedback, and the development of bespoke haptic devices
specifically for laparoscopic surgery. A need also exists to integrate data
from several sources and to be able to switch instantaneously between the
real and virtual worlds. More work is required to enable better simulation
of the behaviours and characteristics of soft tissues.
Cyber sickness, similar to motion sickness, can have
health consequences for VR users, such as fatigue and vertigo. However,
these effects are usually mild and disappear quickly, and they are likely
to become less important with the development of decreased time lag
(between user actions and picture updates) and improved picture quality.9
Much of the technology used in VR is, of course,
extremely expensive. However, VR may partially replace many current
surgical training modalities, which are themselves expensive—for
example, the acquisition and proper maintenance of cadaveric models for
anatomical and surgical training. Moreover, increasing demand for VR
simulators will surely bring down the cost to levels affordable to most
hospitals in the developed world.
Using data acquired from a specific patient, VR will
soon promise the ability to practice on your patient before treatment. The
evolution and further development of haptics will allow you to touch
virtual organs, sense their textures, and rehearse operative procedures in
an immensely immersive environment. Improved fidelity, primarily with
respect to the quality of graphics and their response to user input, as
well as greater computing power and processor speeds will no doubt add to
the realism. Enhanced programming will overcome problems such as accurately
representing the random nature of bleeding and diathermy smoke, while
incorporating patient specific pathology. The current cost of simulators
(on average $100 000 (£54000; €77000) is a major problem, but in the future it is expected
that VR machines will become more affordable as their popularity grows,
following a similar trend to that of personal computers.
Sukhmeet S Panesar, fifth year medical student, Imperial College London
Email: sukhmeet.panesar@imperial.ac.uk
Anish N Shah, final year medical student, Imperial College London
Email: anish.shah@imperial.ac.uk
Iain Mckay-Davies, final year medical student, Imperial College London
Email: iain.mckay-davies@imperial.ac.uk
We thank Fernando Bello, lecturer in surgical graphics
and computing; Sir Ara Darzi, clinical professor; Simon Bann, specialist
registrar; all: Department of Surgical Oncology and Technology, Imperial
College London.
studentBMJ 2005;13:89-132 March ISSN 0966-6494
- McCloy R, Stone R. Science, medicine and the future. Virtual reality in surgery. BMJ 2001; 323:912-5
- Pirisi A. Telerobotics brings surgical skills to remote communities. Lancet 2003; 361:1794-5
- Pineau BC, Paskett ED, Chen GJ, Espeland MA, Phillips K, Han JP, Mikulaninec C, Vining DJ. Virtual colonoscopy using oral contrast compared with colonoscopy for the detection of patients with colorectal polyps. Gastroenterology 2003; 125:304-10
- Surgical Navigation System. MIT artificial lab collaboration with the Surgical Planning Laboratory of Brigham and Women's Hospital. Available online at http://www.ai.mit.edu/projects/medical-vision/surgery/surgical_navigation.html#pubs (last accessed on 16/06/2004)
- Letterie GS. How virtual reality may enhance training in obstetrics and gynecology. Am J Obstet Gynecol. 2002 Sep;187(3 Suppl):S37-40
- Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004; 91:146-50.
- Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002; 16:1324-8
- Ohsuga M, Tatsuno Y, Shimono F, Hirasawa K, Oyama H, Okamura H. Bedside wellness--development of a virtual forest rehabilitation system. Stud Health Technol Inform. 1998; 50:168-74.
- Nichols S, Patel H. Health and safety implications of virtual reality: a review of empirical evidence. Appl Ergon. 2002 May;33:251-71
