This first virtual reality scoliosis surgery simulator was developed by Alexandre Plouznikoff, as his master's research project at the École Polytechnique de Montréal, under the supervision of Prof. Ozell (computer and software engineering department) and Prof. Aubin (mechanical engineering department).

The project started in summer 2001, in collaboration with the Centre for Research on Computation and its Applications (CERCA) and the Biomedical Modeling and Computer Assisted Surgery Lab (BMCASL), located at the Montreal Sainte-Justine Hospital.

Usefulness of a simulator from a surgeon's point of view

Because results of scoliosis surgeries vary greatly depending on a patient's spine configuration, its rigidity and a surgeon's decisions (implants, corrective forces and surgical procedures), we're trying to develop a tool to easily test and plan different surgical maneuvers before surgery, so as to help surgeons identify the best treatment for specific patients.

Moreover, we hope that this simulator will help surgery residents practice and gain confidence before operating on their own near the spinal chord.

Why use virtual reality ?

Virtual reality, compared to traditional computer graphics, can provide a more realistic environment for a surgeon to plan a surgery. Although the equipment can be expensive compared to conventional PCs, virtual reality can immerse a user within a known virtual surgery room, where he'll be able to operate normally using his own hands and traditional surgical tools, and to feel the forces applied on a spine to correct its configuration. The realism of virtual reality comes not only from the graphics but also from the interactions and the feedback we can provide, making the experience almost real, with a near-flat software learning curve.

Scoliosis is a tridimensional deformation of the spine that, as of today, can not be prevented or treated using medications, as its causes aren't yet fully understood. Although scoliosis incidence is low and the number of cases that require surgical treatment even lower, the ones that do are life threatening and need to be treated before complications occur (ex. compression of the rib cage, leading to asphyxia). Scoliosis surgical treatments basically try to rectify the curve of the spine using metal rods, hooks and screws. Each screw is inserted within a vertebra pedicle, close to the spinal chord. A rod is then linked to the screws to impose a more natural configuration to the spine. To obtain additional details on scoliosis, its treatments and the ongoing research, visit the Scoliosis Research Society website (http://www.srs.org/).

Frontal radiograph of a scoliotic spine (before surgery)	Frontal radiograph of a scoliotic spine (after surgery)

The CAVE (Cave Automatic Virtual Environment) is surely the most advanced virtual environment available today. Compared to traditional HMDs (Head Mounted Displays), a CAVE, usually composed of four 3x3 meter walls on which stereoscopic images are displayed, provides a greater sense of immersion and leads to better interactions, enabling the user to grab objects with his own hands. The user's head and hands are tracked electromagnetically to allow the software to determine exactly with which object the user is interacting and how to adjust the stereoscopic perspective on each wall. The École Polytechnique acquired a SGI Onyx 2 driven CAVE almost three years ago (1998), making it one of the first five CAVE owners in Canada.

Peripherals used in virtual reality environments (from left to right: glasses filtering stereoscopic images, a CyberTouch providing tactile feedback, a CyberGrasp giving virtual objects a physical size and shape, and a CyberForce providing force feedback)

The current surgery simulator uses detailed graphic files to represent the different objects located in the virtual operating room and is divided into three main modules.

Graphic files

An operating room from the Montreal Sainte-Justine Hospital has been modeled in 3D, including the surgical table, lights, armoires... The surgical implants have been provided by Medtronic Sofamor Danek, one of the leading manufacturers of scoliosis surgical implants. Finally, a 3D reconstruction of a patient's spine has been provided by the Sainte-Justine Hospital.

Model of a patient's scoliotic spine, reconstructed using frontal and lateral radiographs	Medtronic Sofamor Danek implants used during scoliotic spine surgeries

Graphics loader module (Terra)

We're trying to keep all graphical objects in the VRML (Virtual Reality Modeling Language) format, as it is a standard in virtual reality software. We developed a simple dedicated VRML parser to load our models.

Interactions module (Phoenix)

Phoenix is at the core of the simulator. This interactions module is based on the CAVELib, which handles the user's head and hands position. As of now, Phoenix is based on a user predefined objects hierarchy, allowing user/object interactions, object/object interactions and intelligent autolink functions. A simple gravity engine has also been implemented to mimic basic physics laws inside the operating room.

Phoenix also provides additional features to help surgery planning like holographic views to zoom in regions of interest and automatic movement demultiplication to increase its precision when inserting pedicular screws.

Virtual operating room in the CAVE	User inserting a pedicular screw

Biomechanical module

The current spine biomechanical model has been developed by Valeri Goussev, researcher at the BMCASL. To date, the biomechanical properties have been modeled under Matlab, using a nonlinear mathematical model. We're trying to keep the model as precise and as light as possible to be able to query it and obtain the new vertebrae positions and orientations in real-time. A preliminary version of this model has been interfaced with Phoenix to make our virtual spine answer realistically to corrective forces.

Matlab biomechanical model querying, when applying a corrective force on the 8th thoracic vertebra (T8)

Relay of the biomechanical module's answers to the virtual spine, following the application of a corrective force

• C-E. Aubin, V. Goussev and Y. Petit, "Biomechanical modelling of segmental instrumentation for surgical correction of 3D spinal deformities using Euler-Bernoulli thin-beam elastic deformation equations." Model. and Sim. in Biomedicine, 2004, vol. 42, issue 2, pp. 216-221.
• A. Plouznikoff, "Simulateur de chirurgie d’instrumentation du rachis scoliotique en réalité virtuelle ", Master's Thesis, Department of Biomedical Engineering, École Polytechnique de Montréal, Montreal, Canada, May 2004, 152 pages.
• A. Plouznikoff, C-E. Aubin, B. Ozell and H. Labelle "Virtual Reality Scoliosis Surgery Simulator", in Proceedings of the 5th International Research Society of Spinal Deformities (IRSSD’04), Vancouver, Canada, IOS Press, June 10th-12th 2004, pp. 139-142.