Mixed Simulators: Seamlessly Integrating Physical and Virtual Simulation for Training in Procedural Skills and Safety

Sem Lampotang, PhD; David Lizdas; Albert R. Robinson III, MD; Olga Ihnatsenka, MD; Nikolaus Gravenstein, MD

Simulation has become an accepted component of education and training in health care to the point that a formative simulation session is now a part of the high-stakes Maintenance of Certification in Anesthesiology (MOCA) exam. The APSF played an influential and seminal role in the development of mannequin patient simulators (physical simulation) through steering the nascent anesthesia-centric mannequin patient simulator designs at Stanford University and the University of Florida to include vital signs such as breath and heart sounds, eye signs and chest movement.1,2 This heralded patient simulation through an anatomic and physiological design that would also address training needs of other health care disciplines outside anesthesia. The APSF was also instrumental through its research grant program in advancing virtual simulation in the form of the screen-based, web-enabled Virtual Anesthesia Machine simulation3,4 and other screen based simulators.5

Simulation is not an end in itself but a means to acquire skills that can be classified into 3 main types (affective, cognitive and psychomotor) that form the skills triangle (Figure 1). Affective skills are about interacting with other humans such as interpersonal skills, leadership, and teamwork. Cognitive skills involve thinking and the application of knowledge, e.g., decision making, strategy, risk assessment, and risk mitigation. Psychomotor skills center on qualities of “doing” or performance such as manual dexterity, hand-eye coordination, and spatial ability.

Figure 1. The skills triangle and the simulation triangle (in health care).6

Figure 2. The taxonomy for the physicality-virtuality continuum in simulation6 that forms the base of the simulation triangle depicted in Figure 1.

The simulation technologies used to acquire all 3 types of skills in health care fall into 3 main classes (biologic, virtual and physical) that constitute the simulation triangle (Figure 1). The 2 triangles, as represented in Figure 1, map well onto each other. A physical simulator devoid of virtual elements (such as an intubation head or mannequin patient simulator) is ideal for learning psychomotor skills such as intubation or CPR. A virtual simulator such as a computer based trainer (CBT) has no physical, tangible elements. Interactions are mediated via pointing devices such as a mouse or joystick as in the web-enabled Virtual Anesthesia Machine (VAM) simulation.4 Virtual simulators are well suited to convey cognitive skills and knowledge; e.g., they provide the ability to virtually peel off layers of a human body or a piece of equipment to provide insight into hidden or invisible internal anatomy, structures, mechanisms, and processes.4 Standardized patients (human actors having, or pretending to have, a disease condition) are also considered a form of simulation (biologic) and provide an excellent way to hone affective skills such as bedside manners and conducting a history and physical or an anesthesia preoperative assessment. A logical conclusion of our lab’s APSF-funded research in physical and virtual simulations was to combine both forms of simulation to benefit from the advantages of each in a seamlessly integrated mixed simulator (Figure 2).

What is a mixed simulator? A mixed simulator, as the name implies, contains both physical and virtual components.6 A lay example of a mixed environment is the yellow first down line when watching American football on TV. The yellow first down line is virtual and does not physically exist; spectators in the football stadium do not (yet) see a yellow first down line. For the mixed environment to work, the virtual component must be precisely registered (an engineering term) to its physical environment; that is, it should appear to be in the precise location that it would have been had it been physical. Taking the yellow first down line again as an example, if it was superposed a yard off from where it needed to be it would mislead rather than augment our ability to appreciate the game. Within the context of mixed simulators for medical procedures, the precision of the registration must be sub-millimeter; we do not want the wall of a virtual vein to be off by more than 1 mm from where it would be relative to, for example, a virtual lung or a physical rib in a mixed simulator. In addition, in a mixed simulator that represents anatomy, the virtual components such as the brain, lungs, veins, arteries, nerves, and ligaments are 3-dimensional with sometimes complex, convoluted surfaces compared to a one-dimensional first down line.

Mixed Simulator film 1

Mixed Simulator film 2
Concrete video examples of mixed simulators.

The safe and efficient performance of medical procedures such as central venous access (CVA) and regional anesthesia (RA) requires both 1) cognitive skills (e.g., correct mental model of complex 3D anatomy, defensive strategy [If I miss the vein, will I strike the first rib or the lung along this trajectory?], and correct interpretation of a cross-section produced by handheld ultrasonography during guided procedures) and 2) psychomotor facility (right and left hand dexterity, coordination in advancing the needle and moving the US probe simultaneously while keeping the needle tip visible in the insonation plane, and spatial ability to think and visualize in 3 dimensions).Thus, procedural simulators are a good fit for mixed simulators that combine the 2 forms of simulation technology (physical and virtual) best suited for acquisition of procedural skills (psychomotor and cognitive respectively). Concrete examples of mixed simulators are a CVA7 and an RA simulator; see videos at http://simulation.health.ufl.edu/research/cvl_intro.wmv and http://simulation.health.ufl.edu/research/ra_sim.wmv.

Both the RA and CVA mixed simulators are anatomically authentic as the 3D anatomical components are derived from medical imaging (MRI, CT). The physical components of the mixed simulator such as the ribs, skull, and skin are reproduced via a 3D printer that creates a high fidelity replica of the human model. Many medical procedures involve a needle or catheter with a metal stylet. The tip position (x,y,z) and orientation (roll, yaw, pitch) of a physical needle are tracked with a miniature magnetic sensor with sub-millimeter resolution so that if the tip is inside the space occupied by a virtual 3D component such as a vein, then the correct feedback such as a blue-tinged flashback in the syringe can be simulated. A 3D color visualization (real-time visual augmentation) is part of the mixed simulator design and allows users to visualize a virtual replica of the needle interacting with the internal components in both “blind” and guided procedures. The simulator captures all the user’s actions during the procedure, enabling playback of the procedure for after action review (debriefing) as well as an automated, objective, and transparent scoring algorithm. In general, the soft tissues are simulated as virtual components while the bony structures such as the clavicle, sternal notch, and spinous processes are implemented as physical components and can be palpated and sometimes used as anatomical landmarks. The mechanical interaction between the physical needle and the physical structures in the mixed simulator provide an inexpensive method to provide tactile feedback to users without needing a haptic device. Ultrasonography, used for guided procedures such as CVA and RA, is readily incorporated into a mixed simulation and further enhances the capabilities and realism of the mixed simulator.

The CVA simulator was awarded the First prize for the Best Scientific and Educational Exhibit at ASA 2011 and the RA simulator the APSF Ellison Pierce Award for Best Patient Safety Exhibit at ASA 2013. In a simulated environment, the CVA simulator reduced the incidence of iatrogenic pneumothorax during central venous access,7 a complication recently classified as a Serious Reportable Event (formerly known as a “never event”). By combining the best of virtual and physical simulation, mixed simulators represent the next generation of patient simulators and training tools and hold the promise of further enhancing patient safety by enhancing cognitive (better mental model of 3D anatomy and safe techniques through visualization) and psychomotor skills (keeping the needle tip in the insonation plane) during medical procedures. The crucial and timely funding provided by the APSF research grant program to nurture simulation research continues to bear fruit and advance patient safety.

Sem Lampotang, PhD, David Lizdas, Albert R. Robinson III, MD, Olga Ihnatsenka, MD, Nikolaus Gravenstein, MD, Department of Anesthesiology, Center for Safety, Simulation & Advanced Learning Technologies, University of Florida.


  • Gravenstein JS. Simulators studies for role in patient safety. APSF Newsletter 1988;3(1):7-8. Available at: https://www.apsf.org/newsletters/html/1988/spring/
  • Cooper JB. APSF awards four 1989 patient safety research grants. APSF Newsletter 1988;3(4):34. Available at: https://www.apsf.org/newsletters/html/1988/winter/
  • Brull SJ. APSF awards 4 grants. APSF Newsletter 2004-05;19(4):46-47. Available at: https://www.apsf.org/newsletters/pdf/winter2004.pdf
  • Fischler IS, Kaschub CE, Lizdas DE, Lampotang S. Understanding of anesthesia machine function is enhanced with a transparent reality simulation. Simul Healthc 2008;3:26-32.
  • Keats AS. APSF awards three research grants. APSF Newsletter 1987;2(4):32. Available at: https://www.apsf.org/newsletters/html/1987/winter/
  • Lampotang S, Lizdas D, Rajon D, Luria I, Gravenstein N, Bisht Y, Schwab W, Friedman W, Bova F, Robinson A: Mixed Simulators: Augmented Physical Simulators with Virtual Underlays. Proceedings of the IEEE Virtual Reality 2013 meeting 978-1-4673-4796-9/13: 7-10, 2013.
  • Robinson AR III, Gravenstein N, Cooper LA, et al. A mixed-reality part-task trainer for subclavian venous access. Simul Healthc 2014;9:56-64