doi: 10.56294/gr20238
REVIEW
Augmented reality for surgical skills training, update on the topic
Realidad aumentada para la formación de habilidades quirúrgicas, actualización del tema
Carlos Jesús Canova
Barrios1,2 
*, Mariana Pilar
Hereñú1 ,
Sabrina Macarena Francisco1
1Universidad de Ciencias Empresariales y Sociales (UCES). Buenos Aires, Argentina.
2Grupo de Investigación en Educación en Enfermería. AEUERA.
Cite as: Canova Barrios CJ, Hereñú MP, Francisco SM. Augmented reality for surgical skills training, update on the topic. Gamification and Augmented Reality. 2023; 1:8. https://doi.org/10.56294/gr20238
Submitted: 19-07-2023 Revised: 23-09-2023 Accepted: 29-11-2023 Published: 30-11-2023
Editor:
Adrián Alejandro Vitón-Castillo
ABSTRACT
Augmented reality (AR) combines digital information with physical reality, allowing users to interact with virtual data in their real environment. In medical education, this technology is attractive because it allows access to information without taking your eyes off the surgical field. The integration of surgical simulators in resident training offers flexible practice without direct supervision, with benefits such as objective performance evaluation, practice of unusual procedures, and the development of non-technical skills. In addition, simulators are useful for teaching new techniques to experts. In summary, AR and simulators offer valuable opportunities to improve surgical training. The objective of this review is to update the current state of augmented reality in surgical training. The use of augmented reality as an assessment tool in surgical training presents interesting perspectives that deserve consideration. AR can provide objective performance metrics by measuring the technical competency of surgical trainees, whether in a simulated operating environment or in real-world situations. This offers the opportunity to address subjective variability and potential bias in current assessment methods, which often rely on supervisor observation and rating. However, despite these advantages, there has not yet been a comprehensive review to evaluate the use of AR in surgical training. The cost-benefit and implications for data management have not yet been addressed.
Keywords: Augmented Reality; Training; Surgery; Surgical Techniques.
RESUMEN
La realidad aumentada (RA) combina información digital con la realidad física, permitiendo a los usuarios interactuar con datos virtuales en su entorno real. En la educación médica, esta tecnología es atractiva porque permite acceder a información sin apartar la vista del campo quirúrgico. La integración de simuladores quirúrgicos en la formación de residentes ofrece práctica flexible y sin supervisión directa, con beneficios como la evaluación objetiva del desempeño, la práctica de procedimientos inusuales y el desarrollo de competencias no técnicas. Además, los simuladores son útiles para enseñar nuevas técnicas a expertos. En resumen, la RA y los simuladores ofrecen oportunidades valiosas para mejorar la formación quirúrgica. El objetivo de esta revisión es actualizar el estado actual de la realidad aumentada en el entrenamiento quirúrgico. El uso de la realidad aumentada como herramienta de evaluación en la formación quirúrgica presenta interesantes perspectivas que merecen consideración. La RA puede proporcionar métricas objetivas de rendimiento al medir la competencia técnica de los aprendices quirúrgicos, ya sea en un entorno operativo simulado o en situaciones reales. Esto ofrece la oportunidad de abordar la variabilidad subjetiva y el posible sesgo en los métodos actuales de evaluación, que a menudo dependen de la observación y calificación de los supervisores. Sin embargo, a pesar de estas ventajas, aún no se ha realizado un examen exhaustivo paraevaluar el uso de la RA en la formación quirúrgica. Aún no se han abordado son el costo-beneficio y las implicaciones para la gestión de datos.
Palabras clave: Realidad Aumentada; Entrenamiento; Cirugía; Técnicas Quirúrgicas.
INTRODUCTION
Surgeons traditionally learn surgical skills through the "see one, do one, teach one" method by William Halsted.(1,2) With this, surgeons receive extensive training under the supervision of an experienced surgeon to become familiar with these skills.(3)
However, Halsted's model, the gold standard for surgical science resident training, must be updated for several reasons, including the impact on patient comfort, procedure length, operative time and cost, and the potential for complications.1 Lack of resident training time, limited educational resources at each surgical center, and increased patient safety and quality demands are also limiting for this approach.(3)
By its nature, surgery is a highly visual and tactile specialty that requires a firm understanding of the three-dimensional arrangement of anatomic structures and their relationships to one another. While cadaveric models and expert guidance remain the gold standard of formal surgical education, many centers are investigating novel digital approaches to augment traditional teaching practices and inspire the next generation of surgeons.(4)
The dynamic nature of many surgical procedures demands careful judgment, professional knowledge, and high levels of care.(1) Simulation-based surgical skills training is an educational approach that can allow learners to experience a given task or situation in a safe environment. It provides standardized, reproducible content for practicing skills, allowing trainees to practice skills acquired repeatedly; its importance in resident education is increasingly recognized, and previous studies have demonstrated its effectiveness.(3)
Augmented reality (AR) is a technology that expands the physical world by overlaying digital information such as text, computer-generated images, and 3D graphics onto the user's real-world view, providing a composite view.(1,2,5,6) The blending of digital information with real-world stimuli to create a mixed reality environment that allows the user to interact with virtual information in the context of their real-world environment.(4)
This technology allows users to interact with information in the real world, and medical education, it is attracting attention because users do not have to pause or look away from the surgical field when accessing information while practicing.(3,7)
Incorporating surgical simulators into resident curricula provides greater flexibility and practice without the supervision of an experienced surgeon. Additional benefits include the opportunity to fail at any time without consequences, objective performance evaluation, the creation of unusual surgical procedures, and repeated practice. In addition, simulators are also helpful in teaching new techniques to experts. Finally, simulators can also help develop non-technical skills, including teamwork and communication.(1,7)
The objective of this review is to update the current status of augmented reality in surgical training.
METHODOLOGY
A search for information was carried out in January 2024 in the databases Redalyc, Elsevier Science Direct, PubMed/Medline, and SciELO, as well as the ClinicalKeys services and the Google Scholar search engine. Advanced search strategies were used to retrieve the information by structuring search formulas using the terms "augmented reality," "training," "surgery," and "surgical techniques," as well as their equivalents in English. From the resulting documents, we selected those that provided theoretical and empirical information on cultural competence in health education in Spanish or English.
DEVELOPMENT
Augmented reality has been widely used for several years to guide surgical procedures and build platforms for skills training.(6) This technology has been growing steadily for years, backed by significant investments from technology companies with expected but unproven advantages, including cost savings, reduced complication rate, comprehensive knowledge acquisition, and improved surgical performance.(4,8)
As with introducing any new technique in surgery, patient safety must be a priority. Regarding the rate of postoperative complications, AR has not been associated with severe complications. The rate of minor complications is not reported to differ from historical data. This is an important concept to consider as it supports the safe introduction of this new technology.(4,8)
Real-time visual feedback that provides more information to trainees may influence their performance and would be more advantageous than post-exercise feedback. Physiologic demand in muscle effort can be reduced by using augmented visual feedback in surgical training. However, distraction and stress are two main negative factors that should be considered, along with all the potential advantages of real-time feedback.(6,9)
Augmented reality can also enhance surgeons' field of view and minimize visual misperception by augmenting preoperative patient data on the patient's anatomic structures. A system that achieves this robustly and accurately would be an essential tool for surgeons, leading to safer operations and minimizing implications and risks.(7)
AR allows complementary data to be incorporated into the surgeon's real-world sensory inputs and has been integrated into orthopedic procedures and surgical training. This commonly includes overlaying valid visual data, such as relevant images, on the surgeon's field of view, but can also include auditory or sensory feedback, intraoperative navigation, and telementoring or guidance.(10)
The primary focus of surgical curricula has been the acquisition of technical skills. However, surgical training methods have yet to be developed to train residents to avoid making errors during surgery. It should be essential to train situational awareness, as errors result from misperceptions and suboptimal problem-solving strategies. Modern operating rooms are enriched with an enormous increase in new technologies. This increases the incoming signals and, thus, the mental workload during the performance of surgery. AR enables the transfer of digital information into the real world, thus combining two worlds. In turn, this creates opportunities to filter information from the environment because additional information is within the surgeons' field of view.(11)
AR is well-suited for training programs aimed at situational awareness. Situational awareness training in high-risk environments, such as the operating room, is much needed but needs to be improved in medical education curricula. The benefit of AR could be widespread, from training better surgeons to making fewer mistakes in the OR, ultimately leading to improved patient safety. AR is a new technology in educational methodology. It has demonstrated the enormous potential within the medical field. Undoubtedly, healthcare will be profoundly affected by the evolution of AR.(11)
AR offers a significantly better user experience for hands-on clinical skills training than virtual reality, given AR's ability to provide honest feedback, enhanced haptics, and lack of reliance on complex digital graphics that are often unrealistic.(4)
Many researchers agree that AR can provide real-time graphical guidance to surgeons, help them link images to the surgical scene, show trajectories or cut margins, and minimize anatomical ambiguities. All of these functions create substantial ways to improve the accuracy of the operation to maximize patient safety inside the operating room.(6)
When applied to skills training, AR can guide trainees to practice a complex procedure without frequent pauses in the middle of practice to seek instruction. The necessary instruction can be shown to trainees in their lateral view. This AR feature is helpful for healthcare learners when practicing a procedure composed of multiple steps, such as delivering a baby or inserting a chest tube.(6)
Theology could likely be related to medical specialty. For example, trainees in urology, a branch of surgery that uses endoscopic and robotic techniques for many procedures, might benefit more from using AR in skill acquisition and performance than trainees in other specialties. The literature still needs to be more sparse for meaningful comparisons of AR training in different specialties.(12)
Simulation-based training may be one of the practical educational methods not only to promote patient safety but also to prevent medical errors. Simulation training is widely used in various fields, such as aviation, military, industry, and medicine. It has become common in medical education. Several benefits of medical simulation are commonly accepted:(13)
· minimize ethical issues,
· enhance the educational experience,
· provide learner-centered education and a supportive environment for teachers,
· provide a risk-free environment for the patient,
· enable users to learn and practice new techniques,
· enable performance evaluation.
Augmented Reality Simulators (ARS)
Today, virtual, augmented, and mixed reality (MR) surgical simulators have been developed. AR benefits preoperative surgical preparation, providing useful outcome predictions and intraoperative navigation to minimize potential risks. MRI has enabled three-dimensional (3D) images to be completer and more accurate, thus improving surgical navigation and preprocessing.(1)
Available systems that support surgical procedures using augmented visualization use virtual reality headsets, goggles with translucent displays, or external computer displays. For example, head-mounted displays such as Google Glass or Microsoft Hololens are used in medical procedural training research. Although headsets are often reported to be helpful, they create a physical barrier between surgeons and their environment, imposing an undesirable situation during medical procedures. In addition, most translucent devices require additional external tracking markers to position virtual structures accurately, especially in changing environments. Most of these devices are uncomfortable to wear and use for prolonged periods and often cause eyestrain, headaches, and substantial discomfort. Also, external displays can cause discomfort due to frequent shifting of the surgeon's focus between the patient and the screen, thus prolonging operating time.(8)
In an ARS, a learner can use real surgical instruments with physical materials while performing a task, e.g., instrument navigation, clipping, and pin transfer. Meanwhile, virtual environments are overlaid to express surgical scenes, such as human anatomy.(13)
Microsoft Hololens
HoloLens is a head-mounted augmented reality display. The smart glasses allow a high level of user interactivity through natural interface commands with gaze, gesture, and voice inputs. Training has been used in ureteroscopy, neurosurgery, cholecystectomy, urogynecologic surgery, and lung adenocarcinoma procedures. It has been reported to significantly improve economy of motion and error rates and overall user performance: shorter response time, more positive emotions, and less cognitive load and effort.(14)
Augmented Reality Telementoring System
The Telementoring System with Augmented Reality (STRA) projects operational instructions directly into the user's field of vision. Used in the context of trauma, cricothyroidotomy, and basic surgical skills. Users have demonstrated fewer positioning errors, fewer changes in focus, and greater confidence during the procedure.(14)
Immersive Touch System
The ImmersiveTouch system uses specialized glasses and a robotic stylus to immerse the user in an interactive 3D environment with haptic feedback. Several studies have evaluated this simulation platform in the context of neurosurgery and trauma. The overall performance of ImmersiveTouch in terms of validity and translational outcomes is promising.(14)
Augmented Reality Promis Simulator
The ProMIS simulator retains all the qualities of a traditional box trainer but has the added benefit of providing objective feedback on performance measures. ProMIS has been used to assess and train basic laparoscopic skills. Studies show overall improvement and higher scores.(6)
Google Glass
Google Glass, also known simply as Glass, is a pair of lightweight smart glasses extensively tested in medical research. Their use in urology and basic surgical skills has been described.(14)
Perk Station
The Perk Station simulator was designed for training in percutaneous surgery. Studies found that with the AR simulator, there was less tissue damage and a better success rate than the comparison in both orthopedic and neurosurgical training contexts.(14)
Art Platform
ART Platform is a simulator designed to superimpose a mentor's instruments on the trainee's laparoscopic monitor. Randomized control trials demonstrated faster skill acquisition, quicker task completion time, fewer errors, and a shorter and steeper learning curve than traditional methods.(15)
Encouragingly, the efficacy of AR in surgical skill acquisition through tutoring/telementoring, which provides real-time tutoring for surgical procedures by an expert at a remote location,(14) is encouraging.
Applications
Neurosurgery
Advanced techniques and algorithms in virtual reality, augmented reality, and MRI provide neurosurgeons with the ability to perform accurate surgical planning, intraoperative monitoring, and postoperative follow-up. Virtual and augmented reality simulation has been applied in endoscopic and cranial tumor neurosurgery. There are clinical applications concerning bone dissection, cerebral aneurysm clipping, microvascular decompression, and pedicle screw placement. AR has been helpful in neurooncologic procedures to identify lesions, direct resection, schedule craniotomy, and skin incision before surgery. Image-guided AR neuronavigation systems have been developed for vascular surgery, external ventricular drainage placement, and other applications in cerebral aneurysms, spine surgery, tumor resection, intracranial meningioma, and craniosynostosis surgery.(3)
Maxillofacial Surgery
AR in maxillofacial surgery is beneficial for preoperative planning by providing practical outcome prognostics and intraoperative navigation to minimize potential risks. It has also been applied in dental implantology and visualization of alveolar nerve bundles.(3)
General surgery
It has been reported that AR is mainly used to recognize lesions. AR navigation has been used to improve protection from surgical dissection in pancreaticoduodenectomy, pancreaticoduodenal artery examination, hilar cholangiocarcinoma intervention, and open urologic and liver surgeries.(3)
Laparoscopic and minimally invasive surgery
Unlike open surgery, laparoscopic surgery requires a different skill set and is sometimes more complicated than open surgery. Reduced working hours, fewer training sessions, and patient safety issues result in acquiring these skills outside the operating room. Therefore, both virtual reality and augmented reality modeling have been leveraged. Laparoscopic liver surgery has been one of the main applications of AR. Virtual and augmented reality has also been applied in laparoscopic distal pancreatectomy and gynecologic laparoscopic surgery.(5)
The recent use of AR in minimally invasive surgery has led to the development of hybrid image-guided surgery using endoscopic and robotic video transmissions. An institution dedicated to image-guided hybrid therapies has been developed. However, the delivery of AR in open surgery will require alternative technology for surgeon interface.(5)
The role of AR as an assessment tool merits consideration since objective performance metrics can be measured using AR platforms either in the simulated operating environment or, preferably, in real life to measure technical competence. Consequently, there is an opportunity to address the inherent subjective variability and potential bias of current methods of evaluating surgical trainees, which rely on supervisors observing trainees and rating competence concerning technical expertise, clinical judgment, professionalism, and ethical interpersonal skills.(14)
Despite these exciting perspectives, a comprehensive review to evaluate the use of AR in surgical training has yet to be conducted. 4 Two critical issues have not been addressed to date: cost-benefit, which will need to be weighed against the promise of increased speed of skill acquisition and large-scale implementation of new data-intensive technologies; implications for data management, including ownership and confidentiality, as well as infrastructure requirements, must also be considered.(12)
CONCLUSIONS
Augmented reality is a technology that combines the physical world with computer-generated virtual elements, allowing users to visualize additional information about their environment in real-time. In training surgical techniques, augmented reality offers a valuable tool to improve the accuracy and efficiency of procedures. Augmented reality provides detailed visual guidance during surgery, which can help reduce errors and minimize the risk of complications. In addition, this technology makes it possible to simulate complex surgical scenarios and provide real-time feedback, which facilitates learning and practicing new techniques.