doi: 10.56294/gr202315

REVIEW

Augmented reality in surgery: improving precision and reducing risk

Realidad aumentada en la cirugía: mejorando la precisión y reduciendo el riesgo

Jose Ignacio Robaina Castillo¹ *

¹Universidad de Alcalá de Henares. España.

Cite as: Robaina Castillo JI. Augmented reality in surgery: improving precision and reducing risk. Gamification and Augmented Reality. 2023; 1:15. https://doi.org/10.56294/gr202315

Submitted: 01-08-2023          Revised: 06-10-2023          Accepted: 28-11-2023          Published: 29-11-2023

Editor: Adrián Alejandro Vitón-Castillo

ABSTRACT

Introduction: augmented reality is applied in different spheres and provides broad possibilities as educational technology, it is a way to interact with physical reality in real time, having multiple applications in the field of medicine.

Objective: characterize the application of augmented reality in the field of surgery.

Method: a review of the available literature was carried out using synthetic and historical-logical analytical methods using articles recovered from databases such as SciELO, Dialnet, Scopus, Researchgate, recovering a total of 15 reference articles from available literature related to the topic. in question, included in the time frame between 2018 and 2024.

Results: augmented reality can be used as a tool to facilitate the visual positioning of surgeons in the intervention of minimally invasive surgeries given the continuous evolution of medicine towards minimally invasive treatments, computing is increasingly present. Augmented reality (AR) in medicine facilitates the preparation and development of surgical operations, plays a very significant educational role, is characterized by being a cognitive process with which one learns at the same time as performing the activity, improves training surgical and effectiveness.

Conclusions: currently and in the future, augmented reality constitutes a necessary tool for preclinical training, due to its application in different fields of medicine, including surgery, guiding this technology to improve clinical and surgical capabilities of professionals.

Keywords: Surgery; Augmented Reality; Technology; Virtual Reality.

RESUMEN

Introducción: la realidad aumentada se aplica en diferentes esferas y brinda amplias posibilidades como tecnología educativa, es un modo de poder interactuar con la realidad física en tiempo real teniendo múltiples aplicaciones en el campo de la medicina.

Objetivo: caracterizar la aplicación de la realidad aumentada en el campo de la cirugía.

Método: se realizó una revisión de la bibliografía disponible utilizando los métodos analíticos sintético e histórico lógico mediante los artículos recuperados desde las bases de datos como SciELO, Dialnet, Scopus, Researchgate, recuperándose un total de 15 artículos referenciales de literatura disponible relacionados con el tema en cuestión, comprendidos en el marco de tiempo entre el 2018 y 2024.

Resultados: la realidad aumentada puede ser usada como herramienta para facilitar el posicionamiento visual de cirujanos en la intervención de cirugías mínimamente invasivas dada la continua evolución de la medicina hacia tratamientos mínimamente invasivos, cada vez la informática está más presente. La realidad aumentada (RA) en medicina facilita la preparación y el desarrollo de operaciones de cirugía, juega un papel educativo muy significativo, se caracteriza por ser un proceso cognitivo con el que se aprende al mismo tiempo que se realiza la actividad, mejora la formación quirúrgica y la eficacia.

Conclusiones: actualmente y a futuro la realidad aumentada constituye una herramienta necesaria para el entrenamiento preclínico, por su aplicación en diferentes campos de la medicina, dentro de los que se encuentra la cirugía, orientando esta tecnología a mejorar capacidades clínicas y quirúrgicas de los profesionales.

Palabras clave: Cirugía; Realidad Aumentada; Tecnología; Realidad Virtual.

INTRODUCTION

Augmented Reality, or augmented Reality (AR), is already heard and applied in different spheres and offers vast possibilities as an educational technology. It mixes the natural environment (what can be seen in Reality) and the virtual environment (existing only apparently without being honest). It is a way to interact with physical Reality in real-time. It is used to define a view through a technological device, directly or indirectly, of a real-world physical environment, whose elements are combined with virtual elements to create a real-time mixed reality.⁽¹⁾

In 1901, Frank L. Baum designed a character-maker device that could already be considered a prototype of what is today AR. This device consisted of a large electronic viewfinder that allowed information to be superimposed on the people it focused on.⁽²⁾

The increase in systems that merge Reality with virtual Reality has led to the application of new technologies such as Virtual Reality (from now on VR) and Augmented Reality (from now on AR), where their popularity has attracted the attention of researchers in the health area, indicating that in dentistry, the applications of these new technologies generate many benefits. Augmented Reality is a technique where authentic medical images are enlarged with enriched information that the surgeon cannot see in the real world; this information is generated by a computer and displayed on a display device, merging authentic images with virtual ones.⁽³⁾

There are many fields of application: manufacturing, maintenance, and repair of machinery, equipment, components, entertainment, advertising, education, and medicine. Some of the applications of augmented Reality in medicine are games to help patients with different disorders, simulators, geo-location, 3D vision, and scanners, among others, many of them incorporated into the equipment already available to health services. In each case, its role in staff training and the facilities it provides for learning is always clear, and when it comes to medicine, it also includes medical education.⁽¹⁾

The purpose of AR is to superimpose three-dimensional images on the images provided by the camera. This technique is being used in medical applications mainly because it generates an increase in the visual signals perceived by the surgeon with an enlargement of his visual perception, which is reflected in greater precision and success in the interventions; in addition, it improves the surgeon's hand-eye response thanks to the enlargement of Reality. In contrast to virtual Reality, which recreates a largely artificial environment, augmented Reality enriches Reality, which is more acceptable to the human eye and brain.⁽⁴⁾

The era of digital surgery is characterized by the implementation of new technologies that can improve preoperative planning, increase the availability of therapeutic alternatives, improve surgical training of trainees, optimize postoperative patient outcomes, and reduce possible adverse events. Although the central premise of incorporating these technologies is to improve patients' clinical results, the use of these advances has been accelerated by commercial interests and by the opportunities that large companies have to generate profits worldwide.⁽⁵⁾

Therefore, this review article aims to characterize the application of augmented Reality in the field of surgery.

METHOD

A review of the available literature was conducted using the synthetic and historical-logical analytical methods using articles retrieved from databases such as SciELO, Dialnet, Scopus, and Researchgate, retrieving a total of 15 referential articles of available literature related to the topic in question, comprised of the time frame between 2018 and 2024. Filters were used for the selection of articles in English and Spanish languages. The terms "Surgery," "Augmented Reality," "Technology," and "Virtual Reality" were used as keywords in the article.

RESULTS

AR integrates signals captured from the real world with signals generated by computers, matching them in the construction of new coherent realities, which complement and coexist in the natural and virtual worlds, enriching cognitive experiences in the visual order and undoubtedly improving the quality of communication in the context in which students and teachers work.⁽¹⁾

The human brain more widely accepts augmented reality because it preserves much of the natural visual pathway, in contrast to another popular technology called virtual reality, which is characterized by creating an artificial environment and vision. In recent years, AR medical applications have rapidly expanded, led by advances in hardware (interfaces, haptics, and displays), at the same time as smartphones. Smartphones and tablets have become increasingly popular tools for AR applications in medicine, industry, and education.⁽⁶⁾

An essential example of the application of this technology is computer-assisted surgery. This concept encompasses a set of methods that use computer technology for pre-surgical planning and to guide or perform interventions in real-time, also known as computer-assisted intervention, image-guided surgery, or surgical navigation. Guided systems allow surgeons to simulate, educate, and plan in the preoperative period and guide, assist, execute, and decide in the postoperative period.⁽⁴⁾

León Araujo et al.⁽³⁾ determined in their study that maxillofacial surgery and implantology help us to effectively and efficiently recognize the dental instruments, as well as the anatomical identification of where the implant will be placed; in pediatric dentistry, it motivates pediatric patients to acquire more knowledge about brushing techniques and thus prevent oral diseases; in dental surgery, it improves preparation skills, removal and restoration of dental pieces; in oral rehabilitation it helps us to digitize three-dimensional models for later rehabilitation.

Aguilar et al.⁽⁴⁾ four states that there is a growing use of this technology in the surgical area, not only in training stations but also in guided surgeries, supporting the work of surgeons. This is mainly due to the creation of 3D images from 2D medical images and medical data in real-time, which leads to recreating an environment with more visual information for the surgeon, allowing him to make better decisions within the surgical interventions.

Furthermore, Aguilar et al.⁽⁴⁾ recognized in their study that future research in simulation and training with AR is directed, on the one hand, towards visual realism due to the limiting role of recreating a world close to reality and, on the other hand, towards haptic feedback, as well as dynamic tracking of markers and organs for augmentation with a high degree of affinity with reality.

AR and virtual reality (VR) have played a key role in improving teaching processes in the surgical area in recent years. These technological advances, increasingly available, allow medical students, residents, and fellows to be immersed in simulated and controlled scenarios where they can acquire surgical skills and abilities necessary in their academic training processes.⁽⁵⁾

Among the advantages of AR and VR are the reduction of the learning curve time, the reduction of possible surgical complications by not exposing actual patients for learning purposes and the use of previously established and validated courses.

Likewise, these teaching models have the potential to be adapted in environments where other surgical teaching methods are not available, such as animal models and cadaveric models.⁽⁵⁾

Among the augmented reality-based trainers, systems have been developed that allow medical students first to approximate the central venous access procedure in newborns. This is the case of a system developed at the Universidad Militar Nueva Granada in Colombia, which has tools for position tracking and orientation of a 3D marker, allowing the user to interact with models

of surgical tools such as the syringe, guide wire, dilation device, and catheter, each of them superimposed as virtual content on the marker.⁽⁶⁾

Augmented reality is a technology that can facilitate the visual positioning of surgeons within minimally invasive surgical intervention spaces, allowing the addition of computer-generated information within the surgeon's field of vision to obtain a more intuitive representation.⁽⁶⁾

Aguilar et al.⁽⁷⁾ in their study, performed a standard test with square markers, also called "fiducial" or "Border Markers," and natural markers or "Natural Feature Marker" using commercial augmented reality and computer vision libraries. For the 3D manipulation and the blending of the natural and augmented view, the integrated development environment (IDE) Unity was used as a base. The next step focused on the use of the OpenCV asset for Unity. Being integrated into Unity, this asset allowed us to compile multiplatform applications with excellent performance in real time. Markerlessse worked with Kudan and later with the Vuforia library, which is integrated with the latest version of Unity. The Vuforia add-on library makes use of the SLAM algorithm.

The development of new simulators is essential. Imperial College London is developing a simulator for inguinal hernia repair using the Lichtenstein technique. Limb and Things UK/USA and Pharmacobiotics Ltd are the manufacturers of surgical training simulators such as suture pads, venipuncture simulators, injections, central venous catheter placement, benign lesion simulators, etc., but research is still needed to validate the models and investigate their benefit.⁽⁸⁾

Given the continuous evolution of medicine towards minimally invasive treatments, informatics is increasingly present (in this case, virtual and augmented reality), as it is a powerful aid to display additional information and help to understand the patient's internal state at all times.⁽⁹⁾

Escamilla-Ortiz et al.⁽⁸⁾ point out that the selection of models for simulation in open surgery is limited, but some simulators are used for open surgery; some examples are the open lobectomy bench model and silicone tubing for anastomosis (BOPT), another model is the Virtual Reality Educational Surgical Tool (VREST)-Virtual Lichtenstein Trainer, which is used for inguinal hernia repair.

Furthermore, Escamilla-Ortiz et al.⁽⁸⁾ eight stress that it is important to reiterate that open surgery skills should be a prerequisite for acquiring skills in laparoscopic surgery and that simulation centers can include open surgery simulators, taking into account costs, monitoring, elaboration of checklists, feedback and so on. Doing so will increase resident competencies and patient safety and result in fewer hours in the operating room.

Virtual and augmented reality-based systems allow the simulation of many scenarios adapted to the conditions needed, even simulating unusual pathologies. It is a fact that surgeons trained through simulators acquire skills faster than those trained classically. Calculating points of interest and anatomical regions allows a compact, efficient, and effective representation to study surgical intervention's progress or real-time evaluation.⁽⁹⁾

Negrillo Cárdenas et al.⁽⁹⁾ concluded in their study that virtual reality is a technology prepared to assist in tasks outside the operating room, either before or after surgery. Augmented reality, on the other hand, has a more excellent projection during surgery in terms of assisting specialists. VR and AR are promising in this field but have not yet been sufficiently explored to replace regular medical procedures.

Solarte Correa et al.⁽¹⁰⁾ ten present in their study a prototype of a neurosurgical navigation system based on the augmented reality (AR) technique supported by the emerging Head-Mounted Displays (HMD) technology to create a scenario that generates an enlarged visual sensation of computer-generated neurological structures from magnetic resonance images (MRI). Mounted Displays (HMD) technology creates a scenario that generates an enlarged visual sensation of computer-generated neurological structures from magnetic resonance images (IMR) that allows access to anatomical structures obtained from accurate medical data, avoiding physical access to the patient. Two interaction tests of the system were performed with ten test subjects, concluding that there is a great interest in using the usable, absorbent technology and that it presents an acceptable level of immersion.

On the other hand, endoscopic and laparoscopic surgery has offered multiple surgical techniques; however, the new era of surgery is image-guided surgery. The convergence of robotic and image-guided surgery should be the focus of scientific efforts. The advent of artificial intelligence will aid in automating and improving work actions within the operating room. Image-guided surgery should be a fundamental part of surgeon training.⁽¹¹⁾

Moya-Salazar et al.⁽¹²⁾ states in their article that AR should be considered more than a technological conquest, optimizing the opportunities for human understanding under an integral context of medical training. These new technologies allow us to disaggregate and understand the human being as an active functional structure by bringing us closer to the integral organic structures that arise in the living essence of man. However, these benefits must be democratized among all medical schools worldwide, as they still seem illusory to many student communities.

Augmented reality in medicine facilitates the preparation and performance of surgical operations, although it is true that, in this sense, it is still a technology to be developed. Among other advantages, it helps to choose the most convenient technique for the operation and serves as a guide during the process, offering relevant information. Augmented reality makes it possible to collect patient data in real time using non-invasive sensors such as magnetic resonance imaging, tomography, CT scanning, etc.

MRI, CT, or ultrasound imaging. Augmented reality technology allows an internal view of the patient without the need for surgery, visualization, and precision tasks in the operating room, such as knowing where to drill into the skull or perform a biopsy. It can also be helpful for medical training.⁽²⁾

AR plays a significant educational role; it is characterized as a cognitive process by which one learns while performing the activity. Although its implementation is somewhat recent in the educational field, the positive effects on learning have been proven. When AR is used appropriately, the motivational effects and improvements in student performance are evident. AR can improve a user's interaction with the natural world and the user's perception of the environment.⁽¹³⁾ The latest trends in general surgery promise more precise, patient-centered care. Robotic surgery, augmented reality, and minimally invasive techniques stand out, opening opportunities for significant improvements.⁽¹⁴⁾

Agustín Leonardo et al.⁽¹³⁾ presented a prototype of a mobile application called CGRA (General Surgery in Augmented Reality), which assists students of General Surgery through the use of Augmented Reality. The application allows the visualization of 3D models of surgical instruments on an absolute scale; in addition, the application provides a description and information associated with the instruments.

It is worth mentioning the medical field that, since the last century, has benefited from them, allowing the introduction of their use in procedures such as diagnosis, assistance, rehabilitation, or treatment. In this field, models of representation of the human body compatible with the new virtual and augmented reality requirements are proposed. In addition, systems focused on the specialist (medical training) and the patient (treatments and diagnostic tools) are proposed.⁽⁹⁾

Moncada Granda et al.⁽¹⁵⁾ highlights in their study the high potential of artificial intelligence in surgical prediction and diagnosis, covering areas such as kidney transplantation, cardiac surgery, and fractures. In addition, AI improves surgical training and efficiency through Augmented and Virtual Reality. AI in medicine promises significant improvements in diagnosis and treatment.

CONCLUSIONS

Currently and in the future, augmented reality is a necessary tool for preclinical training due to its application in different fields of medicine, among which is surgery, orienting this technology to improve the clinical and surgical skills of professionals.

REFERENCES

1. Vidal Ledo M, Lío Alonso B, Santiago Garrido A, Muñoz Hernández A. Realidad aumentada angel. Educ Medi Sup s. f.;31.

2. Carballo Muñoz, L, Fernández Rigondeaux, Y. La Realidad Aumentada en el enfrentamiento a la COVID-19 - Dialnet 2020. https://dialnet.unirioja.es/servlet/articulo?codigo=8590355

3. León Araujo CJ. REALIDAD VIRTUAL Y REALIDAD AUMENTADA EN ODONTOLOGÍA: REVISIÓN DE LA LITERATURA. Univ Cat de Cuen 2021.

4. Larrarte EMA, Alban OAV, Navarro JMS. Una aproximación a la realidad aumentada y sus aplicaciones quirúrgicas. UCP 2018;12. https://doi.org/10.31908/19098367.3811.

5. Barajas-Gamboa JS. Redefiniendo la era de la cirugía digital: el rol de la inteligencia artificial, la realidad aumentada y el aprendizaje automático en el campo quirúrgico | MedUNAB. UNAB 2022;25.

6. Aguilar Larrarte EM, Vivas Albán OA, Sabater Navarro JM. Realidad aumentada con marcadores cuadrados y naturales para navegación quirúrgica. Pistas Educativas 2018;39.

7. Aguilar E, Sabater J, Vivas A. Aplicación móvil de realidad aumentada para operaciones de laparoscopia. UGR 2018;5.

8. Escamilla-Ortiz AC, Serrano Pérez J. La simulación en cirugía abierta. Ciru Gener 2022;44.

9. Negrillo Cárdenas J, Feito Higueruela FR. Aplicaciones de la Realidad Virtual y Realidad Aumentada en Medicina: Diagnóstico, tratamiento y rehabilitación. Univ de Jaen 2020.

10. Solarte Correa PL, Sabater Navarro JM, Aguilar Larrarte EM, Vivas Albán OA, Vicente Samper JM. Uso de realidad aumentada como apoyo a un sistema de navegación en neurocirugía. Univ de Extre 2018.

11. Giménez M, Garcia Vazquez A, Yanes L. Cirugía guiada por imágenes. Researchgate 2020.

12. Moya-Salazar J, Diaz A, Paredes J, Contreras-Pulache H, Moya-Salazar J, Diaz A, et al. Algunas consideraciones sobre la Realidad Aumentada en la enseñanza de la medicina. Educ Med Sup 2021; 35.

13. Agustín Leonardo C, Dapoto, S, Thomas, P. Realidad aumentada aplicada al estudio de instrumental quirúrgico para cirugía general veterinaria. UNLP 2023.

14. Crespo Zamora MV, Zapata Toapanta AE, Pilco Rivera ET, Herrera Mejia AJ, Lagua Curipallo LE. Explorando las Últimas Tendencias en Cirugía General: Innovación, Efectividad y Resultados del Paciente. Tesla Revista Científica. Tesla Rev Cient 2023; 3.

15. Moncada Granda KE, Correa Martínez FG. Aplicaciones de la inteligencia artificial en cirugía | Salud ConCiencia. Salud ConCiencia 2023; 2.

FINANCING

The authors did not receive funding for the development of this research.

CONFLICT OF INTEREST

The authors declare that there is no conflict of interest.

AUTHORSHIP CONTRIBUTION

Conceptualization: Jose Ignacio Robaina Castillo.

Research: Jose Ignacio Robaina Castillo.

Writing - original draft: Jose Ignacio Robaina Castillo.

Rediting - revision and editing: Jose Ignacio Robaina Castillo.