Enhancing a laparoscopy training system with augmented reality visualization

Hao Jiang, Siqing Xu, Andrei State, Fan Feng, Henry Fuchs, Minsik Hong, Jerzy Rozenblit

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


We report work in progress towards a system for laparoscopy training that is enhanced with an augmented reality display. The system can be used to perform peg-transfer training tasks using a Microsoft Hololens AR device, with full 6 degrees of freedom (DoF) shared presence with the pegboard workspace. This mode is in contrast to the conventional visualization, whose views are solely through a 2D or stereo laparoscope. In order to achieve this enhanced visualization, the system extracts the pose of the triangular prism being manipulated, and of the laparoscopic instruments with which the user manipulates the prism. These two objects are added to a CAD model of a (fixed) pegboard, and all the objects are visualized within the AR display. In the near future we expect to conduct user studies measuring accuracy and time-to-completion of this peg-transfer task using these new enhancements and compare it to conventional visualization techniques.

Original languageEnglish (US)
Title of host publication2019 Spring Simulation Conference, SpringSim 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781510883888
StatePublished - Apr 2019
Event2019 Spring Simulation Conference, SpringSim 2019 - Tucson, United States
Duration: Apr 29 2019May 2 2019

Publication series

Name2019 Spring Simulation Conference, SpringSim 2019


Conference2019 Spring Simulation Conference, SpringSim 2019
Country/TerritoryUnited States


  • Augmented reality
  • Laparoscopic
  • Visualization

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Control and Optimization
  • Modeling and Simulation


Dive into the research topics of 'Enhancing a laparoscopy training system with augmented reality visualization'. Together they form a unique fingerprint.

Cite this