New methods for medical augmented reality [Elektronische Ressource] / Tobias Sielhorst

Sielhorst Tobias

Sujets

• Gesundheit

Informations

• Publié par: technische_universitat_munchen
• Publié le: 01 janvier 2008
• Nombre de lectures: 16
• Langue: English
• Poids de l'ouvrage: 17 Mo

Extrait

Computer Aided Medical Procedures | Institut für Informatik
New Methods for
Medical Auggymented Reality
Dissertation
Tobias SielhorstTechnische Universitat Munchen, Fakultat fur Informatik
Lehrstuhl fur Computeranwendungen in der Medizin, Univ.-Prof. N. Navab, Ph.D.
New Methods for Medical Augmented Reality
Tobias Sielhorst
Vollstandiger Abdruck der von der Fakultat fur Informatik der Technischen Universitat
Munchen zur Erlangung des akademischen Grades eines
Doktors der Naturwissenschaften (Dr. rer. nat.)
genehmigten Dissertation.
Vorsitzende: Univ.-Prof. G. J. Klinker, PhD
Prufer der Dissertation:
1. Univ.-Prof. Dr. N. Navab, Ph.D.
2. Prof. H. Fuchs, Ph.D.
University of North Carolina, Chapel Hill, USA
Die Dissertation wurde am 25.10.2007 bei der Technischen Universitat Munchen
eingereicht und durch die Fakultat fur Informatik am 25.02.2008 angenommen.Abstract
The increasing amount of imaging data in the operating room o ers new possibilities
for surgeons. Preoperative data like Computed Tomography and Magnetic Resonance
Imaging allow for detailed anatomical information. Recently, intraoperative imaging like
X-ray C-arms and ultrasound have become increasingly accessible in the OR and provide
real-time anatomical images. Also, functional imaging such as PET and fMRI is accessible
in a growing number of institutions. Corresponding intraoperative functional probes allow
more e cient procedures to be introduced in the near future.
The growing amount of imaging data increases the diculty of retrieving the desired
piece of information. Thus, e cient data representation becomes increasingly important
for physicians. Visualization in computer assisted surgical solutions has not coped with
the recent developments and does still not provide solutions which allow surgeons to take
full advantage of the existing heterogeneous imaging data.
Augmented Reality (AR) is a technology that has potential to improve physicians’
performance by bringing the multitude of medical data, surgical actions, and patient into
the same space.
After more than a decade of research in medical Augmented Reality, the technology is
suciently developed to create prototype systems for showrooms. However, it has still not
found its way into operating rooms.
This thesis addresses current issues of medical Augmented Reality in de ning the right
place, right time, and the right way of data representation. The rst contribution is in
providing an exhaustive state-of-the-art report on medical Augmented Reality. It then
features a detailed overview of components that are necessary for a medical AR system
and introduces a new method for temporal registration in AR. It provides details on the
integration of these components and describes the necessary software framework that has
been developed. The thesis presents new methods for assessment of a medical AR system
addressing latency measurement and dynamic error prediction. The thesis concludes with
validation which was done in close partnership with many surgeons.iiCONTENTS
1 Introduction 1
1.1 Overview of medical AR systems and technologies . . . . . . . . . . . . . . 2
1.1.1 Head-mounted display (HMD) based . . . . . . . . . . . . . . . . . 3
1.1.2 Augmented optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.3 AR windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.4 Augmented monitors . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.1.5 Augmented endoscopes . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.1.5.1 Calibration and undistortion of wide angle optics . . . . . 9
1.1.5.2 Tracking of exible endoscopes . . . . . . . . . . . . . . . 10
1.1.5.3 Endoscopy related visualization issues . . . . . . . . . . . 10
1.1.6 Augmented medical imaging devices . . . . . . . . . . . . . . . . . 11
1.1.7 Projections on the patient . . . . . . . . . . . . . . . . . . . . . . . 13
1.2 Potential bene ts of AR visualization . . . . . . . . . . . . . . . . . . . . . 14
1.2.1 Extra value from image fusion . . . . . . . . . . . . . . . . . . . . . 14
1.2.2 Implicit three-dimensional interaction . . . . . . . . . . . . . . . . . 14
1.2.3 Three-dimensional visualization . . . . . . . . . . . . . . . . . . . . 15
1.2.4 Improved hand-eye coordination . . . . . . . . . . . . . . . . . . . . 15
1.3 The NARVIS Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.3.1 Targeted intervention: Minimal invasive spine xation . . . . . . . . 18
1.3.2 Analysis of the surgery . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3.2.1 Port placement . . . . . . . . . . . . . . . . . . . . . . . . 20
1.3.2.2 Laparoscopic access to target area . . . . . . . . . . . . . 21
1.3.2.3 Implant screw placement . . . . . . . . . . . . . . . . . . . 22
1.4 Contribution of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.5 Document overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2 Components 25
2.1 Spatial registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.2 Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1.3 Patient registration . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2 Temporal registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
iiiCONTENTS
2.2.1 Movement comparison . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.1.1 Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2.1.2 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.1.3 Comparison of trajectories . . . . . . . . . . . . . . . . . . 33
2.2.1.4 Comparison of values . . . . . . . . . . . . . . . . . . . . . 34
2.2.1.5 How to match points . . . . . . . . . . . . . . . . . . . . . 35
2.2.1.5.1 Longest Common Subsequence (LCSS) . . . . . . 35
2.2.1.5.2 Dynamic Time Warping (DTW) . . . . . . . . . 38
2.2.1.6 Results of oine synchronization . . . . . . . . . . . . . . 41
2.2.1.7 Preliminary work on online synchronization . . . . . . . . 41
2.2.1.8 Discussion/Conclusion . . . . . . . . . . . . . . . . . . . . 44
2.2.2 Workow recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.2.2.1 Introduction and Related Work . . . . . . . . . . . . . . . 45
2.2.2.2 Description of surgical workow . . . . . . . . . . . . . . . 46
2.2.2.3 Algorithms and methods . . . . . . . . . . . . . . . . . . . 47
2.2.2.3.1 Principle of workow recovery . . . . . . . . . . . 47
2.2.2.3.2 Classical DTW . . . . . . . . . . . . . . . . . . . 48
2.2.2.3.3 Creating an Average Surgery . . . . . . . . . . . 48
2.2.2.3.4 Weighting of Signals From The Curves . . . . . . 49
2.2.2.4 Experiments and Results . . . . . . . . . . . . . . . . . . . 50
2.2.2.4.1 Workow Retrieval . . . . . . . . . . . . . . . . . 50
2.2.2.4.2 Online temporal registration of workow . . . . . 51
2.2.2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.3 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.1 Depth perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.1.1 Misperception . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3.1.2 Adaption . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3.1.3 Motion sickness . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3.2 Three-dimensional data representation . . . . . . . . . . . . . . . . 55
2.3.2.1 Slice rendering . . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.2.2 Surface rendering . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.2.3 Volume Rendering . . . . . . . . . . . . . . . . . . . . . . 56
2.3.3 Selected medical visualization topics . . . . . . . . . . . . . . . . . 57
2.3.3.1 In-situ visualization of endoscopic images . . . . . . . . . 57
2.3.3.1.1 Depth reconstruction . . . . . . . . . . . . . . . . 58
2.3.3.1.2 Automatic horizon recovery . . . . . . . . . . . . 58
2.3.3.1.3 Fallback visualization . . . . . . . . . . . . . . . 59
2.3.3.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . 59
2.3.3.1.5 Discussion and Conclusion . . . . . . . . . . . . . 60
2.3.3.2 Medical navigation . . . . . . . . . . . . . . . . . . . . . . 61
2.4 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
iv