Fusion of medical video images and tomographic volumes [Elektronische Ressource] = Fusion medizinischer Videobilder mit tomographischen Volumendaten / vorgelegt von Michael Scheuering

Sujets

• Gesundheit

Informations

• Publié par: friedrich-alexander-universitat_erlangen-nurnberg
• Publié le: 01 janvier 2004
• Nombre de lectures: 8
• Langue: English
• Poids de l'ouvrage: 9 Mo

Extrait

Fusion of Medical Video Images
And Tomographic Volumes
Fusion medizinischer Videobilder
mit tomographischen Volumendaten
Der Technischen Fakultat¨ der
Universitat¨ Erlangen Nur¨ nberg
zur Erlangung des Grades
DOKTOR INGENIEUR
vorgelegt von
Michael Scheuering
Erlangen – 2003Als Dissertation genehmigt von
der Technischen Fakultat¨ der
Universitat¨ Erlangen Nurnberg¨
Tag der Einreichung: 12. Mai 2003
Tag der Promotion: 24. 07. 2003
Dekan: Prof. Dr. A. Winnacker
Berichterstatter: Prof. Dr. G. Greiner
Prof. Dr. B. PreimAbstract
Minimally invasive surgery has advanced rapidly in the last years because of the
accelerated convalescence of the patient. However, such interventions demand a lot
of experience due to limited access to the field of operation. In particular, the trocar
placement and the orientation within the patient’s body are hampered. Through
the invention of navigation hardware, the tracking of the surgical tools and cam
eras became possible, which revolutionized intra operative image guided surgery
generally. Nowadays, there are a variety of applications that allow the navigation
and guidance of tools with high accuracy, ranging from neurosurgical interventions
to osseous applications. In the case of minimally invasive liver surgery, one pos
sibility for intervention assistance is the fusion of laparoscopic video images and
tomographic volumes at operation time in order to present orientational aids based
on navigation hardware. In this doctoral thesis, different fundamental techniques
are presented in order to merge video images and tomographic volumes with a focus
on high interactivity. Therefore, new techniques and capabilities of modern graphics
adapters are exploited.
One application for fusion are augmented reality systems (ARS) which directly
project pre operative information onto the surgeon’s view, according to the pose of
the surgical camera. In this context, real time hardware accelerated direct volume
rendering based on fragment shader techniques is applied for augmentation of the
laparoscopic video images. Liver parenchyma, hepatic vessels and vascular territo
ries are overlaid for oncologic resection. Furthermore, fiducial markers are used for
rigid registration. The system has been evaluated within real interventions.
Alternatively, laparoscopic video images can be applied in order to perform
real time scene exploration of the visceral space. A basic algorithm that utilizes
hardware acceleration for advanced view dependent object texturing is presented
herein.
Furthermore, this thesis introduces a very fundamental and new technique for
fast intensity based 2D/3D non rigid registration of multiple view video images and
deformable volume renderings using mutual information as a voxel similarity met
ric. This algorithm requires a very fast and flexible volume rendering approach
that is based on interactive volume deformation. Therefore, two possibilities are
presented which are based on advanced hardware acceleration techniques includ
ing pixel shaders and dependent texture reads as a supplement to 3D texture ap
proaches.
iiiWhile image guided intra operative assistance is essential, pre operative plan
ning tools are necessary in order to become familiar with the individual pa
tient’s anatomy. In the case of osseous applications, this work presents a tool for
semi automatic repositioning of bone fracture segments, based on C arm modality,
whereby the physician roughly positions the fragments. An exact alignment, how
ever, is achieved through an optimization procedure.
ivRevision 1.0
c�2003, Copyright Michael Scheuering
All Rights Reserved
Alle Rechte vorbehalten
vviContents
Abstract iii
Table of Contents xi
List of Figures xii
List of Tables xiii
Listings xv
Acknowledgements xvii
I Introduction 1
1 Motivation 3
1.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Outline of the Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Minimally Invasive Procedures and Navigation 9
2.1 Minimally Invasive Liver Interventions . . . . . . . . . . . . . . . . . . 9
2.1.1 Historical Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.2 Clinical Setup in the Operating Room . . . . . . . . . . . . . . . 10
2.2 Navigation in Image Guided Surgery. . . . . . . . . . . . . . . . . . . . 13
2.2.1 Navigation by Stereo Cameras . . . . . . . . . . . . . . . . . . . 14
2.2.2 Electro mechanic and Robotic Navigation . . . . . . . . . . . . . 15
2.2.3 Optical Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.4 Electro magnetic Navigation . . . . . . . . . . . . . . . . . . . . 17
2.2.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
viiII Fusion of Video and Tomographic Images 21
3 Volume Rendering 23
3.1 Direct Volume Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Basics of Graphics Hardware . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Standard 3D Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 2D Multi Texturing and Pixel Shaders. . . . . . . . . . . . . . . . . . . 28
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Augmented Reality - Fusion of CT and VR 31
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3.2 Real time Endoscopic Distortion Correction . . . . . . . . . . . . 36
4.4 Sensor Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.5 Augmented Reality and Image Overlay . . . . . . . . . . . . . . . . . . 40
4.5.1 Rigid Registration Procedure . . . . . . . . . . . . . . . . . . . . 40
4.5.2 System Composition . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.6.1 Registration Accuracy . . . . . . . . . . . . . . . . . . . . . . . . 42
4.6.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.7 Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5 Augmented Virtuality - 3D Scene Exploration 47
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3 Algorithm Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.3.1 Grid Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.3.2 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.3.3 Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.3.4 Texturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.6 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.6.1 Post operative Exploration. . . . . . . . . . . . . . . . . . . . . . 57
5.6.2 Augmentation by Direct Volume Rendering . . . . . . . . . . . . 58
5.7 Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6 Interactive Volume Deformation 63
6.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.2 Hexahedra Deformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.2.1 Piecewise Linear Patches . . . . . . . . . . . . . . . . . . . . . . 65
viii6.2.2 Vertex Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.2.3 Algorithm Implementation . . . . . . . . . . . . . . . . . . . . . . 68
6.2.4 Local Illumination and Fragment Shaders . . . . . . . . . . . . 69
6.2.5 Hierarchy Reduction . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.3 Volume Deformation with 3D Dependent Textures . . . . . . . . . . . . 73
6.3.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.4 Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 78
7 Non rigid 2D/3D Registration 81
7.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
7.2 Calibration and Navigation . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.3 Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.4 Higher Order Deformation Model . . . . . . . . . . . . . . . . . . . . . . 86
7.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
7.4.2 Tensor Product Bezier´ Patches . . . . . . . . . . . . . . . . . . . 88
7.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.6 Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 92
III Me