Image-based incremental reconstruction, rendering and augmented visualization of surfaces for endoscopic surgery [Elektronische Ressource] = Bildbasierte inkrementelle Rekonstruktion, Darstellung und erweiterte Visualisierung von Oberflächen für die endoskopische Chirurgie / vorgelegt von Marco Winter

Marco Winter

Sujets

• Informatik

Informations

• Publié par: friedrich-alexander-universitat_erlangen-nurnberg
• Publié le: 01 janvier 2010
• Nombre de lectures: 27
• Poids de l'ouvrage: 16 Mo

Extrait

Image-basedIncrementalReconstruction,Renderingand
AugmentedVisualizationofSurfaces
forEndoscopicSurgery
BildbasierteinkrementelleRekonstruktion,Darstellung
underweiterteVisualisierungvonOberflächen
fürdieendoskopischeChirurgie
DerTechnischenFakultätder
UniversitätErlangen–Nürnberg
zurErlangungdesGrades
DOKTOR–INGENIEUR
vorgelegtvon
Dipl.–Inf. MarcoWinter
Erlangen—2010AlsDissertationgenehmigtvon
derTechnischenFakultätder
UniversitätErlangen–Nürnberg
TagderEinreichung: 04.09.2009
TagderPromotion: 22.12.2009
Dekan: Prof. Dr.-Ing. ReinhardGerman
Berichterstatter: Prof. Dr. GüntherGreiner
Prof. Dr.-Ing. AndreasKolbRevision1.0
c?2010MarcoWinter
[e-mail]
ATypesetinpdfLT XE
Version3.141592-1.40.3(Web2C7.5.6)i
Abstract
Minimallyinvasivesurgeryrepresentsthestandardinterventiontechniqueforthetreat-
mentofvariousabdominaldiseases. Comparedtoopensurgery,itprovidesmanyben-
efits for the patient, like diminishment of e.g. pain, operative trauma or scarring. To
make such treatments more comfortable for the patient, a novel intervention method
called“NaturalOrificeTranslumenalEndoscopicSurgery”(NOTES)iscurrentlyinves-
tigated. Here, visible scarring is avoided by introducing the needed endoscopes and
instruments through anatomical openings and performing all needed incisions at the
interiorofthebody.
Whilebothinterventionmethodsarepreferablebecauseoffasterrecoveryofthepa-
tient, their execution exhibits a great challenge. In contrast to open surgery, there exist
several drawbacks for the surgeon, which include a missing spatial view at the opera-
tionfield, unusualhand-eyecoordinationandrestrictedmovementofendoscopesand
instruments. To circumvent these problems, there is an ongoing research for develop-
mentofcomputeralgorithmsandhardwaresystemswhichsupportthesurgeonduring
theseinterventions.
Inthiscontext,onestepistherecentexperimentalintroductionofthenoveltechnol-
ogyof“Time-of-Flight”(ToF)camerasintoendoscopicsurgery. Thistechnologyenables
the acquisition of depth images of the operation field at interactive frame rates. This
additional information source provides valuable data for a wide range of medical ap-
plications, provided that computer algorithms are applied that utilize this information
efficiently.
Thisthesiscontributestothedepictedmedicalscenarioandintroducesseveralnew
algorithms for the processing of color and depth image sequences of operation fields.
Based on this input, the algorithms deal with the tasks of surface reconstruction, sur-
face rendering and augmented visualization. All proposed algorithms are capable of
processingtheincomingdataincrementally,thusallowingtheusertointeractivelycon-
troltheprogressofacquisitionandprocessing.ii Abstractiii
Contents
Abstract i
Contents iii
ListofFigures vii
ListofTables xi
ListofListings xiii
Acknowledgements xv
I Introduction 1
1 Background&Motivation 3
1.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 EndoscopicSurgeryoftheAbdominalRegion 9
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 AMedicalApplicationScenario . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 MinimallyInvasiveSurgery . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 NaturalOrificeTranslumenalEndoscopicSurgery . . . . . . . . . . 11
2.3 AssistingHardwareSystems . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.1 RobotSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.2 TrackingSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Time-of-FlightCameraTechnology 15
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 MeasuringDistances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17iv CONTENTS
3.3 AreasofResearchandApplication . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.1 AlgorithmsforProcessingofTime-of-FlightData . . . . . . . . . . 20
3.3.2 Time-of-FlightCamerasinMedicalApplications . . . . . . . . . . . 21
4 ImagePre-processing 25
4.1 ProvidedInputData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2 CameraCalibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 PoseEstimationfromTrackingData . . . . . . . . . . . . . . . . . . . . . . 28
4.4 ReconstructionofDepthMeshes . . . . . . . . . . . . . . . . . . . . . . . . 30
II IncrementalSurfaceReconstruction 33
5 Introduction 35
5.1 ExistingTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2 SelectionCriterions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3 Pre-ProcessingofInputData. . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6 Sphere-basedReconstruction 41
6.1 TheOriginalApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.1.1 Pre-ProcessingofthePointCloud . . . . . . . . . . . . . . . . . . . 41
6.1.2 AdaptiveSphericalCoverofthePointCloud . . . . . . . . . . . . . 42
6.1.3 MeshGenerationandPost-Processing . . . . . . . . . . . . . . . . . 44
6.2 IncrementalSphere-basedReconstruction . . . . . . . . . . . . . . . . . . . 46
6.2.1 ClassificationofPointData . . . . . . . . . . . . . . . . . . . . . . . 47
6.2.2 Pre-ProcessingofthePointData . . . . . . . . . . . . . . . . . . . . 48
6.2.3 SphericalCoverofthePointData . . . . . . . . . . . . . . . . . . . 48
6.2.4 MeshUpdateandPost-Processing . . . . . . . . . . . . . . . . . . . 49
6.2.5 AdaptiontoMulticoreCPUs . . . . . . . . . . . . . . . . . . . . . . 52
7 Ridge-basedReconstruction 55
7.1 TheOriginalApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.1.1 FundamentalsforRidgeDetection . . . . . . . . . . . . . . . . . . . 56
7.1.2 EfficientRidgeExtraction . . . . . . . . . . . . . . . . . . . . . . . . 57
7.2 IncrementalRidge-basedReconstruction. . . . . . . . . . . . . . . . . . . . 59
7.2.1 UpdateofGridStructures . . . . . . . . . . . . . . . . . . . . . . . . 59
7.2.2 ConditionsforMeshExtraction . . . . . . . . . . . . . . . . . . . . . 61
7.2.3 ExtractionandAddingofSubmeshes . . . . . . . . . . . . . . . . . 61
7.2.4 HoleFilling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.2.5 AdaptiontoMulticoreCPUs . . . . . . . . . . . . . . . . . . . . . . 63CONTENTS v
8 MPUImplicits 65
8.1 TheOriginalApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
8.1.1 TheMainAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
8.1.2 DeterminationandComputationofQuadrics. . . . . . . . . . . . . 68
8.1.3 PropertiesoftheAlgorithm . . . . . . . . . . . . . . . . . . . . . . . 69
8.2 IncrementalMPUReconstruction . . . . . . . . . . . . . . . . . . . . . . . . 70
8.2.1 2.5-DimensionalMPUforHeightfields . . . . . . . . . . . . . . . . 71
8.2.2 TwofoldHierarchicalReconstruction . . . . . . . . . . . . . . . . . 75
8.2.3 IncrementalSurfaceExtraction . . . . . . . . . . . . . . . . . . . . . 76
8.2.4 AdaptiontoMulticoreCPUs . . . . . . . . . . . . . . . . . . . . . . 80
9 MeshPost-Processing 81
9.1 MeshSimplification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
9.2 EdgeRelaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
9.3 TriangleFixation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
10 Results&Discussion 87
10.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
10.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
III Image-basedSurfaceRenderingandVisualization 99
11 Introduction 101
11.1 LightFieldDefinitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
11.2 LightFieldApproaches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
12 OptimizedULGRenderingforEndoscopicImageData 105
12.1 TheUnstructuredLumigraph . . . . . . . . . . . . . . . . . . . . . . . . . . 105
12.1.1 TheOriginalAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . 106
12.1.2 TheAvailableImplementation . . . . . . . . . . . . . . . . . . . . . 106
12.2 OptimizedCameraRankingAlgorithms . . . . . . . . . . . . . . . . . . . . 109
12.2.1 LocalSelection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
12.2.2 CameraPreselection . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
12.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
13 GPU-supportedIncrementalReconstructionofSurfaceLightFields 123
13.1 TheSurfaceLightField . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
13.1.1 EfficientOfflineReconstructionandRendering . . . . . . . . . . . . 124
13.1.2 OnlineReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . 128
13.2 GPU-supportedIncrementalImplementation . . . . . . . . . . . . . . . . . 132
13.2.1 VisibilityAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
13.2.2 Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
13.2.3 LightFieldExtension . . . . . . . . . . . . . . . . . . . . . . . . . . . 135vi CONTENTS
13.2.4 ConstructionoftheRend