Spatiotemporal analysis of range imagery [Elektronische Ressource] / put forward by Martin Otmar Schmidt

De
Dissertationsubmitted to theCombined Faculties for theNatural Sciences and for Mathematicsof the Ruperto-Carola University of Heidelberg, Germanyfor the degree ofDoctor of Natural Sciencesput forward byDiplom-Physiker Martin Otmar Schmidtborn in Nur¨ nbergOral examination: 5. November 2008Spatiotemporal Analysis ofRange ImageryReferees: Prof. Dr. Bernd Jahne¨Prof. Dr. Ulrich PlattZusammenfassungDie vorliegende Arbeit befasst sich mit der Fragestellung, wie aus einer Tiefenbildsequenzdas zugehorige dreidimensionale Bewegungsfeld bestimmt werden kann. Wir untersuchen dasSignal von Tiefenkameras, die auf dem Laufzeitverfahren basieren und sich eines neuartigenoptoelektronischen Bauelements bedienen, dem Photomischdetektor (PMD). Dieser liefertneben der Tiefe auch Informationen zur mittleren Strahlungsleistung und deren Modulati-onsamplitude. Wir erortern wie dieser erweiterte Informationsgehalt genutzt werden kann.Die Rekonstruktion eines Bewegungsfeldes aus einer Bildsequenz ist ein schlecht gestelltesinverses Problem und kann allgemeingultig nicht gelost werden. Uberdies enthalt das raum- zeitliche Signal einer PMD-Kamera diverse, teilweise sehr spezi sche, systematische und sta-tistische Fehler von explizit raumlicher wie zeitlicher Abhangigkeit (z.B. Bewegungsartefakte). Wir analysieren die unterschiedlichen Fehler und entwickeln ein Verfahren zur Korrektur sys-tematischer Tiefensignalfehler.
Publié le : mardi 1 janvier 2008
Lecture(s) : 90
Source : ARCHIV.UB.UNI-HEIDELBERG.DE/VOLLTEXTSERVER/VOLLTEXTE/2008/8879/PDF/SPATIOTEMPORALANALYSISOFRANGEIMAGERY_MARTINSCHMIDT2008_HEIDOK_UNPROT.PDF
Nombre de pages : 166
Voir plus Voir moins

Dissertation
submitted to the
Combined Faculties for the
Natural Sciences and for Mathematics
of the Ruperto-Carola University of Heidelberg, Germany
for the degree of
Doctor of Natural Sciences
put forward by
Diplom-Physiker Martin Otmar Schmidt
born in Nur¨ nberg
Oral examination: 5. November 2008Spatiotemporal Analysis of
Range Imagery
Referees: Prof. Dr. Bernd Jahne¨
Prof. Dr. Ulrich PlattZusammenfassung
Die vorliegende Arbeit befasst sich mit der Fragestellung, wie aus einer Tiefenbildsequenz
das zugehorige dreidimensionale Bewegungsfeld bestimmt werden kann. Wir untersuchen das
Signal von Tiefenkameras, die auf dem Laufzeitverfahren basieren und sich eines neuartigen
optoelektronischen Bauelements bedienen, dem Photomischdetektor (PMD). Dieser liefert
neben der Tiefe auch Informationen zur mittleren Strahlungsleistung und deren Modulati-
onsamplitude. Wir erortern wie dieser erweiterte Informationsgehalt genutzt werden kann.
Die Rekonstruktion eines Bewegungsfeldes aus einer Bildsequenz ist ein schlecht gestelltes
inverses Problem und kann allgemeingultig nicht gelost werden. Uberdies enthalt das raum-
zeitliche Signal einer PMD-Kamera diverse, teilweise sehr spezi sche, systematische und sta-
tistische Fehler von explizit raumlicher wie zeitlicher Abhangigkeit (z.B. Bewegungsartefakte).
Wir analysieren die unterschiedlichen Fehler und entwickeln ein Verfahren zur Korrektur sys-
tematischer Tiefensignalfehler. Mit einem neuartigen Two-State-Channel-Smoothing verbes-
sern wir von Rauschen und Ausrei ern verf alschte Tiefenkarten. Wir erweitern das Struktur-
tensorverfahren, um damit erstmals den erweiterten Informationsgehalt der PMD-Kameras
zur Verbesserung der Bewegungsschatzung zu nutzen und Aussagen zur Gute der Schatzung
zu ermoglichen. Bei den entwickelten Algorithmen wurde darauf geachtet, dass deren Berech-
nungskomplexitat eine Verwendung in eingebetteten Systemen nicht ausschlie t. Die Algo-
rithmen werden anhand von synthetischen und realen Einzelbildern wie auch Bildsequenzen
ub erpruft.
Abstract
The present thesis handles the topic of how to determine the three dimensional motion eld
from a corresponding sequence of range images. We investigate signals given by range cameras
that are based on the time-of- ight principle for which they employ the novel optoelectronic
photonic-mixer-device (PMD). Its signal comprises information about the range, the mean
radiant ux and its modulation amplitude. We discuss how to take advantage of this wealth
of information.
The estimation of a motion eld from image sequences is an ill-posed inverse problem which
can not be solved in general. Moreover, the spatiotemporal signal of a PMD-camera is
corrupted by several kind of, partially rather speci c, errors of systematic and statistical
nature depending explicitly on time and space (e.g. motion-artifacts).
We analyze those errors and develop a method to correct for systematic errors in the range
signal. By means of a novel two-state-channel-smoothing we improve range images corrupted
by noise and outliers. We use and extend the structure tensor approach to come for the rst
time to an improved motion estimate that exploits the PMD-signal and provides an inherent
measure for its con dence. The presented algorithms were developed under the premise to be
of a computational complexity that not forbids their application within an embedded system.
They are tested on synthetic and real images and image sequences.Acknowledgments
I gratefully acknowledge the support of many people who contributed in various ways
to the completion of this thesis.
First of all I would like to thank Prof. Dr. Bernd J ahne for giving me the opportunity
to work on various interesting topics of computer vision and for supervising my thesis.
I am grateful for his kind support in both scienti c and organizational issues. I thank
Prof. Dr. Ulrich Platt for agreeing to act as the second referee.
Thanks go to the sta of the IWR and HCI that do an excellent job in keeping
things running, especially to Barbara Werner and Karin Kubessa-Nasri for making
bureaucracy less painful and Dr. Hermann Lauer, Markus Riedinger and Dr. Ole
Hansen letting the data streams ow right were they should.
I am grateful to Pavel Pavlov, the most suave person I know, for giving work at the
o ce a congenial feel and being a inexhaustible source of mathematical knowledge.
I would like to thank Dr. Michael Klar for giving me an introduction to camera
calibration and support in various related algorithmic issues. A big thank-you goes
to PD Dr. Ullrich K othe, who always took the time to answer my questions on various
image processing topics. Thanks to PD Dr. Christoph Garbe for giving suggestion
and tips on various topics.
For proof-reading and comments on the thesis I am deeply grateful to Dr. Achim
Falkenroth, Roland Rocholz, Claudia Kondermann, Zhuang Lin, Andreas Schmidt
and Marion Zuber.
I enjoyed working at the lab, which I blame mostly the Windis for and in partic-
ular Dr. Kai Degreif for introducing me to the small wind-wave- ume, Dr. Achim
Falkenroth, Roland Rocholz for the various discussion on water-wave-measurements,
Dr. Uwe Schimpf, Alexandra Herzog o ering always some tea, Kerstin Richter,
Florian Huhn, Rene Winter, Ste en Haschler, and last but not least Dr. Gun ther
Balschbach for giving excellent administrative support - thank you all.
With respect to the research done for the PMD-cameras I would like to thank Holger
Rapp for all his work with the experimental setup, Mario Frank giving me access to
his range measurements, Matthias Plaue for discussions about the PMD’s working
viiprinciple, Dr. Markus Jehle for experimental and theoretical support, Michael Erz
for the demodulation measurements and Dr. Hagen Spies for advice on range ow
algorithms.
Many thanks go to Dr. Bjorn Menze and Dr. Michael Kelm (telling me what the prior
does in the monastery and why he ROCks under the trees of a random forest), Dr. Li-
nus G orlitz, Daniel Kondermann (helping Charon over the Styx), Dr. Ralf Kusters,
Christoph Sommer, Dr. Nikolaos Gianniotis, Dr. Marc Kirchner, Prof. Dr. Fred
A. Hamprecht, Bj orn Andres, Bernhard Renard, Michael Hanselmann, Frederik
Kaster, Sebastian Boppel, Bjoern Voss, J org Greis, Stephan Kassemeyer, Lars Feist-
ner and Natalie Muller.
I would like to thank all the people of the IWR, HCI and IUP that gave me a cheerful
time in Heidelberg, particularly the members of DIP, MIP and IPA group.
Along my time at the IWR I worked together with numerous external collaborators
on various projects whom I would like to thank as well.
Thanks to all members of the LOCOMOTOR team for interesting discussions and
friendly collaboration; especially Dr. Ingo Stuke for giving me support with his multi-
ple motion algorithms, Dr. Hanno Scharr for illuminating talks and Dr. Kai Krajsek
for a crash course in Kriging.
I enjoyed working together with the members of the Bosch corporate research team
in Hildesheim (CR/AEM5), in particular Henning Voelz; very likely this was the rst
and last time in my life, that I can say my job is to drive a BMW through sun, rain,
and snow around Heidelberg.
I appreciate the support of PD Dr. Michael Felsberg by giving me access to his
channel smoothing algorithm.
I would like to thank all collaborators within the Smartvision project, in particular
Hermann Hoepken for the fruitful and pleasant cooperation on the demonstrator.
Working within the LYNKEUS project was a pleasant experience. Thanks to all
collaborators and in particular to Sandra Stecher for her friendly and straight coop-
eration, Prof. Dr. Andreas Kolb and Maik Keller, giving me support for the TOF-
Simulator and trying to nd solutions for my application speci c problems, and
Stefan Fuchs for the egomotion sequences. I gratefully acknowledged the nan-
cial support of the BMBF within the project LYNKEUS (promotional reference:
16SV2296).
Last but not least, I would like to thank my parents, my brothers and my friends (in
particular the Kumperla) for their words of encouragement and emotional support.Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
I Theory 7
2 Range Data and Time-of-Flight Measuring Principle 9
2.1 Optical Range Measurement Techniques . . . . . . . . . . . . . . . . . 9
2.1.1 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.2 Time-of-Flight Based Methods . . . . . . . . . . . . . . . . . . 11
2.1.3 Interferometry . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 The Photonic Mixer Device . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 Demodulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.1.1 Sinusoidal Modulation . . . . . . . . . . . . . . . . . . 17
2.2.1.2 Rectangular Modulation . . . . . . . . . . . . . . . . 19
2.2.1.3 Demodulation Contrast . . . . . . . . . . . . . . . . . 20
2.2.2 Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.2.1 Systematic Errors . . . . . . . . . . . . . . . . . . . . 22
Periodic Phase Error . . . . . . . . . . . . . . . . . . . . 23
Fourier Approximation . . . . . . . . . . . . . . . . . . . 23
Constant Phase Error per Pixel . . . . . . . . . . . . . . 25
Overexposure and Saturation . . . . . . . . . . . . . . . 27
Exposure Time / Amplitude Dependent Phase Deviation 28
2.2.2.2 Random Errors . . . . . . . . . . . . . . . . . . . . . . 30
3 Image Processing and Filters 33
3.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1.1 Discretization and Sampling . . . . . . . . . . . . . . . . . . . . 33
Derivatives and Gradient . . . . . . . . . . . . . . . . . 33
ix3.1.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.3 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.1.4 Convolution, Point Spread Function and Transfer Function . . 39
Filter Design and Optimization . . . . . . . . . . . . . . 41
3.1.5 Normalized Averaging . . . . . . . . . . . . . . . . . . . . . . . 42
Band Enlarging Operators . . . . . . . . . . . . . . . . . 43
3.2 Edge Preserving Smoothing . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.1 Robust Estimators . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.2 Bilateral and Di usion Filtering . . . . . . . . . . . . . . . . . 49
3.3 Two State Channel Smoothing . . . . . . . . . . . . . . . . . . . . . . 53
4 Motion Estimation 61
4.1 Optical Flow and Range Flow . . . . . . . . . . . . . . . . . . . . . . . 62
4.1.1 Optical Flow and Motion Field . . . . . . . . . . . . . . . . . . 62
4.1.1.1 Barber’s pole illusion and complex motion . . . . . . 63
4.1.2 Brightness Change Constraint Equation . . . . . . . . . . . . . 65
4.1.3 Aperture Problem . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1.4 Range Flow Constraint Equation . . . . . . . . . . . . . . . . . 69
4.1.5 Aperture Problem Revisited . . . . . . . . . . . . . . . . . . . . 72
4.1.6 Local and Global Flow Estimation . . . . . . . . . . . . . . . . 74
4.1.6.1 Local Total Least Squares Estimation . . . . . . . . . 74
Gradient Based Weighting . . . . . . . . . . . . . . . . . 77
Minimum Norm Solutions . . . . . . . . . . . . . . . . . 78
4.1.6.2 Regularization of Local Flow Estimates . . . . . . . . 79
4.1.6.3 Performance Issues . . . . . . . . . . . . . . . . . . . . 80
4.1.7 Con dence and Type Measure . . . . . . . . . . . . . . . . . . 81
4.1.8 Combining Range and Intensity Data . . . . . . . . . . . . . . 83
4.1.9 Equilibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.2 Motion Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
II Experiments and Applications 93
5 Testbench Measurements 95
5.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.1.1 Power Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.2 Depth and Amplitude Analysis . . . . . . . . . . . . . . . . . . . . . . 100
5.2.1 Fixed Pattern Noise . . . . . . . . . . . . . . . . . . . . . . . . 101
x

Soyez le premier à déposer un commentaire !

17/1000 caractères maximum.