3D motion analysis via energy minimization [Elektronische Ressource] / vorgelegt von Andreas Wedel

Andreas Wedel

Informations

• Publié par: rheinische_friedrich-wilhelms-universitat_bonn
• Publié le: 01 janvier 2009
• Nombre de lectures: 19
• Langue: English
• Poids de l'ouvrage: 9 Mo

Extrait

3D Motion Analysis
via Energy Minimization
Dissertation
zur
Erlangung des Doktorgrades (Dr. rer. nat.)
der
Mathematisch-Naturwissenschaftlichen Fakult at
der
Rheinischen Friedrich-Wilhelms-Universit at Bonn
vorgelegt von
Andreas Wedel
aus
Siegburg
Bonn 2009Angefertigt mit Genehmigung der Mathematisch-Naturwissenschaftlichen Fakult at der
Rheinischen Friedrich-Wilhelms-Universit at Bonn.
1. Gutachter: Prof. Dr. Daniel Cremers
2. Gutachter: Prof. Dr. Bodo Rosenhahn
Tag der Promotion: 16. Oktober 2009
Diese Dissertation ist auf dem Hochschulserver der ULB Bonn unter
http://hss.ulb.uni-bonn.de/diss_online elektronisch publiziert.
Erscheinungsjahr: 2009iAcknowledgments
This work would not have been possible with-
out the support and encouragement of my su-
pervisors, Prof. Dr. Daniel Cremers and Dr. Uwe
Franke. It was them, who motivated and
prepared me for a good start into the fasci-
nating journey of computer vision. They chal-
lenged me on my way to think outside of
the box and to stay focused on a state-of-
the-art research path. They will also be
the ones to always stand up for me and ad-
vocate my work. I want to express my
deep gratefulness for all their scienti c, su-
pervisory and professional mentoring. I also
want to express my gratitude to the mem-
bers of the audit committee, Prof. Dr. Bodo
Rosenhahn, Prof. Dr.-Ing. Wolfgang F orstner, and
Prof. Dr. Reinhard Klein.
Special thanks goes to Dr. Thomas Pock and
Dr. Thomas Brox, who greatly in uenced my work.
c creative commons, air-on, www. ickr.com
I have rarely met people with such clear and pro-
found ability to explain even the most complex mathematical relationships as these ne men.
Thomas Pock was the one who showed me the path of total variation denoising for optical ow,
a path he himself has been traveling on. I thank Thomas Brox for his patience in explaining
small and simple, though very essential, mathematical basics to me.
Many thanks also to my co-authors Tobi Vaudrey, Jurgen Braun, Dr. Clemens Rabe, Dr. Jens
Klappstein, Dr. Hern an Badino, Dr. Thomas Schoenemann and Prof. Dr. Reinhard Klette. Es-
pecially, I want to thank Tobi Vaudrey and Clemens Rabe, whose support has been of great
value and who contributed with numerous illustrations to this thesis.
Concerning support related to beverage and food nourishment during my PhD journey, I
would like to thank Fridtjof and Heidi for espresso and cappuccino, Mario for steaks, Clemens
for Pizzas and Dave for doughnuts. For my spiritual nutrition, I was helped along the way by
the SYMs group at IBC Stuttgart and the Baptist church in B oblingen with a special thanks
to Hans-Martin Beutel. Through all times on my journey, I have learned to think positive and
put my faith in God.
On my journey I experienced both sunny and cloudy days. I have met many people who
pointed out directions, supported and prayed for me, and helped me over the deep canyons in
troubled times. There are many more people that have helped along the way who I owe a huge
dept of gratitude, but they are too numerous to name. With some people I have lost sight during
my journey, much to my regret.
It is all you people, who you have build the pathways and bridges I walked to write this thesis.
You all are part of this work and without every single one of you, I would not have be able to
successfully manage the challenges of a PhD thesis.
iiAbstract
This work deals with 3D motion analysis from stereo image sequences for driver assistance
systems. It consists of two parts: the estimation of motion from the image data and the seg-
mentation of moving objects in the input images. The content can be summarized with the
technical term machine visual kinesthesia, the sensation or perception and cognition of motion.
In the rst three chapters, the importance of motion information is discussed for driver assis-
tance systems, for machine vision in general, and for the estimation of ego motion. The next two
chapters delineate on motion perception, analyzing the apparent movement of pixels in image
sequences for both a monocular and binocular camera setup. Then, the obtained motion infor-
mation is used to segment moving objects in the input video. Thus, one can clearly identify the
thread from analyzing the input images to describing the input images by means of stationary
and moving objects. Finally, I present possibilities for future applications based on the contents
of this thesis. Previous work in each case is presented in the respective chapters.
Although the overarching issue of motion estimation from image sequences is related to prac-
tice, there is nothing as practical as a good theory (Kurt Lewin). Several problems in computer
vision are formulated as intricate energy minimization problems. In this thesis, motion analysis
in image sequences is thoroughly investigated, showing that splitting an original complex prob-
lem into simpli ed sub-problems yields improved accuracy, increased robustness, and a clear
and accessible approach to state-of-the-art motion estimation techniques.
In Chapter 4, optical ow is considered. Optical ow is commonly estimated by minimizing
the combined energy, consisting of a data term and a smoothness term. These two parts are
decoupled, yielding a novel and iterative approach to optical ow. The derived Re nement Op-
tical Flow framework is a clear and straight-forward approach to computing the apparent image
motion vector eld. Furthermore this results currently in the most accurate motion estimation
techniques in literature. Much as this is an engineering approach of ne-tuning precision to
the last detail, it helps to get a better insight into the problem of motion estimation. This
profoundly contributes to state-of-the-art research in motion analysis, in particular facilitating
the use of motion estimation in a wide range of applications.
In Chapter 5, scene ow is rethought. Scene ow stands for the three-dimensional motion
vector eld for every image pixel, computed from a stereo image sequence. Again, decoupling
of the commonly coupled approach of estimating three-dimensional position and three dimen-
sional motion yields anh to scene ow estimation with more accurate results and a
considerably lower computational load. It results in a dense scene ow eld and enables addi-
tional applications based on the dense three-dimensional motion vector eld, which are to be
investigated in the future. One such application is the segmentation of moving objects in an
image sequence. Detecting moving objects within the scene is one of the most important fea-
tures to extract in image sequences from a dynamic environment. This is presented in Chapter 6.
Scene ow and the segmentation of independently moving objects are only rst steps towards
machine visual kinesthesia. Throughout this work, I present possible future work to improve the
estimation of optical ow and scene ow. Chapter 7 additionally presents an outlook on future
research for driver assistance applications. But there is much more to the full understanding of
the three-dimensional dynamic scene. This work is meant to inspire the reader to think outside
the box and contribute to the vision of building perceiving machines.
iiiContents
Acknowledgments ii
Abstract iii
Introduction 1
1 Introduction 1
1.1 Eyes for Intelligent Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Intelligent Vehicle Safety Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Motion Analysis via Visual Kinesthesia . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Energy Minimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Contributions of this Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5.1 Visual Kinesthetic Perception (Theory) . . . . . . . . . . . . . . . . . . . 7
1.5.2 Cognition (Application) . . . . . . . . . . . . . . . . . 8
2 Vision in Natural Sciences 9
2.1 Biologically Inspired Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 History of Motion Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Categorization of Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3 Ego Motion Estimation 13
3.1 Image Formation and Coordinate Systems . . . . . . . . . . . . . . . . . . . . . . 14
3.1.1 Image Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.2 Coordinate Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Fundamental Matrix Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3 Robust Ego-Motion Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.1 The Linear Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.2 The Non-Linear Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.3 Re-weighted Least Squares and Parametrization . . . . . . . . . . . . . . 20
I Visual Kinesthetic Perception (Theory) 21
4 Optical Flow 22
4.1 Approaches in Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.1 Census Based Optical Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.2 The Optical Flow Constraint . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.3 Total Variation Opti