Vision-based posture detection and tracking for interactive scenarios [Elektronische Ressource] / Joachim Schmidt. Technische Fakultät - AG Angewandte Informatik

Joachim Schmidt

Sujets

• Informatik

Informations

• Publié par: universitat_bielefeld
• Publié le: 01 janvier 2011
• Nombre de lectures: 32
• Langue: English
• Poids de l'ouvrage: 21 Mo

Extrait

Vision-based Posture Detection
and Tracking for Interactive
Scenarios
Dissertation zur Erlangung des akademischen Grades
Doktor der Ingenieurwissenschaften (Dr.-Ing.)
der Technischen Fakultät der Universität Bielefeld
vorgelegt von
Joachim SchmidtGedruckt auf alterungsbeständigem Papier nach ISO 97062.1 Person Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Pose Reconstruction and Motion Tracking . . . . . . . . . . . . . . . . . . . 7
2.3 Model Acquisition, Initialization and Error Recovery . . . . . . . . . . . . 12
2.4 Vision for Human Robot Interaction . . . . . . . . . . . . . . . . . . . . . . 14
3.1 Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.1 Definition of an Optimization Problem . . . . . . . . . . . . . . . . 17
3.1.2 Problem Classification . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.3 Optimality Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Deterministic Optimization Algorithms . . . . . . . . . . . . . . . . . . . . 21
3.2.1 The Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.2 The Mean Shift . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 Probabilistic Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . 26
3.3.1 Particle Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3.2 Kernel Particle Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.3 Evolutionary Computation . . . . . . . . . . . . . . . . . . . . . . . 37
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1 Applicability to Different Scenarios . . . . . . . . . . . . . . . . . . . . . . 45
4.1.1 Industrial Working Cell Safety . . . . . . . . . . . . . . . . . . . . . 46
4.1.2 Scene Exploration with a Mobile Robot . . . . . . . . . . . . . . . . 46
4.2 Person Localization System Design . . . . . . . . . . . . . . . . . . . . . . . 47
4.3 6D Point Cloud Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.1 Velocity Computation using a Stereo Camera Setup . . . . . . . . . 48
4.3.2 V using a Time-of-Flight Sensor . . . . . . . . 52
4.4 Generation and Tracking of Object Hypotheses . . . . . . . . . . . . . . . . 56
4.4.1 Over-Segmentation for Motion-Attributed Clusters . . . . . . . . . 56
4.4.2 Weak Model for Object Hypotheses . . . . . . . . . . . . . . . . . . 57
4.4.3 Kernel Particle Filter for Object Localization . . . . . . . . . . . . . 57
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.1 Human Robot Interaction Scenario . . . . . . . . . . . . . . . . . . . . . . . 63
5.2 Body Pose Tracking System Overview . . . . . . . . . . . . . . . . . . . . . 64
i
PBoContents1Lo563folizationroachesPAppIntroRelatedT245RecognizingcaHumanserson5431OptimizationductionTrackingechniquesose17rdyContents
5.3 Modeling the Appearence of Humans . . . . . . . . . . . . . . . . . . . . . 65
5.3.1 Articulated 3D Body Model . . . . . . . . . . . . . . . . . . . . . . . 66
5.3.2 The Monocular Challenge . . . . . . . . . . . . . . . . . . . . . . . . 71
5.3.3 Image Cues for Body Pose Tracking . . . . . . . . . . . . . . . . . . 72
5.3.4 Body Pose Observation Model . . . . . . . . . . . . . . . . . . . . . 82
5.4 Kernel Particle Filtering for Body Pose Tracking . . . . . . . . . . . . . . . 84
5.4.1 Refinement of the Particle Distribution . . . . . . . . . . . . . . . . 85
5.4.2 Extracting the Best Body Pose . . . . . . . . . . . . . . . . . . . . . 87
5.4.3 Motion Models for Body Pose Tracking . . . . . . . . . . . . . . . . 87
5.4.4 Random Noise Propagation . . . . . . . . . . . . . . . . . . . . . . . 88
5.5 Body Model Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.5.1 Automatic Procedure Overview . . . . . . . . . . . . 93
5.5.2 Face and Hands Detection . . . . . . . . . . . . . . . . . . . . . . . . 95
5.5.3 Integration into the Body Pose Tracking System . . . . . . . . . . . 97
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.1 Evaluating the Person Localization . . . . . . . . . . . . . . . . . . . . . . . 101
6.2 Ev the Body Pose Tracking . . . . . . . . . . . . . . . . . . . . . . . 102
6.2.1 Marker-Based Ground Truth . . . . . . . . . . . . . . . . . . . . . . 103
6.2.2 Error Measure Definition . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2.3 Evaluating the Accuracy of the Body Pose Tracking . . . . . . . . . 109
6.3 Automatic Parameter Optimization for Body Pose T . . . . . . . . 112
6.3.1 Genetic Algorithms for Parameter Optimization . . . . . . . . . . . 113
6.3.2 Parameter Optimization Results . . . . . . . . . . . . . . . . . . . . 117
6.4 Evaluating the Automatic Initialization Procedure . . . . . . . . . . . . . . 124
7.1 Person Localization for Scene Reconstruction . . . . . . . . . . . . . . . . . 127
7.2 Body Pose Tracking for Object Attention . . . . . . . . . . . . . . . . . . . 131
7.2.1 Object Attention System Overview . . . . . . . . . . . . . . . . . . . 132
7.2.2 Trajectory-Based Gesture Recognition . . . . . . . . . . . . . . . . . 132
7.2.3 Object Attention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.2.4 Evaluating the System Performance . . . . . . . . . . . . . . . . . . 134
7.3 Hand Gesture Detection using the Body Pose Tracking . . . . . . . . . . . 136
7.4 Motionese Developmental Studies . . . . . . . . . . . . . . . . . . . . . . . 139
ii
ok127Optimization87Bibliography6101SystemOutloEva141luation153andApplications“Man has learned much from studies of natural systems, using what has been
learned to develop new algorithmic models to solve complex problems. [...] A
major thrust in algorithmic development is the design of algorithmic models to
solve increasingly complex problems. Enormous successes have been achieved
through the modelling of biological and natural intelligence, resulting in so-called
’intelligent systems’.”
Andries P. Engelbrecht (2007) [43]
“At the basic level, the name given to the science dedicated to the broad area of
human movement is kinesiology. It is an emerging discipline blending aspects of
psychology, motor learning, and exercise physiology as well as biomechanics.
Biomechanics, as an outgrowth of both life and physical sciences, is built on the
basic body of knowledge of physics, chemistry, mathematics, physiology, and
anatomy. It is amazing to note that the first real ’biomechanicians’ date back to
Leonardo DaVinci, Galileo, Lagrange, Bernoulli, Euler, and Young. All these
scientists had primary interests in the application of mechanics to biological
problems.”
David A. Winter (1990) [171]
Vitruvian Man. Painting by Leonardo Da Vinci (1485/90, Venedig,
Galleria dell’ Accademia), Photo by Luc Viatour.
1
.1:Figure1ductionIntro11 Introduction
As Engelbrecht and Winter mention, it has often been nature that inspired man to
develop new ideas and that encouraged us to use these ideas for applications that can
affect our daily life. For any scientist, curiosity and amazement are two substantial char-
acteristics. For me this has often shown in amazement about the solutions that nature
provides for many big and small problems and in curiosity, how theories, concepts and
finally algorithms and systems could eventually be derived from that. These are the
kind of thoughts that have driven my research for the last years.
The thesis here present is about the perception of the human body and the environment
with means of computer vision and the analysis of these information for applications
in the field of human robot interaction. The discussion will mostly be about real-world
scenarios involving the observation of real humans; that means we will have to deal
with an ever-changing and dynamic environment and possibly large variations in the
appearance of an object to be observed. This poses a huge challenge to automated vision
techniques. Additional constraints can ease the problem, but also make the resulting
system less flexible. The presented work combines various techniques from computer
vision and optimization theory. The scenarios that are addressed are wide spread:
worker safety in an industry environment, interacting with a mobile robot and even
understanding the relevance of gestures for learning in children. The common ground
for all these scenarios is the fact that methods from computer vision are applied to
enable or to understand an interaction between humans among themselves and humans
and machines. The best way to outline the scope of this thesis is to describe the topics
covered.
Computer vision is a broad discipline, as are the applications where automated vision
techniques are applied. To get a better focus on the relevant topics, the Chapter (2) gives
an overview of related approaches and techniques that are of special importance for
this thesis. The basic step for any of the presented approaches is to find the human in
the scene. For camera images, humans can be found based on their appearance. More
detailed methods are able to find individual body parts and can put them in relation
with each other to reconstruct the pose of the human. Besides working with the 2D
information from a single image, using volumetric data has become more and more
common with the availability of afford