Biologically inspired binaural sound source localization and tracking for mobile robots [Elektronische Ressource] / vorgelegt von Laurent Calmes

Laurent Calmes

Sujets

• Informatik

Informations

• Publié par: rheinisch-westfalischen_technischen_hochschule_-rwth-_aachen
• Publié le: 01 janvier 2009
• Nombre de lectures: 2
• Langue: English
• Poids de l'ouvrage: 4 Mo

Extrait

Biologically Inspired Binaural Sound Source
Localization and Tracking for Mobile Robots
Von der Fakult at fur Mathematik, Informatik und Naturwissenschaften der RWTH
Aachen University zur Erlangung des akademischen Grades eines Doktors der
Naturwissenschaften genehmigte Dissertation
vorgelegt von
Diplom-Informatiker
Laurent Calmes
aus Luxemburg
Berichter: Universit atsprofessor Gerhard Lakemeyer, Ph.D.
Universit atsprofessor Dr. rer. nat. Hermann Wagner
Tag der mundlic hen Prufung: 23.12.2009
Diese Dissertation ist auf den Internetseiten der Hochschulbibliothek verfugbar.i
Acknowledgments
I would like to thank my supervisors Gerhard Lakemeyer and Hermann Wagner for
their support and patience and for giving me the opportunity to work on this fascinating
project on the intersection of Biology, Computer Science and Robotics. I am also grateful
for the freedom they gave me to develop my own ideas.
My warmest thanks go to Stefan Schi er, with whom I had many passionate dis-
cussions and whose insights and contributions to MCMCDA were invaluable. His non-
scienti c input was equally as important as his scienti c collaboration.
I would also like to thank Dominik R ottsches, Daniel Peger and Tobias Kr amer, who
made signi cant contributions to this project with their diploma theses.
I am also very grateful to all the friends and colleagues in the Knowledge-Based
Systems Group and the Institute for Biology II who made life in academia easier and
more enjoyable, through their scienti c and non-scienti c contributions, especially my
o ce colleague, Frank Endler.
And nally, my warmest gratitude goes to Katrin, who kept me going by providing
the right nudge at the right time.ii
Abstract
This thesis proposes, rstly, biologically inspired methods of binaural sound
source localization for mobile robots. Secondly, we propose a method for modulat-
ing the robot’s attention inspired from the barn owl and thirdly a tracking system
which makes it possible for a robot to track objects emitting sounds.
Regarding sound source localization, the method that was best understood
and evaluated is an algorithm based on the evaluation of interaural time di er-
ences (ITDs). There is a very simple reason for this state of a airs. Interaural
time di erences are in uenced mainly by the inter-microphone distance, provided
that there is no major obstruction (like an arti cial head) between them. This
would make the sound waves bend around the structure and thus increase the
path length and consequentially ITD in a frequency-speci c manner. As long as
there is no obstruction between the microphones and the far- eld assumption is
satis ed, the interaural time di erence directly relates to azimuth through a simple
equation, where only the additional parameters of inter-microphone distance (con-
stant) and speed of sound (can be regarded constant) are required. Under these
conditions, it is straightforward to adapt ITD localization to di erent hardware
platforms: it only requires mounting the microphones and providing the correct
microphone baseline value to the software. The method we use for ITD based
sound localization relies on detecting phase coincidence for individual frequencies
in the frequency domain and subsequent frequency integration to eliminate phase
ambiguities. Overall, the results of the system are excellent. Broadband signals
could be localized with an accuracy of about2 . The localization of pure tones
was highly erratic, as was to be expected. The only unexpected behavior was the
low accuracy in localizing 100 Hz | 1 kHz bandpass noise. By performing simu-
lations in which the room acoustics could be controlled, we could show that this is
caused by sound re ections from the environment. In larger rooms or, equivalently,
rooms with a lower direct-to-reverberant ratio, localization precision of broadband
signals also degrades signi cantly, which becomes evident in experiments on a real
robot. All in all, care has to be taken as to the acoustic environment in which the
ITD based sound source localization is to be deployed, in order to achieve best
performance.
Interaural level di erences based sound source localization by principle relies
on the acoustical properties of the microphone mount assembly and supporting
structures. This means that adapting ILD localization to a new platform is more
di cult. It requires mounting the microphones and then calibrating the whole
setup to record the resulting azimuth / elevation / frequency dependent ILD val-
ues, which can then be used by the ILD based sound source localization algorithm.
This is a quite elaborate, time-consuming procedure which has to be repeated ev-
ery time something changes in the way the microphones are mounted - or, indeed,
if the microphones themselves are changed. Experiments with arti cial owl ru s
illustrate this point: even small changes in the ru can have a huge impact on
the ILDs (and, to a lesser degree, on the ITDs). The method for ILD based
sound source localization relies on a neuronal model of the barn owl’s auditory
intensity pathway. Speci cally, the neuronal responses in the VLVp and the ICc
ls as well as the connections between these areas are modeled. The results of
the experiments with the algorithm are very encouraging. The rst tests showed
that the system was able to accurately localize broadband sound sources in the
range of 30 ::: +30 . The more elaborate arti cial ru s experiments con rmed
these results. Furthermore, with the correct acoustic design of the arti cial ru ,
it is possible to use the ILDs for various purposes as for example localization in
elevation and/or veri cation/correction of the ITD based azimuth estimates.iii
With the attentional module based on a neuronal saliency map it is possible to
preactivate a robot’s attention to a speci c region of interest. With this method
it was possible to successfully reproduce with a robotic pan-tilt unit attentional
latency experiments that were performed with barn owls. But the system we
propose can easily be generalized to modulate (in several instances) the attention
of the robot at various levels, from the basic sensor level up to the planning level.
The Markov chain Monte Carlo based combined sound source and dynamic
object tracking had a few problems accurately tracking our simulated entities.
Although the general viability of the method could be shown, the algorithm still
has several shortcomings. MCMCDA with a virtual sensor is able to correctly
track sound sources and objects alone, but the combination of both modalities in
one track proved to be di cult. As long as the individual entities are in clearly
distinct positions, correct tracks are produced, but if they approach each other or
- even worse - cross paths, tracking breaks down. This seems to be caused mainly
by the lack of distance information in the sound source localization modality. As
long as these shortcomings are not addressed, it makes little sense to test the
method on a real robot. This is why the MCMCDA experiments in this thesis
were limited to simulations.iv
Zusammenfassung
Diese Dissertation befasst sich mit, erstens, biologisch inspirierten Methoden
der binauralen Schallquellenlokalisierung fur mobile Roboter. Zweitens wird eine
von der Schleiereule inspirierte Methode zur Modulierung der Aufmerksamkeit
des Roboters vorgestellt und drittens ein System, das es dem Roboter ermoglicht,
Schall erzeugende Objekte zu verfolgen.
Die Schalllokalisierungsmethode die am besten verstanden und untersucht wur-
de ist ein Algorithmus der auf der Auswertung von interauralen Zeitdi erenzen (in-
teraural time di erences; ITDs) beruht. Daf ur gibt es einen sehr einfachen Grund.
Interaurale Zeitdi erenzen werden haupts achlich durch den Mikrophonabstand
beein usst, vorausgesetzt, die Mikrophone stehen frei - es be ndet sich also z.B.
kein Kunstkopf zwischen ihnen. Durch eine solche Struktur wurden die Schallwellen
frequenzabhangig um den Kopf herum \gebogen" werden, was den Weg der Schall-
welle von einem Mikro zum anderen verlangert und somit die ITD erhoht. Solange
sich keine Struktur zwischen den Mikrophonen bendet und die Schallquelle sich
im sog. Fernfeld be ndet, wird die ITD vom Schallquellenazimut ub er eine ein-
fache Gleichung bestimmt, die als einzige zusatzliche Parameter den Mikrophonab-
stand (konstant) und die Schallgeschwindigkeit (kann man als konstant annehmen)
enthalt. Unter diesen Bedingungen lasst sich die Schalllokalisierung sehr einfach
an verschiedene Hardware-Plattformen anpassen: man braucht nur die beiden
Mikrophone zu montieren und der Software den korrekten Abstand mitzuteilen.
Die Methode, die wir fur die ITD-basierte Schalllokalisierung benutzen detek-
tiert Phasenkoinzidenz fur individuelle Frequenzbander im Frequenzbereich und
eliminiert Phasenmehrdeutigkeiten durch anschlie ende Frequenzintegration. Die
Ergebnisse, die mit dem System erzielt wurden sind ausgezeichnet. Breitbandige
Signale konnten mit einer Prazision von2 lokalisiert werden. Die Lokalisierung
von Sinustonen war erwartungsgema extrem unzuverl assig. Das einzig Uner-
wartete war die niedrige Lokalisierungsprazision fur 100 Hz | 1 kHz Rauschen.
Durch Simulationen der Raumakustik konnte ermittelt werden, dass dies auf von
den Wanden herruhrenden Echos zuruckzufuhren ist. In gro eren R aumen (oder