Classi cation and Feature Extractionin Man and MachineDissertationzur Erlangung des Grades eines Doktorsder Naturwissenschaftender Fakult at fur Mathematik und Physikder Eberhard-Karls-Universit at zu Tubingenvorgelegt vonArnulf B.A. Grafaus Lausanne (Schweiz)2004Tag der mundlic hen Prufung: 18.10.2004DekanProf. Dr. Peter Schmid1. BerichterstatterProf. Dr. Hanns Ruder und Prof. Dr. Bernhard Sch olkopf2.Prof. Dr. Heinrich H. Bultho AbstractThis dissertation attempts to shed new light on the mechanisms used by hu-man subjects to extract features from visual stimuli and for their subsequentclassi cation. A methodology combining human psychophysics and machinelearning is introduced, where feature extractors are modeled using methodsfrom unsupervised machine learning whereas supervised machine learning isconsidered for classi cation. We consider a gender classi cation task usingstimuli drawn from the Max Planck Institute face database. Once a featureextractor is chosen and the corresponding data representation is computed,the resulting feature vector is classi ed using a separating hyperplane (SH)between the classes. The behavioral responses of humans to one stimulus,in our study the gender estimate and its corresponding reaction time andcon dence rating, are compared and correlated to the distance of the fea-ture vector of this stimulus to the SH.