Advanced probabilistic network modeling framework with qualitative prior knowledge [Elektronische Ressource] / Rui Chang

Sujets

• Informatik

Informations

• Publié par: technische_universitat_munchen
• Publié le: 01 janvier 2008
• Nombre de lectures: 12
• Langue: English
• Poids de l'ouvrage: 9 Mo

Extrait

Institut fu¨r Informatik
der Technischen Universit¨at Mu¨nchen
Advanced Probabilistic Network
Modeling Framework with Qualitative
Prior Knowledge
Rui Chang
Vollsta¨ndiger Abdruck der von der Fakult¨at fur¨ Informatik der Technischen
Universit¨at Munc¨ hen zur Erlangung des Akademischen Grades eines
Doktors der Naturwissenschaften (Dr. rer. nat.)
genehmigten Dissertation.
Vorsitzender: Univ.-Prof. Dr. H. J. Schmidhuber
Pruf¨ er der Dissertation:
1. Univ.-Prof. Dr. Dr.h.c.mult. W. Brauer, em.
2. Univ.-Prof. B. Brug¨ ge, Ph.D.
DieDissertation wurdeam 07.01.2008 beiderFakult¨at der TechnischenUniver-
sita¨t Munc¨ hen eingereicht und durch die Fakult¨at fur¨ Informatik am 28.01.2008
angenommen.To my parentsAbstract
The ever increasing amount of information in every scientific and industrial
domain have been an exciting challenge for computer scientist to handle vast
amountofdataandtorepresenthumanunderstandingsofadomaininasystem-
aticandmathematicway. Overdecades,probabilisticmodelingwithprobability
theory and statistical learning algorithms has been popular for accomplishing
this task due to the stochastic characteristics of the nature. Quantitative mea-
surements are generated from various kinds of ”sensors” in all types of science
and industry and we need to make sense of these data, i.e. to extract im-
portant patterns and trends, and understand ”what the data says”. This is
often called learning from data, reverse-engineering or bottom-up modeling.
Among these learning algorithms, Bayesian network computational framework
has become particular popular due to the ability of Bayesian network to model
cause-effect interactions between the variables in a domain. For example, in
bioinformatics, vast amount of ”-omics” data are generated by high-throughput
screening techniques. Learning method with Bayesian networks has been used
to construct gene regulatory networks from transcriptomic data and to predict
protein-protein interactions based on proteomic data.
In practice, the data basis in reverse-engineering approach can be very
sparse. Therefore, it is hardly sufficient to select one adequate model, i.e. there
isconsiderablemodeluncertainty. SelectingonesingleBayesianmodelcanthen
lead to strongly biased inference results. In this case, full Bayesian approach
with model averaging can be used to alleviate the bias. In this approach, one
majordifficultyistospecifypriordistributionfunctionontheBayesiannetwork
structure space and parameter space in order to compute a posterior probabil-
ity. One important information resources that could provide solutions to this
problem is qualitative prior distribution which largely exists in every science
and industry domain. In addition, human have a deep intuition that causal-
ity is a central and cohesive aspect of their perceptions, therefore, one subtype
of these qualitative prior knowledge, i.e. qualitative causal knowledge which
describes the cause-effect relations between multiple entities with any form of
uncertainties, are particularly well-suited to represent human understandings
and to get approximated characterizations of the behavior of the interested do-
main. For example, in a qualitative causal statement: ”smoking increases the
risk of lung cancer”, two entities: smoking and lung cancer are related to each
other. Moreover, smoking positively influences lung cancer since lung cancer
risk is increased in case of smoking. It is therefore desirable to make use of this
body of evidence in probabilistic modeling with Bayesian network.
This thesis is concerned with developing a powerful probabilistic modeling
framework to represent human understandings of a domain based on qualita-
2tive prior knowledge. More precisely, to construct a Bayesian network structure
with cause-effect relationships between the entities in a domain and parame-
terize these interactions according to the semantics of qualitative knowledge.
One problem here is that qualitative knowledge provides no quantitative infor-
mation to parameterize edges in Bayesian network and parameters need to be
configured based on soley qualitative information. We attack this problem by
proposing a qualitative knowledge model which is responsible for constructing
mathematical constraints to define parameter distribution based on the quali-
tative knowledge. This approach incorporates the concept of model uncertainty
due to the qualitative nature of the statements and automatically select a class
of possible Bayesian models which are consistent with the semantics of the
statements. Quantitative Bayesian network inference is performed by averaging
inferences of each Bayesian network in this class with full Bayesian approach.
However,knowledgeiswell-knowntobeinconsistentandincomplete. Knowl-
edgehasspatialandtemporalpropertieslikeotherphysicalsystems, i.e. knowl-
edge exist in space-time dimension. The spatial property describes that knowl-
edge represents information on a specific sub-structure of a domain and the
temporal property states that knowledge represents human understandings at
a particular time point. Thus, these knowledge are incomplete and may be
updated by complementary discovery. Moreover, another significant drawback
of knowledge is inconsistency. In the same domain, there may exist contra-
dicting qualitative statements on dependency, causality and parameters over a
set of entities. In this thesis, we propose several successful methods to deal
with knowledge incompleteness and inconsistency, and integrate the Bayesian
networks based on the set of knowledge to form an complete and coherent rep-
resentation of the underlying system.Acknowledgements
This dissertation is developed based on my work within a corporative Ph.D
program between the Informatics Institute of Technical University of Munich
and the Learning System Department (CT IC4) of Siemens AG. In the past
3 years, it has been my extreme pleasure and fortune to work within such
a friendly and professional atmosphere provided by both parties of my Ph.D
program. This thesis would not have been possible without the kindness and
generosity of all the members of our group. I feel grateful and indebted to have
received their helps and suggestions.
First of all, I would like to thank my academic supervisor, Prof. Wilfried
Brauer from the Institute of Informatics of Technical University of Munich who
has been constantly supporting me on my researches by providing me his re-
markable insights and constructive suggestions. Prof. Wilfried Brauer is such
kind person so that whenever I need his help, he is always there for me. I am
verygratefultohiskindnessandIamdeeplyimpressedbyhisopen-mindedness
and astonishing academic achievements.
Secondly, I would like to thank Siemens AG, especially Siemens Corporate
Technology, for endowing me such a great opportunity to carry out the cutting-
edge researches in machine learning and Bioinformatics field and for providing
metheSiemensDoctoralscholarship. IfeelgratefultoProf. BerndSchur¨ mann,
theleaderoftheLearningSystemsDepartmentatSiemensAG,forhisconstant
support to my research. I appreciate his emphasis on both scientific research
and real-world applications.
Specially, I am indebted to my co-supervisor at Siemens AG, Dr. Martin
Stetter, the principle investigator of my team, who has brought me into the
world of statistical learning and Bioinformatics. He has the greatest influence
in my intellectual, knowledge development and my commitment to my future
career. I am deeply impressed by his enthusiasm, intelligence, open-mindedness
whichconstantlyencouragesmetoovercomedifficultiesandtopursuesuccesses
in my research and make my time at Siemens AG memorable.
I appreciate the friendship and fellowship to my colleagues, Dr. Math¨aus
Dejori, who often enlighten me with his smart ideas. His solid knowledge in
machine learning and sharp way of thinking often bring us surprises. Also, I
enjoyed the scientific discussions with Andreas Na¨gele who often provide me
useful advices from another point of view. His logic way of thinking impressed
me a lot. Meanwhile, I would like to thank Holger Arndt who answered my
questions on programming and computer hardware. Specially, I would like to
thank Dr. Jakub Pijewski with whom I enjoyed to talk about biology with his
talented minds. I would like to thank to the secretary Mrs. Christina Singer at
Siemens AG and Mrs. Erika Leber from Technical University of Munich, who
4kindly helped me to deal with administration documents. Moreover, I would
liketothanktoDr. VolkeTresp,Dr. KaiYu, Dr. ShipengYu, Dr. XuZhao, Yi
HuangandHuaienGao, whooftenanswermyquestionsandgivepleasantchats
which make my research joyful. Finally, I would like to thank to my parents
and Ms. Wenbo He who constantly give me the most love, support and care in
my personal life without which I could not have ever accomplished my Ph.D.Contents
1 Introduction 2
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Overview of Data-driven Bayesian Modeling Approach . . . . . . 9
1.2.1 Bayesian Networks . . . . . . . . . . . . . . . . .