Concepts and Techniques Spring 2009

Asif Iqbal

Description

http://en.wikipedia.org/wiki/Naive_Bayes_classifier ..... then P("soccer"|Sports) = 37/153. Word “soccer" occurs 1+1+1 = 3 times in politics docs.

Sujets

• Sport

Informations

• Publié par: Sportif
• Nombre de lectures: 70
• Langue: English

Extrait

CSE-590/634 Data Mining Concepts and Tec   h Spring 2009 inuqe Bayesian Classification s   Presented by: Muhammad A. Islam, 106506983 MoieedAhmed, 106867769   Guided by: Prof. Anita Wasilewska  

ƒ  •  • Bibliography DATA MINING Concepts and n T d echniques, JiaweiHan, MichelineKamber Morgan Kaufman Publishers, 2Edition.  ƒ Chapter 6, Classification and Prediction, Section 6.4. Computer Science, Carnegie Mellon University – http://www.cs.cmu.edu/~awm/tutorials – http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch6.pdf – http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html Wikipedia: – –     h h t t t t p p : : / / / / e e n n . . w w i i k k i i p p e e d d i i a a . . o o r r g g / / w w i i k k i i / /B N a ai y v e e si _ a B n a _ y p e r s o _ b c a l b a i s l s i i t f y ier

Outline • Introduction to Bayesian Classification – Probability – BayesTheorem – Naïve Ba   yesClassifier – Classification E   xample • Text Classification –an Application • Paper: “Text Mining: Fi   nding Nuggets in Mountains of Textual Data”

Introduction to Bayesian  Classification yB Muhammad A. Islam 106506983  

` Bayesian Classification What is it ? ◦ Statistical method for classification. ◦ Su   pervised Learning Method. ◦ As   sumes an underlying probabilistic model, the Bayes the   orem. ◦ Can solve diagnostic and predictive problems. ◦ Ca   n solve problems involving both categorical and co   ntinuous valued attributes. ◦ Named after Thomas Bayes, who proposed the Bayes Th   eorem.  

 •  • Basic Probability Concepts Sample space S is a set of all possible outcomes – S = {1,2,3,4,5,6} for a dice roll – S = {H,T} for a coin toss.  yna si A tnevE nAecapS elpmaS eht fo tesbus – Seeing a 1 on the dice roll  – oc a no daeh gnitteGssot ni

 •  •  •  • Random Variables A is a random variable if A denotes an event, and there is some degree of uncertainty as to whether A occurs. – A = The US president in 2016 will be male – A = You see a head on a coin toss – A = The weather will be sunny tomorrow Boolean random variables – A is either true or false. Discrete random variables – Weather is one of {sunny, rain, cloudy, snow} Continuous random variables – Temp=21.6.

ƒ Probability We write P(A) as “the fraction of possible worlds in which A is true” Event space of all possible worlds Its area is 1 Worlds in which A is true Worlds in which A is False P(A) = Area of reddish oval http://www.cs.cmu.edu/~awm/tutorials

The axioms of Probability ` 0 <= P(A) <= 1 ` ` P P ( (T F r a u ls e e )  )  = =     1 0 ` P(A or B) = P(A) + P(B) -P(A and B) From these we can prove: ` P(notA) = P(~A) = 1-P(A) ` ^ A(P + )B ^ A(P = )A(P)B~  http://www.cs.cmu.edu/~awm/tutorials

 • Conditional Probability P(A|B) = Fraction of worlds in which B is true that also have A true F H H = “Have a headache” F = “Coming down with Flu” P(H) = 1/10 P(F) = 1/40 P(H|F) = 1/2 “Headaches are rare and flu is rarer, but if you’re coming down with ‘flu there’s a 50-50 chance you’ll have a headache.” http://www.cs.cmu.edu/~awm/tutorials

Conditional Probability F H H = “Have a headache” F = “Coming down with Flu” P(H) = 1/10 P(F) = 1/40 P(H|F) = 1/2 P(H|F) = Fraction of flu-inflicted worlds in which you have a headache  = #worlds with flu and headache #worlds with flu Area of “H and F”region =   Area of “F”region P(H ^ F)    =  )F(P http://www.cs.cmu.edu/~awm/tutorials