SVM-Tutorial-Guestrin

Guestrin

Description

Two SVM tutorials linked in class website (please, read both): High-level presentation with applications (Hearst 1998) Detailed tutorial (Burges 1998)SVMs, Duality and the Kernel TrickMachine Learning – 10701/15781Carlos GuestrinCarnegie Mellon UniversitythFebruary 27 , 20062006 Carlos Guestrin 1Announcements Third homework is outst Due March 1 Final assigned by registrar: May 12, 1-4p.m Location TBD Midtermth March 8 , a week from Wednesday Open book, notes, papers, etc. No computers22006 Carlos GuestrinSVMs reminder32006 Carlos GuestrinToday’s lecture Learn one of the most interesting and exciting recent advancements in machine learning The “kernel trick” High dimensional feature spaces at no extra cost! But first, a detour Constrained optimization!42006 Carlos GuestrinConstrained optimization52006 Carlos GuestrinLagrange multipliers – Dual variablesMoving the constraint to objective functionLagrangian:Solve:62006 Carlos GuestrinLagrange multipliers – Dual variablesSolving:72006 Carlos GuestrinDual SVM derivation (1) –the linearly separable case82006 Carlos GuestrinDual SVM derivation (2) –the linearly separable case92006 Carlos GuestrinDual SVM interpretation102006 Carlos Guestrinw.x+ b = 0Dual SVM formulation –the linearly separable case112006 Carlos GuestrinDual SVM derivation –the non-separable case122006 Carlos GuestrinDual SVM formulation –the ...

Informations

• Publié par: Vowyong
• Nombre de lectures: 71
• Langue: English

Extrait

Two SVM tutorials linked in class website (please, read both): High-level presentation with applications (Hearst 1998) Detailed tutorial (Burges 1998)

SVMs, Duality and the Kernel Trick

Machine Learning – 10701/15781 Carlos Guestrin Carnegie Mellon University

February 27 th , 2006 2006 Carlos Guestrin



Announcements

 Third homework  is out  Due March 1 st

 Final assigned by registrar:  May 12, 1-4p.m  Location TBD

 Midterm  March 8 th , a week from Wednesday  Open book, notes, papers, etc. No computers

2006 Carlos Guestrin



SVMs

reminder

2006 Carlos Guestrin



Today’s lecture

 Learn one of the most interesting and exciting recent advancements in machine learning  The “kernel trick”  High dimensional feature spaces at no extra cost!  But first, a detour  Constrained optimization!

2006 Carlos Guestrin



Constrained optimization

2006 Carlos Guestrin



Lagrange multipliers – Dual variables

Solve:

Moving the constraint to objective function Lagrangian:

2006 Carlos Guestrin



Lagrange multipliers –

Solving:

2006 Carlos Guestrin

Dual variables



Dual SVM derivation (1) – the linearly separable case

2006 Carlos Guestrin



Dual SVM derivation (2) – the linearly separable case

2006 Carlos Guestrin



Dual SVM interpretation

2006 Carlos Guestrin



Dual SVM formulation – the linearly separable case

2006 Carlos Guestrin

