Exponential Families Mixture Models Software library

Olivier Schwander

Description

Niveau: Supérieur
Exponential Families Mixture Models Software library Comparison Simplification and comparison of mixtures of Exponential families Séminaire MIA, La Rochelle Olivier Schwander Joint work with Frank Nielsen (LIX/Sony CSL), Julie Bernauer (LIX/INRIA), Adelene Sim, Michael Levitt (Stanford) LIX – École Polytechnique March 20, 2012 Olivier Schwander Simplification and comparison of mixture models

• mixture models


• wald distributions


• outline exponential


• exponential families


• adelene sim


• hardy-weinberg distribution


• source parameter

Sujets

• Levitt
• École polytechnique
• Bernauer
• Nielsen
• Mixture model
• Exponential family

Informations

• Publié par: chaeh
• Nombre de lectures: 99
• Poids de l'ouvrage: 1 Mo

Extrait

Exponential Families
Mixture Models
Software library
Comparison
Simplification and comparison of mixtures of
Exponential families
Séminaire MIA, La Rochelle
Olivier Schwander
Joint work with Frank Nielsen (LIX/Sony CSL), Julie Bernauer
(LIX/INRIA), Adelene Sim, Michael Levitt (Stanford)
LIX – École Polytechnique
March 20, 2012
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models
Software library
Comparison
Outline
Exponential Families
Definition and Examples
Bregman divergences
Mixture Models
Statistical mixtures
Getting
Simplification of GMM
Software library
Presentation
Comparison
Majoring Kullback-Liebler
Other ground distances
Comparing and EMD
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
An ubiquitous family
Game: guess corresponding distributions Images from Wikipedia
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
An ubiquitous family
Gaussian or normal (generic, isotropic Gaussian, diagonal Gaussian,
rectified Gaussian or Wald distributions, log-normal), Poisson,
Bernoulli, binomial, multinomial (trinomial, Hardy-Weinberg
distribution), Laplacian, Gamma (including the chi-squared), Beta,
exponential, Wishart, Dirichlet, Rayleigh, probability simplex,
negative binomial distribution, Weibull, Fisher-von Mises, Pareto
distributions, skew logistic, hyperbolic secant, negative binomial,
etc.
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Exponential families
Definition
p(x;) = p (x;) = exp (ht(x)ji F () +k(x))F
I source parameter
I t(x) sufficient statistic
I natural parameter
I F () log-normalizer
I k(x) carrier measure
F is a stricly convex and differentiable function
h j i is a scalar product
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Examples
Poisson distribution
x
p(x;) = exp( )
x!
I t(x) = x
I = log
I F () = exp()
I k(x) = 0
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Multivariate normal distribution
Exponential family

t 11 (x ) (x )
p(x; ; ) = p exp
22 det

1 1 1I = ( ; ) = ; 1 2 2
1 1 T 1 dI F () = tr logdet + log2 11 24 2 2
tI t(x) = (x; x x)
I k(x) = 0
Composite vector-matrix inner product
0 t 0 t 0h;i = +tr( )1 1 2 2
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Bregman divergences
Bregman divergence
B (p;q) = F (p) F (q) +hp qjrF (q)iF
F is a strictly convex and differentiable function
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Examples
Olivier Schwander Simplification and comparison of mixture modelsExponential Families
Mixture Models Definition and Examples
Software library Bregman divergences
Comparison
Bregman ball
Itakura-Saito
I Left-sided ball
I Right-sided ball
Left-sided ball is convex since D ( k q) is convexF
Very useful in sound processing applications
Olivier Schwander Simplification and comparison of mixture models