La lecture en ligne est gratuite
Le téléchargement nécessite un accès à la bibliothèque YouScribe
Tout savoir sur nos offres

Partagez cette publication

Vous aimerez aussi

de Imko

de Imko

### Cours de technique vocale pour les choristes

de Imko

suivant
Modeling with Bounded Partition Functions
Cavendish Laboratory University of Cambridge http://www.inference.phy.cam.ac.uk/rpa23/
16 July 2008
Overall Talk Message
I I I
I
I
Energy functions are nice models for data. Inference in energy models is often hard. If you can draw exact samples, you can do MCMC inference. I have a trick for generating exact data from many energy models. This trick is probably a bad idea.
Outline
Motivation Examples of Energy Models Inference Quick Review of MCMC Doubly-Intractable Posterior Distributions
Exchange Sampling Concept Auxiliary Variables Baby and Toy
Exact Sampling from Energy Models
Outline
Motivation Examples of Energy Models Inference Quick Review of MCMC Doubly-Intractable Posterior Distributions
Exchange Sampling Concept Auxiliary Variables Baby and Toy
Exact Sampling from Energy Models
Energy-based Models of Data
For some spaceX, write an energy:E(x;
Turn this into a probability distribution via:
p(x|θ) =Z(1θ)exp{−E(x;θ)}
θ)
θ)}:
Big energy implies small probability. Normalised byZ(θ) =ZXdxexp{−E(x; ICalled thepartition function. IDepends on the parametersθ. IIntractable in many interesting models.
Examples of Energy-based Models
Exponential Family Distributions
I
I
E(x;θ) =θTT(x) +h(x)
Gaussian, Gamma, Poisson, etc.
Typically easy.
Examples of Energy-based Models
Undirected Graphical Models
E(x;θ) =xTV xhTH hxTJ hxTαhTβ
IIsing/Potts models, Boltzmann machines IPerhaps hidden unitsh. IOften with ﬁnite states. IHard!
Examples of Energy-based
Nonparametric Models
I I I
E(x;θ) =g(x)
g(x)a nonparametric function. Logistic Gaussian process. Hard!
Models
Inference
p(θ| D)dθp(pD(θ|)θ)N =Rp(θ)nYp(xn|θ) =1
For interesting problems,θis often complex.
Use Markov chain Monte Carlo?
Quick Review of MCMC
We havep0(θ)p(θ)and want to draw samples.
Markov chain (MC): a stochastic rule for wandering around in the space ofθ.
MC can have anequilibrium distribution.
Simulate a MC for a while andθis close to being a sample from the equilibrium distribution.
Write down a rule usingp0(θ)so that the MC hasp(θ)as its equilibrium distribution.
MetropolisHastings
Metropolis–Hastings is a popular MCMC variant.
MH Markov Transition Rule Current state isθ. ˆ ˆ 1.Make a proposalθq(θθ). 2.Evaluate theacceptance ratio:
ˆ ˆ aq(θˆθ)p0(θ) = q(θθ)p0(θ)
ˆ 3.Acceptθwith probability min(a,1), otherwise keepθ.
Un pour Un
Permettre à tous d'accéder à la lecture
Pour chaque accès à la bibliothèque, YouScribe donne un accès à une personne dans le besoin