A SHORT PROOF OF THE “CONCAVITY OF ENTROPY POWER”

Description

A SHORT PROOF OF THE “CONCAVITY OF ENTROPY POWER” C. VILLANI Abstract. We give a simple proof of the “concavity of entropy power”. Key words : Entropy power, Fisher information, heat semigroup. Let f be a probability measure on Rn. We define the action of the heat semigroup (Pt)t≥0 on f , by the solution of the partial differential equation ∂ ∂tPtf = ∆(Ptf). Equivalently, Ptf is the convolution of f with the n-dimensional Gauss- ian density having mean vector 0 and covariance matrix 2tIn, where In is the identity matrix. The “concavity of entropy power” theorem states that (1) d 2 dt2N(Ptf) ≤ 0. Here N(f) = e 2H(f) n 2pie , H(f) = ? ∫ R n f log f. The functional N(f) is the so-called ”entropy power” of f , as intro- duced by Shannon, while H(f) is Shannon's entropy functional (which coincides to Boltzmann's entropy up to a change of sign). The normal- izing factor 2pie is nonessential and we mention it only to stick to the conventions of Shannon. Inequality (1) is due to Costa [4].

• entropy power


• dimensional gauss- ian


• proof does


• ecole normale


• ?u ·


• blachman-stam inequality


• boltzmann's entropy up


• remainder term


• differential equation

Sujets

• École normale
• Series (mathematics)
• Differential equation

Informations

• Publié par: profil-urra-2012
• Nombre de lectures: 22
• Langue: English

Extrait

A SHORT PROOF OF THE “CONCAVITY OF
ENTROPY POWER”
C. VILLANI
Abstract. We give a simple proof of the “concavity of entropy
power”.
Key words : Entropy power, Fisher information, heat semigroup.
nLet f be a probability measure onR . We define the action of the
heat semigroup (P ) on f, by the solution of the partial differentialt t‚0
equation
@
P f =Δ(P f):t t
@t
Equivalently,P f istheconvolutionoff withthen-dimensionalGauss-t
ian density having mean vector 0 and covariance matrix 2tI , wheren
I is the identity matrix. The “concavity of entropy power” theoremn
states that
2d
(1) N(P f)•0:t2dt
Here
2H(f) Z
ne
N(f)= ; H(f)=¡ flogf:
2…e n
The functional N(f) is the so-called ”entropy power” of f, as intro-
duced by Shannon, while H(f) is Shannon’s entropy functional (which
coincides to Boltzmann’s entropy up to a change of sign). The normal-
izing factor 2…e is nonessential and we mention it only to stick to the
conventions of Shannon.
Inequality (1) is due to Costa [4]. Later, Dembo [5, 6] simplified the
proof, by an argument based on the Blachman-Stam inequality [3],
1 1 1
• + :
I(f⁄g) I(f) I(g)
Here f and g are two arbitrary probability densities, and
Z
2jrfj
I(f)=
n f
RR2 C. VILLANI
stands for the Fisher information of f. Actually, Dembo proved in-
equality (1) in the equivalent form
fl
2 flI(f) 1 dfl(2) J(f)‚ ; J(f)=¡ I(P f):tfln 2 dt
t=0
Using basic considerations on the heat equation, like the continuity of
H(P f) w.r.t f (when f varies in a class s.t H(f) stays bounded), itt
is sufficient to prove (1), or equivalently (2), for a very smooth initial
datum f, with fast decay at infinity. In order not to worry about
2logarithms, we may also impose thatjlogf(x)j•C(1+jxj ) for some
constant C. The general case will follow by density.
Our goal here is to give a direct proof of (2), in a strengthened form,
with an exact error term. Our proof relies on the following lemma,
well-known in certain circles.
Lemma. Let f be a smooth, rapidly decaying probability density, such
that logf has growth at most polynomial at infinity. Then,
ZX £ ⁄2
J(f)= f @ (logf)ij
n
ij
• ‚Z 2X @ f @f@ fij i j
= f ¡ :
2
n f f
ij
Here the summation is taken over all indices 1• i• n, 1• j • n.
This computation (or actually a variant of it) was performed by McK-
ean [7] in one dimension of space, and easily generalized by Toscani [8]
to the n-dimensional case. But this lemma is also a particular case
of the identities of Bakry and Emery [2], established through the so-
called Γ calculus as part of their famous work on logarithmic Sobolev2
inequalities and hypercontractive diffusions. For the sake of complete-
ness, we give here a simple proof which is inspired from Bakry and
Emery.
Proof of Lemma. Write the Fisher information in the form
Z
2I(f)= fjr(logf)j ;
so that, by differentiation under the integral sign,
fl Z Z ? ¶
fld Δf
2fl(3) I(P f)= Δfjr(logf)j +2 fr(logf)¢r :tfldt ft=0
RRCONCAVITY OF ENTROPY POWER 3
We express Δf=f in terms of logf, thanks to the elementary identity
Δf 2=Δ(logf)+jr(logf)j ;
f
so that (3) becomes
(4)Z Z Z
2 2Δfjr(logf)j +2 fr(logf)¢rΔ(logf)+2 fr(logf)¢rjr(logf)j :
The first integral in (4) can of course be rewritten as
Z
2fΔjr(logf)j ;
while the third one is
Z Z
2 22 rf¢rjrlogfj =¡2 fΔjr(logf)j :
On the whole, (3) is equal to
• ‚Z
2f 2r(logf)¢rΔ(logf)¡Δjr(logf)j :
We conclude by the elementary identity (in which the reader may rec-
ognize a trivial particular case of Bochner’s formula)
X
2 22ru¢rΔu¡Δjruj =¡2 (@ u) :ij
ij
⁄
With this lemma at hand, the proof of (2) is almost immediate.
Proposition. Let f be a smooth, rapidly decaying probability density,
such that logf has growth at most polynomial at infinity. Then,
2I(f)
J(f)‚ :
n
Proof. Consider the nonnegative quantity
? ¶Z 2X @ f @f@ fij i j
A(‚)= ¡ +‚– f;ij2f f
ij
and expand this expression as a trinom in ‚. Since
Z Z Z
2X (@f)i
@ f =0; =I(f); f =1ii
f
i4 C. VILLANI
we obtain
? ¶Z 2X @ f @f@ fij i j 2A(‚)= ¡ f¡2‚I(f)+‚ n:
2f f
ij
Now, the choice ‚=I(f)=n yields the equality
Z ? ¶22 XI(f) @ f @f@ f I(f)ij i j
(5) J(f)¡ = ¡ + – f ‚0:ij2n f f n
ij
⁄
Remarks.
(1) It is easy to check that, at least under suitable smoothness
assumptions,equalityin(2)occursifandonlyiff isanisotropic
Gaussian.
(2) As one of the referees pointed out, a proof of the Proposition in
the same spirit as the above argument is implicit in the notes
by D. Bakry [1, p.103, remarks following the proof of Propo-
sition 6.7]. Namely, applying the Cauchy-Schwarz inequality
twice,
X£ ⁄ £ ⁄12 2
@ (logf) ‚ Δ(logf) ;ij
n
ij
Z ?Z ¶2£ ⁄2 2f Δ(logf) ‚ fΔ(logf) =I(f)
R
(where we used again f = 1). While this proof does not give
any simple remainder term, one advantage is that –as again
pointed out by the referee– it also works for Riemannian man-
ifolds with nonnegative Ricci curvature.
References
[1] Bakry, D. L’hypercontractivit´e et son utilisation en th´eorie des semigroupes.
In Ecole d’´et´e de Probabilit´es de Saint-Flour, no. 1581 in Lect. Notes in Math.
Springer, 1994.
[2] Bakry, D., and Emery, M. Diffusions hypercontractives. In S´em. Proba.
XIX, no. 1123 in Lect. Notes in Math. Springer, 1985, pp. 177–206.
[3] Blachman, N.M. The convolution inequality for entropy powers. IEEE
Trans. Inform. Theory 2 (1965), 267–271.
[4] Costa, M. A new entropy power inequality. IEEE Trans. Inform. Theory 31
(1985), 751–760.
[5] Dembo, A. A simple proof of the concavity of the entropy power with re-
spect to the variance of additive normal noise. IEEE Trans. Inform. Theory
35 (1989), 887–888.CONCAVITY OF ENTROPY POWER 5
[6] Dembo, A., Cover, T., and Thomas, J. Informationtheoreticinequalities.
IEEE Trans. Inform. Theory 37, 6 (1991), 1501–1518.
[7] McJKean, H.P. Jr. Speed of approach to equilibrium for Kac’s caricature
of a Maxwellian gas Arch. Rat. Mech. Anal. 21 (1966), 343–367.
[8] Toscani, G. Entropy production and the rate of convergence to equilibrium
for the Fokker-Planck equation. Quart. Appl. Math. 57, 3 (1999), 521–541.
DMA, Ecole normale suprieure, 45 rue d’Ulm, 75230 Paris Cedex 05.
e-mail [e-mail]