OurBigBook.comSite Source code

Gibbs inequality

For probability distributions

p

and

q

on the same finite set,

D_{KL} (p ∥ q) = \sum_{i} p_{i} lo g \frac{p _{i}}{q _{i}} \geq 0,

with equality exactly when

p = q

on the support of

p

.

 Ancestors (5)

 Incoming links (1)

Solution