User:Jclaer/Jensen

Inequality measurement

Inequality within a population has many definitions. Diverse people would like to manipulate it.

An economist might measure people's income inequality and propose policies to change it. But how should inequality be measured?

A well log analyst visualizes the earth as layers of distinct composition. At any depth there should be no reflectivity except where there is a change from one material to another. Instrumental parameters are adjusted to achieve a population of many small values (interior to a rock type) and a few large ones (at rock transitions).

Physicists and chemists seek to characterize equilibrium within an isolated volume. The material in the volume is viewed as a population. The presence of energy causes fluctuations. With time these fluctuations are distributed more uniformly as inequality within the population is minimized. The concept of entropy is associated (negatively) with inequality within the population. Maximizing entropy is associated with driving the population towards uniformity and the volume to equilibrium.

Examples of averages

Start with two positive numbers, x and y, and with two kinds of average, the arithmetic mean A and the geometric mean G.

$A={\frac {x+y}{2}}$

$G={\sqrt {xy}}.$

If the values x and y are equal, then the two means A and G are equal. Otherwise G ≤ A. In other words, the product of square roots of two values is smaller than half the sum of the values. We could define inequality by the ratio 0 ≤ G/A ≤ 1, but that's only one of very many possible definitions. There are many more ways to define averages and many more ratios to chose from. Here are some averages for positive numbers:

$H={\frac {n}{{\frac {1}{a_{1}}}+{\frac {1}{a_{2}}}+...+{\frac {1}{a_{n}}}}}$  (harmonic mean),

$G={\sqrt[{n}]{a_{1}\cdot a_{2}\cdot ...\cdot a_{n}}}$  (geometric mean),

$A={\frac {a_{1}+a_{2}+...+a_{n}}{n}}$  (arithmetic mean),

$Q={\sqrt {\frac {a_{1}^{2}+a_{2}^{2}+...+a_{n}^{2}}{n}}}$  (quadratic mean)

The harmonic mean H is a good way to average velocities.

These various means have inequalities among them

$H\leq G\leq A\leq Q$

Population inequality could be measured by the ratio of any mean to any larger one. But wait! There are still more possibilities.

An inequality can be based on any convex function

Let f be a function with a positive second derivative. Such a function is called "convex" and satisfies the inequality

${\frac {f(a)+f(b)}{2}}-f\left({\frac {a+b}{2}}\right)\quad \geq \quad 0$

This equation relates a function of an average to an average of the function. The average can be weighted, for example,

${{\frac {1}{3}}\,f(a)\ +\ {\frac {2}{3}}\,f(b)}\ -\ f\left({{\frac {1}{3}}a+{\frac {2}{3}}b}\right)\quad \geq \quad 0$

This is made clear by a graphical illustration (for the convex function $f=x^{2}$).

There is nothing special about $f=x^{2}$, except that it is convex. Given three numbers a, b, and c, the inequality can first be applied to a and b; to the sum c can be included. Thus, recursively, an inequality for two numbers can build up a weighted average of three or more numbers. Define weights $w_{j}\geq 0$ that are normalized $(\sum w_{j}=1)$. The result is known as the Jensen inequality.

$S(p_{j})=\sum _{j=1}^{N}w_{j}f(p_{j})\ -\ f\left(\sum _{j=1}^{N}w_{j}p_{j}\right)\quad \geq \quad 0$

If the $p_{j}$ are all identical, then $p_{j}$ comes out of the summations; the two parts of S are the same, and S vanishes. If you had a procedure to drive S to zero, then you would be driving all the $p_{j}$ to be identical. The populist politician claims to intend to drive S to zero, but how should he define S?

Examples of Jensen inequalities

The most familiar example of a Jensen inequality occurs when the weights are all equal to 1/N and the convex function is $f(x)=x^{2}$. In this case the Jensen inequality gives the familiar result that the mean square exceeds the square of the mean:

$Q={1 \over N}\sum _{i=1}^{N}x_{i}^{2}\ -\ \left({1 \over N}\sum _{i=1}^{N}x_{i}\right)^{2}\quad \geq \quad 0$

When the population consists of positive members p_i the function f(p) need have a positive second derivative only for positive values of p. The function f(p)=1/p yields a Jensen inequality for the harmonic mean:

$H=\sum {w_{i} \over p_{i}}\ -\ {1 \over \sum w_{i}p_{i}}\quad \geq \quad 0$

A more important case is the geometric inequality. Here $f(p)=-\ln(p)$, and

$G=-\sum w_{i}\ln p_{i}\ +\ \ln \sum w_{i}p_{i}\quad \geq \quad 0$

The more familiar form of the geometric inequality results from exponentiation and a choice of weights equal to 1/N:

${1 \over N}\sum _{i=1}^{N}p_{i}\quad \geq \quad \prod _{i=1}^{N}p_{i}^{1/N}$

The basic equation of statistical mechanics

An important inequality in statistical mechanics, information theory, and thermodynamics is the one based on

$f(p)=p^{1+\epsilon }$,

where $\epsilon$ is a small positive number tending to zero. This is a convex function, but just barely so.

Some algebraic steps lead to

${\sum w_{i}p_{i}\ln p_{i} \over \sum w_{i}p_{i}}\quad \geq \quad \ln \sum w_{i}p_{i}$

We can now define a positive variable S' with or without the positive scaling factor $\sum wp$:

$S'_{\rm {intensive}}={\sum w_{i}p_{i}\ln p_{i} \over \sum w_{i}p_{i}}\ -\ \ln \sum w_{i}p_{i}\quad \geq \quad 0$

$S'_{\rm {extensive}}=\sum w_{i}p_{i}\ln p_{i}\ -\ \left(\sum w_{i}p_{i}\right)\ \ln \left(\sum w_{i}p_{i}\right)\quad \geq \quad 0$

These equations resemble those of Relative entropy.

Here find additional speculations on what convex function should be used in data analysis.