Metric entropy

The metric entropy , or entropy of Kolmogorov (also says itself in English measure-theoretic entropy ) is a tool developed by Kolmogorov about the middle of the Années 1950 resulting from the probabilistic concept of entropy of the Information theory of Shannon. Kolmogorov showed how the metric entropy can be used to show if two dynamic systems are not not combined. It is a fundamental invariant of the systems dynamic measured. Moreover, the metric entropy allows one qualitative definition of the chaos: a chaotic transformation can to be seen like a transformation of nonnull entropy.

First of all let us present the mathematical framework in which one place. $(X, \ mathfrak {M}, \ driven)$ is a Espace of probability, and $f$

X \ to X is a measurable application, which represents the law

of evolution of a dynamic system at discrete times on the space of phases X . One forces F to preserve measurement, i.e.

\ forall M \ in \ mathfrak {M}, \ driven (f^ {- 1} (M)) = \ driven (M)

. On the basis of one initial state X , one can define the continuation as of its reiterated by F :

x, F (X), \ dowries, f^n (X), \ dots

the unit

\ {f^n (X): N \ geq 0 \}

states by which the system passes calls the orbit of X .

Construction of the metric entropy

If one gives oneself a finished partition α of X made up measurable units $\ alpha = \ {A_1, \ dowries, A_p \}$ and a state initial X , the states $f^n (X)$ ( $n \ geq 0$ ) by which the system master key fall each one in one from the parts of the partition α. The continuation of these parts provides information on the initial state X . The entropy corresponds to the average quantity of information brought by an iteration. The construction of the metric entropy is a process which proceeds in three stages, that we will clarify below. In a first time, one defines the entropy $\ mathcal {H} (\ alpha)$ of a partition α (average information resulting from the knowledge of the part of α in which a point of X is). Then, one the entropy $h (F, \ alpha) defines$ of the transformation F relative with the partition α (average information brought by an iteration). Lastly, the metric entropy H (F) is the terminal higher of the entropies of F relative than the partitions of X .

Entropy of a partition

That is to say α a finished partition of X in measurable units. A point $x \ in X$ is of as much better localized than it is located in a part $A \ in \ alpha$ of weak measurement $\ driven (A)$ . This justifies the introduction of the function information $I (\ alpha): X \ to; +
\ infty]$ defined by:

$\ forall X \ in X, I (\ alpha) (X) = - \ sum_ {has \ in \ alpha} \ log \ driven (A) \ chi_A (X)$

i.e. $I (\ alpha) (X) = - \ log \ driven (A)$ if $x \ in A$ .

The entropy of the partition α is the average of $I (\ alpha)$ :

$\ mathcal {H} (\ alpha) = \ frac {1} {\ driven (X)} \ int_X I (\ alpha) (X) D \ driven (X) = - \ sum_ {has \ in$

\ alpha} \ driven (A) \ log \ driven (A)

One takes $0 \ log 0$ equal to 0. If α and β are two measurable partitions of X , one defines the joint of α and β, $\ alpha \ vee \ beta$ the smallest partition finer than α and β: $\ alpha \ vee \ beta = \ {has \ course b: has \ in
\ alpha, B \ in \ beta, has \ course B \ neq \ emptyset \}$ . It is said that β is finer than α, and one notes $\ beta \ geq \ alpha$ if all element of has α is written like union of elements of β.

The entropy of a partition checks the intuitive properties following:

If α and β are two measurable partitions, then $\ mathcal {H} (\ alpha \ vee \ beta) \ Leq \ mathcal {H} (\ alpha) + \ mathcal {H} (\ beta)$ .
Let us note $f^ {- 1} (\ alpha) = \ {f^ {- 1} (A): With \ in \ alpha \}$ . One a: $\ mathcal {H} (\ alpha) = \ mathcal {H} (f^ {- 1} (\ alpha))$ .

The first property means that the information brought by simultaneous knowledge of the positions of the states of the system relative with two partitions is lower than the sum of information brought relative to each partition. second property comes owing to the fact that F preserves measurement.

Entropy of a transformation relative to a partition

α is a measurable partition. The entropy is defined $h (F, \ alpha)$ of the transformation F relative with α by:

$h (F, \ alpha) = \ lim_ {N \ to + \ infty} \ frac {1} {N} \ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n-1} f^ {- I} (\ alpha) \ Bigg)$

One can see the transformation F like the one day passage to the following at the time of one experiment. At time zero, one does not manage to distinguish all them states, one gathers the nondistinguishable states per packages, one forms in this manner a partition α. $\bigvee_{i=0}^{n-1}
f^ {- I} (\ alpha)$ thus represents all the possible results with boils of N days. $h (F, \ alpha)$ is thus average information daily that one obtains by carrying out the experiment.

The definite limit exists well. If one notes $a_n = \ mathcal {H} \ Bigg (
\ bigvee_ {i=0} ^ {n-1} f^ {- I} (\ alpha) \ Bigg)$ , then the continuation $(a_n) _ {N
\ in \ N^*}$ is under-additive bus:

$a_ {n+p} = \ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n+p-1} f^ {- I} (\ alpha) \ Bigg) \ Leq \ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n-1} f^ {- I} (\ alpha) \ Bigg) + \ mathcal {H} \ Bigg (\ bigvee_ {i=n} ^ {n+p-1} f^ {- I} (\ alpha) \ Bigg) \ Leq a_n + a_p$

One respectively used the two properties of the preceding section. $(a_n/n) _ {N \ in \ N^*}$ thus admits a limit.

Last stage: metric entropy of a transformation

The metric entropy of F , noted H (F) is the upper limit of entropies of F relative to the measurable partitions finished of X

h (F) = \ sup_ {\ alpha} H (F, \ alpha)

H (F) is possibly infinite.

Dynamic examples of systems and calculation of entropy

The calculation of the metric entropy is facilitated when the terminal higher is reached, i.e when there exists a partition α such as the metric entropy and the entropy relative with α is confused. As example, let us treat the case of the identity application of X . Then,

$H (id, \ alpha) = \ frac {1} {N} \ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n-1} id^ {- I} (\ alpha) \ Bigg) = \ frac {1} {N} \ mathcal {H} (\ alpha) \ longrightarrow_ {N \ to + \ infty} 0$

The identity has a null entropy, which is foreseeable because of its not very chaotic character.

In much of less commonplace cases, the following theorem, of The Kolmogorov-Sinai, is one of the most practical tools for to calculate an entropy, because it avoids taking the upper limit on all the measurable partitions of X .

If α is a measurable partition of X such as the continuation $\ Big (\ bigvee_ {i=0} ^ {N} f^ {- I} (\ alpha) \ Big) _ {N \ in \ NR}$ generates tribe $\ mathfrak {M}$ , or if F is invertible ( F ^-1 is measurable and preserves measurement) and the continuation $\ Big (\ bigvee_ {i=-n} ^ {N} f^ {- I} (\ alpha) \ Big) _ {N \ in \ NR}$ generates tribe $\ mathfrak {M}$ then one says that α is generating.

The theorem of the Kolmogorov-Sinai affirms that if α is generator, then $h (F) = H (F, \ alpha)$ .

Rotations of the circle

$\ mathbb {U} = \ R/\ Z$ is the circle unit, provided with the measurement of angle dθ. Let us analyze the effect of a rotation

$f : X \ mapsto has + Z \ MOD 1$

when $a = p/q$ is rational. That is to say α one partition:

$h (F, \ alpha) = \ lim_ {N \ to + \ infty} \ frac {1} {qn}$

\ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {qn-1} f^ {- I} (\ alpha) \ Bigg) = \ lim_ {N \ to + \ infty} \ frac {1} {qn} \ Bigg (\ bigvee_ {i=0} ^ {q-1} f^ {- I} (\ alpha) \ Bigg) = 0

If has is irrational, it is also shown that the metric entropy of F is null.

Doubly angles

Always on the circle unit, one takes this time the application

$f : X \ mapsto 2x \ MOD 1$

who doubles the angles. The same one is considered partition

$\ alpha = \ Big \ {\ Big \ displaystyle \ frac {1} {2} \ Big [,
\ Big 1 \ Big [\ Big \}$

It is observed that:

$\ alpha \ vee f^ {- 1} (\ alpha) = \ Big \ {\ Big \ frac {1} {4} \ Big [,
\ Big \ frac {1} {2} \ Big [, \ Big [\ frac {1} {2},
\ frac {3} {4} \ Big \ Big [\ frac {3} {4}, 1 \ Big [\ Big \}$

Then by recurrence, one deduces more generally than:

$\bigvee_{i=0}^{n-1}$

f^ {- I} (\ alpha) = \ Big \ {\ Big: 0 \ Leq I \ Leq 2^n -1 \ Big \}

Like the whole of the type \ Big generate the tribe $\ mathfrak {M}$ , the theorem of the Kolmogorov-Sinai show that $h (F) = H (F, \ alpha)$ and:

$\ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n-1} f^ {- I} (\ alpha) \ Bigg) = -$

\ sum_ {i=0} ^ {2^n - 1} \ driven \ Bigg (\ Big \ log \ driven \ Bigg (\ Big = N \ log 2

The metric entropy of F is thus log 2.

Shift of Bernoulli

One has an alphabet finished $\ Lambda = \ {1, \ dowries, K \}$ . That is to say $(p_i) _ {1 \ Leq I \ Leq K}$ of the strictly positive numbers of nap 1. One assigns with each letter I the probability $m (\ {I \})$

p_i of appearance. $(\ Lambda, 2^ \ Lambda,$

m) is a space of probability. The space of the words is introduced infinite

(\ Lambda^ \ Z, \ mathfrak {M}, \ driven) =
\ displaystyle \ prod_ {- \ infty} ^ {+ \ infty} (\ Lambda, 2^ \ Lambda, m)

. One the application shift σ by

\ sigma (X) defines _n = x_ {n+1}

for

n \ in \ Z

(\ Lambda^ \ Z, \ mathfrak {M}, \ driven, \ sigma)

is one invertible dynamic system. One partitionne

\ Lambda^ \ Z

\ alpha

\ {P_i \} _ {1 \ Leq I \ Leq K} where $P_i$ is the whole of the words

(x_n) _ {N \ in \ Z}

such as

x_0 = i

\bigvee_{i=-n}^n
f^ {- I} (\ alpha)

is the partition by the cylinders

\ mathcal {C} _ {\ lambda_ {- N}, \ dowries, \ lambda_n} = \ {(x_n) \ in
\ Lambda^ \ Z: x_i = \ lambda_i, - N \ Leq I \ Leq N \}

. The whole of these cylinders generate the tribe of

\ Lambda^ \ Z

and the theorem of The Kolmogorov-Sinai applies. One calculates then easily:

$\ mathcal {H} \ Bigg (\ bigvee_ {i=0} ^ {n-1} \ sigma^ {- I} (\ alpha) \ Bigg) = - \ sum_ {(\ lambda_0, \ dowries, \ lambda_ {n-1}) \ in$

\ Lambda^n} p_ {\ lambda_0} p_ {\ lambda_1} \ cdots p_ {\ lambda_ {n-1}} \ log (p_ {\ lambda_0} p_ {\ lambda_1} \ cdots p_ {\ lambda_ {n-1}}) = - N \ sum_ {\ lambda \ in \ Lambda} p_ \ lambda \ log p_ \ lambda

Thus $h (\ sigma) = H (\ sigma, \ alpha) = - \ displaystyle \ sum_ {\ lambda \ in
\ Lambda} p_ \ lambda \ log p_ \ lambda$ .

See too

Random links: