Independence (probabilities)

The independence is a probabilistic concept qualifying in an intuitive way of random events not having any influence one on the other. It is thus about a crucial notion of the probability theory.

For example, the value of a first throw of dice does not have any influence on the value of the second throw. In the same way, for a throw makes it obtain a lower value or equal to four does not influence of anything the Probabilité that the result is even : the two events are known as independent .

The independence or not of two event is not always easy to establish.

Concept of independence

Two events has and B is independent if and only if the realization of the one do not influence that of the other. Into mathematical term, that is translated as follows:

Two events has and B is independent if and only if they check

$Pr\; (has\; \backslash \; course\; B)\; =\; Pr\; (A)\; \backslash \; cdot\; Pr\; (B)$

The concept can be wide with N events: N events is known as independent if and only if any combination of these events unspecified number is independent. This concept is stronger than N independent events two to two.

It corresponds to it the full number of conditions according to

$C\_\; \{N\}\; ^\; \{2\}\; +\; C\_\; \{N\}\; ^\; \{3\}\; +\dots \; +\; C\_\; \{N\}\; ^\; \{N\}\; =\; 2^\; \{N\}\; -\; (n+1)$.

If they all are checked, one can then write

$\backslash \; Pr\; (A\_1\; \backslash \; course\; \backslash \; cdots\; \backslash \; course\; A\_n)\; =\; \backslash \; Pr\; (A\_1)\; \backslash ,\; \backslash \; cdots\; \backslash ,\; \backslash \; Pr\; (A\_n).$

Independent events

The preceding definition refers to the independent events. In this case, one can also add that the Conditional probability of has being given B is the same one as the probability of has .

$\backslash \; Pr\; (has\; \backslash \; mid\; B)=\; \backslash \; Pr\; (A).\; \backslash ,$

There exist two good reasons not to take this criterion like definition of independence.

• the two events has and B does not play a symmetrical part there (one cannot invert has and B without calling in question the equality).

• the conditional probability excludes that B is of null probability. Indeed,

$\backslash \; Pr\; (has\; \backslash \; mid\; B)=\; \{\backslash \; Pr\; (has\; \backslash \; course\; B)\; \backslash \; over\; \backslash \; Pr\; (B)\},$ (with Pr ( B ) ≠ 0)

Naturally, we find our first definition

$\backslash \; Pr\; (has\; \backslash \; course\; B)=\; \backslash \; Pr\; (A)\; \backslash \; Pr\; (B)$

To finish, let us add that, contrary to the common direction, an event can be independent compared to itself, if it has a probability of 1 or 0, as the definition shows it

$\backslash \; Pr\; (A)\; =\; \backslash \; Pr\; (has\; \backslash \; course\; A)\; =\; \backslash \; Pr\; (A)\; \backslash \; Pr\; (A)\; \backslash ,$

Independent random variables

What precedes defines in way the most general possible independence. Now, we will consider the case of the random variable .

That is to say y₁ and y₂ two random variables (scalar for simplicity). Independence can then be defined via the functions of density of probabilities.

• Is p (y₁, y₂) the function of density of probability (fdp) joint of y₁ and y₂,

• p₁ (y₁) the marginal fdp of y₁, i.e.

$p\_\; \{1\}\; (y\_\; \{1\})\; =\; \backslash \; int\; p\; (y\_\; \{1\},\; y\_\; \{2\})\; \backslash ,\; dy\_\; \{2\}$

• and of the same for p₂ (y₂)

Then y₁ and y₂ are independent if and only if the joint fdp is factorisable as follows

$p\; (y\_\; \{1\},\; y\_\; \{2\})\; =\; p\_\; \{1\}\; (y\_\; \{1\})\; p\_\; \{2\}\; (y\_\; \{2\})$

This definition extends naturally for variable N , and the joint density will be then a product of N terms.

Criterion of independence

The foregoing definition can be used to derive an important consequence from independence.

That is to say two independent random variables y₁ and y₂, and two functions h₁ and h₂, one can always write

$E\; \backslash \; \{h\_\; \{1\}\; y\_\; \{1\}\; h\_\; \{2\}\; y\_\; \{2\}\; \backslash \}\; =\; E\; \backslash \; \{h\_\; \{1\}\; y\_\; \{1\}\; \backslash \}\; E\; \backslash \; \{h\_\; \{2\}\; y\_\; \{2\}\; \backslash \}$.

Indeed, independence makes it possible to carry out the following operations

$E\; \backslash \; \{h\_\; \{1\}\; y\_\; \{1\}\; h\_\; \{2\}\; y\_\; \{2\}\; \backslash \}$

$=\; \backslash \; int\; \backslash \; int\; h\_\; \{1\}\; (y\_\; \{1\})\; h\_\; \{2\}\; (y\_\; \{2\})\; p\; (y\_\; \{1\},\; y\_\; \{2\})\; \backslash ,\; dy\_\; \{1\}\; \backslash ,\; dy\_\; \{2\}$

$=\; \backslash \; int\; \backslash \; int\; h\_\; \{1\}\; (y\_\; \{1\})\; p\_\; \{1\}\; (y\_\; \{1\})\; h\_\; \{2\}\; (y\_\; \{2\})\; p\_\; \{2\}\; (y\_\; \{2\})\; \backslash ,\; dy\_\; \{1\}\; \backslash ,\; dy\_\; \{2\}$

$=\; \backslash \; int\; h\_\; \{1\}\; (y\_\; \{1\})\; p\_\; \{1\}\; (y\_\; \{1\})\; \backslash ,\; dy\_\; \{1\}\; \backslash \; int\; h\_\; \{2\}\; (y\_\; \{2\})\; p\_\; \{2\}\; (y\_\; \{2\})\; \backslash ,\; dy\_\; \{2\}$

$=\; E\; \backslash \; \{h\_\; \{1\}\; y\_\; \{1\}\; \backslash \}\; E\; \backslash \; \{h\_\; \{2\}\; y\_\; \{2\}\; \backslash \}$

Independence and correlation

Nonthe Corrélation is a property weaker than independence. Indeed, in the case of two independent random variables, one can always write (while basing itself on the preceding criterion) that

$E\; \backslash \; \{y\_\; \{1\}\; y\_\; \{2\}\; \backslash \}\; -\; E\; \backslash \; \{y\_\; \{1\}\; \backslash \}\; E\; \backslash \; \{y\_\; \{2\}\; \backslash \}\; =\; 0$.

But the reverse is not inevitably true. That is to say a distribution (y₁, y₂) of probability 1/4 with for values (0,1), (0, - 1), (1,0) and (- 1,0). If the two variables are not correlated, they are however not independent, since

$E\; \backslash \; \{y\_\; \{1\}\; ^\; \{2\}\; y\_\; \{2\}\; ^\; \{2\}\; \backslash \}\; =\; 0\; \backslash \; 1/4\; =\; E\; \backslash \; \{y\_\; \{1\}\; ^\; \{2\}\; \backslash \}\; E\; \backslash \; \{y\_\; \{2\}\; ^\; \{2\}\; \backslash \}$.

And thus the criterion of independence is not respected.

Independence and information

Another way of apprehending this concept of independence between two events is to pass by the information (within the meaning of the Information theory): two events are independent if the furnished information by the first event does not give any information on the second event.

That is to say to draw two balls (red and white) from a ballot box. If one carries out the experiment without giving the drawn ball in the ballot box, and that the first drawn ball is red, one can deduce from this information which the second drawn ball will be white. The two events are thus not independent.

On the other hand, if one gives the first ball in the ballot box before the second pulling, the information of the first event (the ball is red) does not give us any information on the color of the second ball. The two events are thus independent.

This approach is in particular used in Analyze in independent components.

Applications and uses

• Lemma of Borel-Cantelli

• Law of the 0-1
• Number universe
• Law of the great numbers
• Theorem of the central limit
• random Walk

• Analysis in independent components

Random links:

List schools of animation | Jean Valton | Grêmio Esportivo Jaciara | Howard Gordon | Olida | Chaîne_électronique_de_tactique_militaire

© 2007-2008 speedlook.com; article text available under the terms of GFDL, from fr.wikipedia.org