Originally published 7 September 2002 Preceded by Probability Sun | ||
![]() | ||
Genres Fiction, Speculative fiction Awards John W. Campbell Memorial Award for Best Science Fiction Novel Nominations Locus Award for Best Science Fiction Novel Similar Works by Nancy Kress, Other books |
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process (or “experiment”) consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind. One proposes that each time a situation of that kind arises, the set of possible outcomes is the same and the probabilities are also the same.
Contents
- Introduction
- Definition
- Discrete case
- General case
- Non atomic case
- Complete probability space
- Example 1
- Example 2
- Example 3
- Example 4
- Example 5
- Probability distribution
- Random variables
- Defining the events in terms of the sample space
- Conditional probability
- Independence
- Mutual exclusivity
- References
A probability space consists of three parts:
- A sample space,
Ω , which is the set of all possible outcomes. - A set of events
F , where each event is a set containing zero or more outcomes. - The assignment of probabilities to the events; that is, a function
P from events to probabilities.
An outcome is the result of a single execution of the model. Since individual outcomes might be of little practical use, more complex events are used to characterize groups of outcomes. The collection of all such events is a σ-algebra
Once the probability space is established, it is assumed that “nature” makes its move and selects a single outcome,
The Russian mathematician Andrey Kolmogorov introduced the notion of probability space, together with other axioms of probability, in the 1930s. Nowadays alternative approaches for axiomatization of probability theory exist; see “Algebra of random variables”, for example.
This article is concerned with the mathematics of manipulating probabilities. The article probability interpretations outlines several alternative views of what “probability” means and how it should be interpreted. In addition, there have been attempts to construct theories for quantities that are notionally similar to probabilities but do not obey all their rules; see, for example, Free probability, Fuzzy logic, Possibility theory, Negative probability and Quantum probability.
Introduction
A probability space is a mathematical triplet
Not every subset of the sample space
Definition
In short, a probability space is a measure space such that the measure of the whole space is equal to one.
The expanded definition is the following: a probability space is a triple
Discrete case
Discrete probability theory needs only at most countable sample spaces
The case
General case
If Ω is uncountable, still, it may happen that p(ω) ≠ 0 for some ω; such ω are called atoms. They are an at most countable (maybe empty) set, whose probability is the sum of probabilities of all atoms. If this sum is equal to 1 then all other points can safely be excluded from the sample space, returning us to the discrete case. Otherwise, if the sum of probabilities of all atoms is between 0 and 1, then the probability space decomposes into a discrete (atomic) part (maybe empty) and a non-atomic part.
Non-atomic case
If p(ω) = 0 for all ω∈Ω (in this case, Ω must be uncountable, because otherwise P(Ω)=1 could not be satisfied), then equation (∗) fails: the probability of a set is not the sum over its elements, as summation is only defined for countable amount of elements. This makes the probability space theory much more technical. A formulation stronger than summation, measure theory is applicable. Initially the probabilities are ascribed to some “generator” sets (see the examples). Then a limiting procedure allows assigning probabilities to sets that are limits of sequences of generator sets, or limits of limits, and so on. All these sets are the σ-algebra
Complete probability space
A probability space
Example 1
If the experiment consists of just one flip of a fair coin, then the outcome is either heads or tails:
Example 2
The fair coin is tossed three times. There are 8 possible outcomes: Ω = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT} (here “HTH” for example means that first time the coin landed heads, the second time tails, and the last time heads again). The complete information is described by the σ-algebra
Alice knows the outcome of the second toss only. Thus her incomplete information is described by the partition Ω = A1 ⊔ A2 = {HHH, HHT, THH, THT} ⊔ {HTH, HTT, TTH, TTT}, where ⊔ is the disjoint union, and the corresponding σ-algebra
The two σ-algebras are incomparable: neither
Example 3
If 100 voters are to be drawn randomly from among all voters in California and asked whom they will vote for governor, then the set of all sequences of 100 Californian voters would be the sample space Ω. We assume that sampling without replacement is used: only sequences of 100 different voters are allowed. For simplicity an ordered sample is considered, that is a sequence {Alice, Bryan} is different from {Bryan, Alice}. We also take for granted that each potential voter knows exactly his/her future choice, that is he/she doesn’t choose randomly.
Alice knows only whether or not Arnold Schwarzenegger has received at least 60 votes. Her incomplete information is described by the σ-algebra
Bryan knows the exact number of voters who are going to vote for Schwarzenegger. His incomplete information is described by the corresponding partition Ω = B0 ⊔ B1 ... ⊔ B100 and the σ-algebra
In this case Alice’s σ-algebra is a subset of Bryan’s:
Example 4
A number between 0 and 1 is chosen at random, uniformly. Here Ω = [0,1],
In this case the open intervals of the form (a,b), where 0 < a < b < 1, could be taken as the generator sets. Each such set can be ascribed the probability of P((a,b)) = (b − a), which generates the Lebesgue measure on [0,1], and the Borel σ-algebra on Ω.
Example 5
A fair coin is tossed endlessly. Here one can take Ω = {0,1}∞, the set of all infinite sequences of numbers 0 and 1. Cylinder sets {(x1, x2, ...) ∈ Ω : x1 = a1, ..., xn = an} may be used as the generator sets. Each such set describes an event in which the first n tosses have resulted in a fixed sequence (a1, ..., an), and the rest of the sequence may be arbitrary. Each such event can be naturally given the probability of 2−n.
These two non-atomic examples are closely related: a sequence (x1,x2,...) ∈ {0,1}∞ leads to the number 2−1x1 + 2−2x2 + ... ∈ [0,1]. This is not a one-to-one correspondence between {0,1}∞ and [0,1] however: it is an isomorphism modulo zero, which allows for treating the two probability spaces as two forms of the same probability space. In fact, all non-pathologic non-atomic probability spaces are the same in this sense. They are so-called standard probability spaces. Basic applications of probability spaces are insensitive to standardness. However, non-discrete conditioning is easy and natural on standard probability spaces, otherwise it becomes obscure.
Probability distribution
Any probability distribution defines a probability measure.
Random variables
A random variable X is a measurable function X: Ω → S from the sample space Ω to another measurable space S called the state space.
If A ⊂ S, the notation Pr(X ∈ A) is a commonly used shorthand for P({ω ∈ Ω: X(ω) ∈ A}).
Defining the events in terms of the sample space
If Ω is countable we almost always define
On the other hand, if Ω is uncountable and we use
Conditional probability
Kolmogorov’s definition of probability spaces gives rise to the natural concept of conditional probability. Every set A with non-zero probability (that is, P(A) > 0) defines another probability measure
on the space. This is usually pronounced as the “probability of B given A”.
For any event B such that P(B) > 0 the function Q defined by Q(A) = P(A|B) for all events A is itself a probability measure.
Independence
Two events, A and B are said to be independent if P(A∩B)=P(A)P(B).
Two random variables, X and Y, are said to be independent if any event defined in terms of X is independent of any event defined in terms of Y. Formally, they generate independent σ-algebras, where two σ-algebras G and H, which are subsets of F are said to be independent if any element of G is independent of any element of H.
Mutual exclusivity
Two events, A and B are said to be mutually exclusive or disjoint if P(A∩B) = 0. (This is weaker than A∩B = ∅, which is the definition of disjoint for sets).
If A and B are disjoint events, then P(A∪B) = P(A) + P(B). This extends to a (finite or countably infinite) sequence of events. However, the probability of the union of an uncountable set of events is not the sum of their probabilities. For example, if Z is a normally distributed random variable, then P(Z=x) is 0 for any x, but P(Z∈R) = 1.
The event A∩B is referred to as “A and B”, and the event A∪B as “A or B”.