In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.
Let
(
X
,
A
)
,
(
Y
,
B
)
be measurable spaces. A Markov kernel with source
(
X
,
A
)
and target
(
Y
,
B
)
is a map
κ
:
X
×
B
→
[
0
,
1
]
with the following properties:
- The map
x
↦
κ
(
x
,
B
)
is
A
- measureable for every
B
∈
B
.
- The map
B
↦
κ
(
x
,
B
)
is a probability measure on
(
Y
,
B
)
for every
x
∈
X
.
(i.e. It associates to each point
x
∈
X
a probability measure
κ
(
x
,
.
)
on
(
Y
,
B
)
such that, for every measurable set
B
∈
B
, the map
x
↦
κ
(
x
,
B
)
is measurable with respect to the
σ
-algebra
A
.)
Simple random walk: Take
X
=
Y
=
Z
and
A
=
B
=
P
(
Z
)
, then the Markov kernel
κ
with
κ
(
x
,
B
)
=
1
2
1
B
(
x
−
1
)
+
1
2
1
B
(
x
+
1
)
,
∀
x
∈
Z
,
∀
B
∈
P
(
Z
)
,
describes the transition rule for the random walk on
Z
, where
1
is the indicator function.
Galton-Watson process: Take
X
=
Y
=
N
,
A
=
B
=
P
(
N
)
, then
κ
(
x
,
B
)
=
{
1
B
(
0
)
x
=
0
,
P
[
ξ
1
+
⋯
+
ξ
x
∈
B
]
else,
with i.i.d. random variables
ξ
i
.
General Markov processes with finite state space: Take
X
=
Y
,
A
=
B
=
P
(
X
)
=
P
(
Y
)
and
|
X
|
=
|
Y
|
=
n
, then the transition rule can be represented as a stochastic matrix
(
K
i
j
)
1
≤
i
,
j
≤
n
with
Σ
j
∈
Y
K
i
j
=
1
for every
i
∈
X
. In the convention of Markov kernels we write
κ
(
i
,
B
)
=
Σ
j
∈
B
K
i
j
,
∀
i
∈
X
,
∀
B
∈
B
.
Construction of a Markov kernel: If
ν
is a finite measure on
(
Y
,
B
)
and
k
:
X
×
Y
→
R
+
is a measurable function with respect to the product
σ
-algebra
A
⊗
B
and has the property
∫
X
k
(
x
,
y
)
ν
(
d
y
)
=
1
,
for all
x
∈
X
, then the mapping
κ
:
X
×
B
→
[
0
,
1
]
κ
(
x
,
B
)
=
∫
B
k
(
x
,
y
)
ν
(
d
y
)
,
defines a Markov kernel.
Let
(
X
,
A
,
P
)
be a probability space and
κ
a Markov kernel from
(
X
,
A
)
to some
(
Y
,
B
)
.
Then there exists a unique measure
Q
on
(
X
×
Y
,
A
⊗
B
)
, such that
Q
(
A
×
B
)
=
∫
A
κ
(
x
,
B
)
d
P
(
x
)
,
∀
A
∈
A
,
∀
B
∈
B
.
Let
(
S
,
Y
)
be a Borel space,
X
a
(
S
,
Y
)
- valued random variable on the measure space
(
Ω
,
F
,
P
)
and
G
⊆
F
a sub-
σ
-algebra.
Then there exists a Markov kernel
κ
from
(
Ω
,
G
)
to
(
S
,
Y
)
, such that
κ
(
.
,
B
)
is a version of the conditional expectation
E
[
1
{
X
∈
B
}
|
G
]
for every
B
∈
Y
, i.e.
P
[
X
∈
B
|
G
]
=
E
[
1
{
X
∈
B
}
|
G
]
=
κ
(
ω
,
B
)
,
P
−
a
.
s
.
∀
B
∈
G
.
It is called regular conditional distribution of
X
given
G
and is not uniquely defined.
