In statistics, kernel-independent component analysis (kernel ICA) is an efficient algorithm for independent component analysis which estimates source components by optimizing a generalized variance contrast function, which is based on representations in a reproducing kernel Hilbert space. Those contrast functions use the notion of mutual information as a measure of statistical independence.
Main idea
Kernel ICA is based on the idea that correlations between two random variables can be represented in a reproducing kernel Hilbert space (RKHS), denoted by
F
, associated with a feature map
L
x
:
F
↦
R
defined for a fixed
x
∈
R
. The
F
-correlation between two random variables
X
and
Y
is defined as
ρ
F
(
X
,
Y
)
=
max
f
,
g
∈
F
corr
(
⟨
L
X
,
f
⟩
,
⟨
L
Y
,
g
⟩
)
where the functions
f
,
g
:
R
→
R
range over
F
and
corr
(
⟨
L
X
,
f
⟩
,
⟨
L
Y
,
g
⟩
)
:=
cov
(
f
(
X
)
,
g
(
Y
)
)
var
(
f
(
X
)
)
1
/
2
var
(
g
(
Y
)
)
1
/
2
for fixed
f
,
g
∈
F
. Note that the reproducing property implies that
f
(
x
)
=
⟨
L
x
,
f
⟩
for fixed
x
∈
R
and
f
∈
F
. It follows then that the
F
-correlation between two independent random variables is zero.
This notion of
F
-correlations is used for defining contrast functions that are optimized in the Kernel ICA algorithm. Specifically, if
X
:=
(
x
i
j
)
∈
R
n
×
m
is a prewhitened data matrix, that is, the sample mean of each column is zero and the sample covariance of the rows is the
m
×
m
dimensional identity matrix, Kernel ICA estimates a
m
×
m
dimensional orthogonal matrix
A
so as to minimize finite-sample
F
-correlations between the columns of
S
:=
X
A
′
.
