Honkela A. Solutions to Selected Problems on Independent Component Analysis
Подождите немного. Документ загружается.
Antti Honkela ICA: Exercise solutions
and thus
y
2
=
x
2
− m
21
x
1
1 − m
12
m
21
(127)
and similarly
y
1
= x
1
− m
12
x
2
− m
21
x
1
1 − m
12
m
21
=
x
1
− m
12
x
2
1 − m
12
m
21
. (128)
Thus the update rules become
∆m
12
= µ
µ
x
1
− m
12
x
2
1 − m
12
m
21
¶
3
x
2
− m
21
x
1
1 − m
12
m
21
(129)
∆m
21
= µ
µ
x
2
− m
21
x
1
1 − m
12
m
21
¶
3
x
1
− m
12
x
2
1 − m
12
m
21
. (130)
Problem 12.6
By Eq. (12.41):
∆W = µg(y)[z
T
− g(y
T
)W]. (131)
By Eq. (12.21):
∆V = µ(I − zz
T
)V. (132)
∆B = ∆WV + W∆V = µg(y)[z
T
− g(y
T
)W]V + µW(I − zz
T
)V
= µ[g(y)(z
T
W
t
)(WV) − g(y
T
)(WV) + WV − (Wz)(z
T
W
T
)(WV)]
= µ[g(y)y
T
B − g(y
T
)B + B − yy
T
B] = µ[g(y)y
T
− g(y
T
) + I − yy
T
]B
(133)
as claimed.
Problems for Chapter 13
Problem 13.1
The Fourier transform is a linear operation and thus
d
x(t) =
\
As(t) = A
d
s(t), (134)
i.e. the ICA model still holds for the Fourier transforms of the original independent components.
Problem 13.2
Theorem: If x(t) and s(t) follow the basic ICA model, then the innovation processes
˜
x(t) and
˜
s(t) follow the ICA model as well. In particular, the components ˜s
i
(t) are independent from each
other.
Proof: The independece of ˜s
i
(t) is easy to show:
˜s
i
(t) = s
i
(t) − E{s
i
(t)|s(t − 1), s(t − 2), . . . } = s
i
(t) − E{s
i
(t)|s
i
(t − 1), s
i
(t − 2), . . . } (135)
and these are independent by the independence of s
i
(t).
HUT, Neural Networks Research Centre 20
Antti Honkela ICA: Exercise solutions
It remains to show that
˜
x(t) = A
˜
s(t).
˜
x(t) = x(t) − E{x(t)|x(t − 1), x(t − 2), . . . }
= As(t) − E{As(t)|x(t − 1), x(t − 2), . . . }
= A(s(t) − E{s(t)|x(t − 1), x(t − 2), . . . })
= A(s(t) − E{s(t)|s(t − 1), s(t − 2), . . . }) = A
˜
s(t)
(136)
Here we have used the result
E{s(t)|x(t − 1), x(t − 2), . . . } = E{s(t)|s(t − 1), s(t − 2), . . . }. (137)
The intuitive justification for this is that as x = As and A is invertible, both s and x contain the
same information and thus the conditional expectations must be equal.
Formally this follows from
p(x(t − 1)) =
1
|det(A)|
p(s(t − 1)) (138)
p(s(t), x(t − 1)) =
1
|det(A)|
p(s(t), s(t − 1)) (139)
and thus
p(s(t)|s(t − 1)) =
p(s(t), s(t − 1))
p(s(t − 1))
=
p(s(t), x(t − 1))
p(x(t − 1))
= p(s(t)|x(t − 1)). (140)
¤
HUT, Neural Networks Research Centre 21