Independent Component Analysis
Given a data set with large number of observations and large number of
dimensions.
Question: what is the best model that can transfer the model and errors
with the fewest bits.
To construct a mode with reduced size, we try to transform the data
set into a new space with fewer number of dimensions which capture as much
information as possible. In last class, we discussed the principal component
analysis which find a new set of orthogonal axes for the transformed space.
Independent component analysis is another approach.
ICA demonstration
(with FX data of German Mark and British Pound)
ICA characteristics
1) decouple the data into statistically independent components.
2) seek to determine the structure of the data set.
3) improve analysis the forecasting techniques.
ICA algorithm
ICA is based on the following data flow assumption:
1) The data are originated from the non normal, stationary, and statistically
independent sources: S.
2) These independent data sources are mixed in the real world, that
produces the observed data set: X, which is the data set we have.
3) Through a de-mixing process such as ICA, the observed data set X
can be decoupled into a set of statistically independent components Y.
4) The decoupled statistically independent data components Y, assumed,
reflect the real data sources S.
This data flow is depicted as follows:
S ==> |Mixing| ==> X
==> |De-mixing| ==> Y
Unlike PCA, which only make the covariance (the second centra moments)
among the different components of the decoupled data set equal zero. The
ICA ensures that the different components of the decoupled data set are
statistically independent, i.e. all of the higher order of moments among
the different components of the decoupled data set are equal to zero.
JADE algorithm for independent components analysis.
Implementations
Analysis of stock returns using ICA;
stock price data from Japanese companies
Analysis of independent components.
independent component which capture the 1987 market
crisis
how many independent components can capture the
major characteristics of the original data.