Initialization of Directions in Projection Pursuit Learning

The Projection Pursuit Learner is a multi-class classifier that resembles a two-layer neural network in which the sigmoid activation functions of the hidden neurons have been replaced by an interpolating polynomial. This modification increases the flexibility of the model but also makes it more inclined to get stuck in a local minimum during gradient-based training. This problem can be alleviated to a certain extent by replacing the random initialization of the parameters by proper heuristics. In this paper we propose to initialize the projection directions by means of feature space transformation methods such as independent component analysis (ICA), principal component analysis (PCA), linear discriminant analysis (LDA) and springy discriminant analysis (SDA). We find that with this refinement the number of processing units can be reduced by 10 - 40%.