Improving a basis function based classification method using
feature selection algorithms

The recently introduced Convex Networks (CN) method [] is a convex reformulation of several well-known machine learning algorithms like certain boosting methods and various Support Vector Machine algorithms. The special feature of the CN method is that it employs a combination of basis functions to solve a classification task of machine learning. The nonlinear Gauss-Seidel iteration process for solving the CN problem converges globally and fast, according to the corresponding proof. The most important property of the CN solution is its sparsity, which means that the number of basis functions with nonzero coefficients is small, and can effectively be controlled by heuristics. The proposed techniques were inspired by an area of artificial intelligence, called Feature Selection. Numerical results and comparisons demonstrate the effectiveness of the proposed methods on publicly available datasets. As will be shown, the CN approach can perform learning tasks using far fewer basis functions and generate sparse solutions.