Kernel Methods

The Kernel-Idea

Theoretical developments generally have their own very different, unique histories before they find any practical application. One such example is the 'kernel-idea', which had appeared in several fields of mathematics and mathematical physics before it became a key notion in machine learning. The basic idea behind the kernel technique was originally introduced for pattern recognition by Aizerman and later was again employed in the general purpose Support Vector Machine, which was followed by a great number of novel kernel-based methods.

We can say that the kernel-idea is nothing more than the implicit transformation of a problem into a space probably more suitable for the solution. It has been proved in a number of studies that the extra computation needed for the transformation is beneficial when solving numerous machine learning problems.

Research Area

My research objectives in the field of Kernel Methods are:

- constructing Kernel feature extraction methods
- developing Kernel-based classification and regression methods
- developing Kernel-based non-linear visualisation methods
- solving large-scale problems with heuristic methods

Selected Papers on the Topic

MMDA  (Margin Maximizing Discriminant Analysis)

Kocsor, A., Kovács, K., Szepesvári, Cs.: Margin Maximizing Discriminant Analysis, J. F. Boulicaut et al. (Eds.), 15th European Conference on Machine Learning (ECML 2004), LNAI 3201, pp. 227-238, Springer Verlag, 2004. [pdf][abstract][bib]

PCA, Kernel-PCA, ICA, Kernel-ICA, LDA, Kernel-LDA, SDA, Kernel-SDA (A Unified View!)

Kocsor, A., Tóth, L.: Kernel-Based Feature Extraction with a Speech Technology Application, IEEE Transaction on Signal Processing, Vol. 52, No. 8, pp. 2250-2263, 2004. [pdf][abstract][bib]

Kocsor, A., Tóth, L.: Application of Kernel-Based Feature Space Transformations and Learning Methods to Phoneme Classification, Applied Intelligence, Vol. 21, No. 2, pp. 129-142, 2004. [pdf][abstract][bib

KERNEL-SDA (Kernel Springy Discriminat Analysis)

Kocsor, A., Kovács, K.: Kernel Springy Discriminant Analysis and Its Application to a Phonological Awareness Teaching System, in: P. Sojka, I. Kopecek, K. Pala (Eds.): Proceedings of Text, Speech and Dialogue: 5th International Conference, TSD 2002, LNAI 2448, pp. 325-328, Springer Verlag, 2002. [pdf][abstract][bib]

KERNEL-ICA (The first Kernel Independent Component Analysis)

Kocsor, A., Csirik, J.: Fast Independent Component Analysis in Kernel Feature Spaces, in: L. Pacholski and P. Ruzicka (Eds.): Proceedings of SOFSEM 2001: Theory and Practice of Informatics: 28th Conference on Current Trends in Theory and Practice of Informatics , LNCS 2234, pp. 271-281, Springer Verlag, 2001. [pdf][abstract][bib]

KERNEL-LDA (Kernel Linear Discriminant Analysis)

(This method is an extension of the Kernel Fischer's Discriminant Analysis and it is virtually equivalent to the Generalized Discriminant Analysis.)

Kocsor, A., Tóth, L., Paczolay, D.: A Nonlinearized Discriminant Analysis and its Application to Speech Impediment Therapy, in: V. Matousek, P. Mautner, R. Moucek, K. Tauser (Eds.): Proceedings of Text, Speech and Dialogue: 4th International Conference, TSD 2001, LNAI 2166, pp. 249-257, Springer Verlag, 2001. [pdf][abstract][bib]

KERNEL-PCA (An application to Kernel Principal Component Analysis.)

Kocsor, A., Kuba, A. Jr., Tóth, L.: Phoneme Classification Using Kernel Principal Component Analysis, Periodica Polytechnica, Electrical Engineering, Vol. 44, No. 1, pp. 77-90, 2000. [pdf][abstract][bib]

 

 

 
/map>