NDLI: Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis

Content Provider	Springer Nature Link
Author	Böhmer, Wendelin Grünewälder, Steffen Nickisch, Hannes Obermayer, Klaus
Copyright Year	2012
Abstract	Without non-linear basis functions many problems can not be solved by linear algorithms. This article proposes a method to automatically construct such basis functions with slow feature analysis (SFA). Non-linear optimization of this unsupervised learning method generates an orthogonal basis on the unknown latent space for a given time series. In contrast to methods like PCA, SFA is thus well suited for techniques that make direct use of the latent space. Real-world time series can be complex, and current SFA algorithms are either not powerful enough or tend to over-fit. We make use of the kernel trick in combination with sparsification to develop a kernelized SFA algorithm which provides a powerful function class for large data sets. Sparsity is achieved by a novel matching pursuit approach that can be applied to other tasks as well. For small data sets, however, the kernel SFA approach leads to over-fitting and numerical instabilities. To enforce a stable solution, we introduce regularization to the SFA objective. We hypothesize that our algorithm generates a feature space that resembles a Fourier basis in the unknown space of latent variables underlying a given real-world time series. We evaluate this hypothesis at the example of a vowel classification task in comparison to sparse kernel PCA. Our results show excellent classification accuracy and demonstrate the superiority of kernel SFA over kernel PCA in encoding latent variables.
Starting Page	67
Ending Page	86
Page Count	20
File Format	PDF
ISSN	08856125
Journal	Machine Learning
Volume Number	89
Issue Number	1-2
e-ISSN	15730565
Language	English
Publisher	Springer US
Publisher Date	2012-06-13
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Time series Latent variables Unsupervised learning Slow feature analysis Sparse kernel methods Linear classification Language Translation and Linguistics Control, Robotics, Mechatronics Computing Methodologies Artificial Intelligence (incl. Robotics) Simulation and Modeling
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Unsupervised feature selection with ensemble learning

Regularized feature selection in reinforcement learning

Online Multiple Kernel Classification

Transfer bounds for linear feature learning

Sparse conjugate directions pursuit with application to fixed-size kernel models

Sparse canonical correlation analysis

Sparse kernel SVMs via cutting-plane training

Multi-view kernel construction

Block coordinate descent algorithms for large-scale sparse multiclass classification

Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis

Similar Documents

Unsupervised feature selection with ensemble learning

Regularized feature selection in reinforcement learning

Online Multiple Kernel Classification

Transfer bounds for linear feature learning

Sparse conjugate directions pursuit with application to fixed-size kernel models

Sparse canonical correlation analysis

Sparse kernel SVMs via cutting-plane training

Multi-view kernel construction

Block coordinate descent algorithms for large-scale sparse multiclass classification

Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis