NDLI: Unsupervised learning of parsimonious mixtures on large spaces with integrated feature and component selection

Content Provider	IEEE Xplore Digital Library
Author	Graham, M.W. Miller, D.J.
Copyright Year	1991
Abstract	Estimating the number of components (the order) in a mixture model is often addressed using criteria such as the Bayesian information criterion (BIC) and minimum message length. However, when the feature space is very large, use of these criteria may grossly underestimate the order. Here, it is suggested that this failure is not mainly attributable to the criterion (e.g., BIC), but rather to the lack of "structure" in standard mixtures-these models trade off data fitness and model complexity only by varying the order. The authors of the present paper propose mixtures with a richer set of tradeoffs. The proposed model allows each component its own informative feature subset, with all other features explained by a common model (shared by all components). Parameter sharing greatly reduces complexity at a given order. Since the space of these parsimonious modeling solutions is vast, this space is searched in an efficient manner, integrating the component and feature selection within the generalized expectation-maximization (GEM) learning for the mixture parameters. The quality of the proposed (unsupervised) solutions is evaluated using both classification error and test set data likelihood. On text data, the proposed multinomial version-learned without labeled examples, without knowing the "true" number of topics, and without feature preprocessing-compares quite favorably with both alternative unsupervised methods and with a supervised naive Bayes classifier. A Gaussian version compares favorably with a recent method introducing "feature saliency" in mixtures.
Sponsorship	IEEE Signal Processing Society
Starting Page	1289
Ending Page	1303
Page Count	15
File Size	667433
File Format	PDF
ISSN	1053587X
Volume Number	54
Issue Number	4
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2006-04-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Unsupervised learning Bayesian methods Testing Clustering algorithms Length measurement Estimation error unsupervised feature selection Bayesian Information Criterion (BIC) document clustering EM algorithm mixture models model order selection
Content Type	Text
Resource Type	Article
Subject	Signal Processing Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Joint Parsimonious Modeling and Model Order Selection for Multivariate Gaussian Mixtures

Simultaneous feature selection and clustering using mixture models (2004)

Simultaneous feature selection and clustering using mixture models

Model Selection with BIC and ICL Criteria for Binned Data Clustering by Bin-EM-CEM Algorithms

Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures

Bayesian feature and model selection for Gaussian mixture models

Localized feature selection for Gaussian mixtures using variational learning

Unsupervised selection and estimation of finite mixture models

Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting

Unsupervised learning of parsimonious mixtures on large spaces with integrated feature and component selection

Similar Documents

Joint Parsimonious Modeling and Model Order Selection for Multivariate Gaussian Mixtures

Simultaneous feature selection and clustering using mixture models (2004)

Simultaneous feature selection and clustering using mixture models

Model Selection with BIC and ICL Criteria for Binned Data Clustering by Bin-EM-CEM Algorithms

Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures

Bayesian feature and model selection for Gaussian mixture models

Localized feature selection for Gaussian mixtures using variational learning

Unsupervised selection and estimation of finite mixture models

Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting

Unsupervised learning of parsimonious mixtures on large spaces with integrated feature and component selection