NDLI: Conditional NML Universal Models

Content Provider	IEEE Xplore Digital Library
Author	Rissanen, Jorma Roos, Teemu
Copyright Year	2007
Description	Author affiliation: Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki and Helsinki University of Technology, Email: jorma.rissanen@mdl-research.org (Rissanen, Jorma) \|\| Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki and Helsinki University of Technology, Email: teemu.roos@cs.helsinki.fi (Roos, Teemu)
Abstract	The NML (normalized maximum likelihood) universal model has certain minmax optimal properties but it has two shortcomings: the normalizing coefficient can be evaluated in a closed form only for special model classes, and it does not define a random process so that it cannot be used for prediction. We present a universal conditional NML model, which has minmax optimal properties similar to those of the regular NML model. However, unlike NML, the conditional NML model defines a random process which can be used for prediction. It also admits a recursive evaluation for data compression. The conditional normalizing coefficient is much easier to evaluate, for instance, for tree machines than the integral of the square root of the Fisher information in the NML model. For Bernoulli distributions, the conditional NML model gives a predictive probability, which behaves like the Krichevsky-Trofimov predictive probability, actually slightly better for extremely skewed strings. For some model classes, it agrees with the predictive probability found earlier by Takimoto and Warmuth, as the solution to a different more restrictive minmax problem. We also calculate the CNML models for the generalized Gaussian regression models, and in particular for the cases where the loss function is quadratic, and show that the CNML model achieves asymptotic optimality in terms of the mean ideal code length. Moreover, the quadratic loss, which represents fitting errors as noise rather than prediction errors, can be shown to be smaller than what can be achieved with the NML as well as with the so-called plug-in or the predictive MDL model.
Starting Page	337
Ending Page	341
File Size	296373
Page Count	5
File Format	PDF
ISBN	9780615153148
DOI	10.1109/ITA.2007.4357600
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2007-01-29
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	UCSD
Subject Keyword	Maximum likelihood estimation Data compression Predictive models Minimax techniques Density functional theory Probability distribution Random processes Parametric statistics Information technology
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A generalized minmax bound for universal coding

Mixture models achieving optimal coding regret

Researches on Properties of Generalized Normalization Coefficient in NML Distribution

Estimation For One Random Variable With Data Compression

Coding on Countably Infinite Alphabets

Some results on constrained maximum likelihood estimation

Maximum entropy approach to probability density estimation

Asymptotically minimax regret by Bayes mixtures

Universal Models for the Exponential Distribution

Conditional NML Universal Models

Similar Documents

A generalized minmax bound for universal coding

Mixture models achieving optimal coding regret

Researches on Properties of Generalized Normalization Coefficient in NML Distribution

Estimation For One Random Variable With Data Compression

Coding on Countably Infinite Alphabets

Some results on constrained maximum likelihood estimation

Maximum entropy approach to probability density estimation

Asymptotically minimax regret by Bayes mixtures

Universal Models for the Exponential Distribution

Conditional NML Universal Models