NDLI: Lower and Upper Bounds for Misclassification Probability Based on Renyi's Information

Content Provider	Springer Nature Link
Author	Erdogmus, Deniz Principe, Jose C.
Copyright Year	2004
Abstract	Fano's inequality has proven to be one important result in Shannon's information theory having found applications in numerous proofs of convergence. It also provides us with a lower bound on the symbol error probability in a communication channel, in terms of Shannon's definitions of entropy and mutual information. This result is also significant in that it suggests insights on how the classification performance is influenced by the amount of information transferred through the classifier. We have previously extended Fano's lower bound on the probability of error to a family of lower and upper bounds based on Renyi's definitions of entropy and mutual information. These new bounds however, despite their theoretical appeal, were practically incomputable. In this paper, we present some modifications to these bounds that will allow us to utilize them in practical situations. The significance of these new bounds is threefold: Illustrating a theoretical use of Renyi's definition of information, extending Fano's result to include an upper bound for probability of classification error, and providing insights on how the information transfer through a classifier affects its performance. The performance of the modified bounds is investigated in various numerical examples, including applications to digital communication channels that are designed to point out the major conclusions.
Starting Page	305
Ending Page	317
Page Count	13
File Format	PDF
ISSN	09225773
Journal	Journal of Signal Processing Systems
Volume Number	37
Issue Number	2-3
Language	English
Publisher	Kluwer Academic Publishers
Publisher Date	2004-06-01
Publisher Place	Boston
Access Restriction	Subscribed
Subject Keyword	Signal, Image and Speech Processing Circuits and Systems Electrical Engineering Image Processing and Computer Vision Pattern Recognition Computer Imaging, Vision, Pattern Recognition and Graphics
Content Type	Text
Resource Type	Article
Subject	Signal Processing Information Systems Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

CORDIC-Based Computation of ArcCos

Advanced RNN Based NARMA Predictors

On Probability Density for Modeling Video Traffic

Approximate Signal Processing

Editorial Message

Greedy Detection

Power-Aware 3D Computer Graphics Rendering

A FPGA-based Library for On-Line Signal Processing

Bounds on SIMO and MIMO Channel Estimation and Equalization with Side Information

Lower and Upper Bounds for Misclassification Probability Based on Renyi's Information

Similar Documents

CORDIC-Based Computation of ArcCos

Advanced RNN Based NARMA Predictors

On Probability Density for Modeling Video Traffic

Approximate Signal Processing

Editorial Message

Greedy Detection

Power-Aware 3D Computer Graphics Rendering

A FPGA-based Library for On-Line Signal Processing

Bounds on SIMO and MIMO Channel Estimation and Equalization with Side Information

Lower and Upper Bounds for Misclassification Probability Based on Renyi's Information