NDLI: Entropy of the Mixture of Sources and Entropy Dimension

Content Provider	IEEE Xplore Digital Library
Author	Smieja, M. Tabor, J.
Copyright Year	1963
Abstract	Suppose that we are given two sources S₁, S₂ which both send us information from the data space X . We assume that we lossy-code information coming from S₁ and S₂ with the same maximal error but with different alphabets P₁ and P₂, respectively. Consider a new source S which sends a signal produced by source S₁ with probability a₁ and by source S₂ with probability a₂=1-a₁ . We provide a simple greedy algorithm which constructs a coding alphabet P which encodes data from S with the same value of maximal error as single sources, such that the entropy h(S;P) satisfies: h(S;P) ≤ a₁h(S₁;P₁)+a₂h(S₂;P₂) +1. In the proof of the aforementioned formula, the basic role is played by a new equivalent definition of entropy based on measures instead of partitions. As a consequence, we decompose the entropy dimension of the mixture of sources in terms of the convex combination of the entropy dimensions of the single sources. In the case of probability measures in \BBR^N, this allows us to link the upper local dimension at point with the upper entropy dimension of a measure by an improved version of Young estimation.
Sponsorship	IEEE Information Theory Society
Starting Page	2719
Ending Page	2728
Page Count	10
File Size	242619
File Format	PDF
ISSN	00189448
Volume Number	58
Issue Number	5
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-05-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Entropy Q measurement Weight measurement Estimation Channel coding Measurement uncertainty Shannon entropy Entropy coding entropy dimension lossy coding mixture of sources Rényi information dimension
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Rényi entropy dimension of the mixture of measures

On the Conditional Rényi Entropy

Encoding Tasks and Rényi Entropy

Optimal Phase Transitions in Compressed Sensing

Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression

Lossy Joint Source-Channel Coding in the Finite Blocklength Regime

Coding Theorems for Compound Problems via Quantum Rényi Divergences

Redundancy-Related Bounds for Generalized Huffman Codes

Channel Coding and Lossy Source Coding Using a Generator of Constrained Random Numbers

Entropy of the Mixture of Sources and Entropy Dimension

Similar Documents

Rényi entropy dimension of the mixture of measures

On the Conditional Rényi Entropy

Encoding Tasks and Rényi Entropy

Optimal Phase Transitions in Compressed Sensing

Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression

Lossy Joint Source-Channel Coding in the Finite Blocklength Regime

Coding Theorems for Compound Problems via Quantum Rényi Divergences

Redundancy-Related Bounds for Generalized Huffman Codes

Channel Coding and Lossy Source Coding Using a Generator of Constrained Random Numbers

Entropy of the Mixture of Sources and Entropy Dimension