NDLI: Information Theoretic Bounds for Distributed Computation Over Networks of Point-to-Point Channels

Content Provider	IEEE Xplore Digital Library
Author	Ayaso, O. Shah, D. Dahleh, M.A.
Copyright Year	1963
Abstract	A network of nodes communicate via point-to-point memoryless independent noisy channels. Each node has some real-valued initial measurement or message. The goal of each of the nodes is to acquire an estimate of a given function of all the initial measurements in the network. As the main contribution of this paper, a lower bound on computation time is derived. This bound must be satisfied by any algorithm used by the nodes to communicate and compute, so that the mean-square error in the nodes' estimate is within a given interval around zero. The derivation utilizes information theoretic inequalities reminiscent of those used in rate distortion theory along with a novel “perturbation” technique so as to be broadly applicable. To understand the tightness of the bound, a specific scenario is considered. Nodes are required to learn a linear combination of the initial values in the network while communicating over erasure channels. A distributed quantized algorithm is developed, and it is shown that the computation time essentially scales as is implied by the lower bound. In particular, the computation time depends reciprocally on “conductance”, which is a property of the network that captures the information-flow bottleneck. As a by-product, this leads to a quantized algorithm, for computing separable functions in a network, with minimal computation time.
Sponsorship	IEEE Information Theory Society
Starting Page	6020
Ending Page	6039
Page Count	20
File Size	374542
File Format	PDF
ISSN	00189448
Volume Number	56
Issue Number	12
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-12-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Rate distortion theory Noise measurement Memoryless systems Distributed computing quantized summation Computation time conductance distributed computing noisy networks
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

1information theoretic bounds for distributed computation.

1 information theoretic bounds for distributed computation over networks of point-to-point channels.

1Information Theoretic Bounds for Distributed Computation

Approximate Sparsity Pattern Recovery: Information-Theoretic Lower Bounds

Information theoretic performance bounds for noisy compressive sensing

Universally attainable error exponents for rate-distortion coding of noisy sources

Quantum rate-distortion theory for memoryless sources

Operational rate-distortion design for joint source-channel coding over noisy channels

Information Theoretic Bounds for Compressed Sensing

Information Theoretic Bounds for Distributed Computation Over Networks of Point-to-Point Channels

Similar Documents

1information theoretic bounds for distributed computation.

1 information theoretic bounds for distributed computation over networks of point-to-point channels.

1Information Theoretic Bounds for Distributed Computation

Approximate Sparsity Pattern Recovery: Information-Theoretic Lower Bounds

Information theoretic performance bounds for noisy compressive sensing

Universally attainable error exponents for rate-distortion coding of noisy sources

Quantum rate-distortion theory for memoryless sources

Operational rate-distortion design for joint source-channel coding over noisy channels

Information Theoretic Bounds for Compressed Sensing

Information Theoretic Bounds for Distributed Computation Over Networks of Point-to-Point Channels