NDLI: All-distances sketches, revisited: HIP estimators for massive graphs analysis

Please wait, while we are loading the content...

Incomplete data: what went wrong, and how to fix it

Expressive languages for querying the semantic web

Weaker forms of monotonicity for declarative networking: a more fine-grained answer to the calm-conjecture

Model-data Ecosystems: challenges, tools, and trends

All-distances sketches, revisited: HIP estimators for massive graphs analysis

Enumerating answers to first-order queries over databases of low degree

Database principles in information extraction

Nested dependencies: structure and reasoning

Skew in parallel query processing

Independent range sampling

Expressiveness of guarded existential rule languages

Composable core-sets for diversity and coverage maximization

Counting solutions to conjunctive queries: structural and hybrid tractability

Cleaning inconsistencies in information extraction via prioritized repairs

Does query evaluation tractability help query containment?

The input/output complexity of triangle enumeration

A dynamic I/O-efficient structure for one-dimensional top-k range reporting

Containment and equivalence of well-designed SPARQL

Is min-wise hashing optimal for summarizing set intersection?

A dichotomy for non-repeating queries with negation in probabilistic databases

Generating low-cost plans from proofs

Beyond worst-case analysis for joins with minesweeper

Categorical range maxima queries

On scale independence for querying big data

Cost-oblivious storage reallocation

All-distances sketches, revisited: HIP estimators for massive graphs analysis

Content Provider	ACM Digital Library
Author	Cohen, Edith
Abstract	Graph datasets with billions of edges, such as social and Web graphs, are prevalent. To be feasible, computation on such large graphs should scale linearly with graph size. All-distances sketches (ADSs) are emerging as a powerful tool for scalable computation of some basic properties of individual nodes or the whole graph. ADSs were first proposed two decades ago (Cohen 1994) and more recent algorithms include ANF (Palmer, Gibbons, and Faloutsos 2002) and hyperANF (Boldi, Rosa, and Vigna 2011). A sketch of logarithmic size is computed for each node in the graph and the computation in total requires only a near linear number of edge relaxations. From the ADS of a node, we can estimate neighborhood cardinalities (the number of nodes within some query distance) and closeness centrality. More generally we can estimate the distance distribution, effective diameter, similarities, and other parameters of the full graph. We make several contributions which facilitate a more effective use of ADSs for scalable analysis of massive graphs. We provide, for the first time, a unified exposition of ADS algorithms and applications. We present the Historic Inverse Probability (HIP) estimators which are applied to the ADS of a node to estimate a large natural class of queries including neighborhood cardinalities and closeness centralities. We show that our HIP estimators have at most half the variance of previous neighborhood cardinality estimators and that this is essentially optimal. Moreover, HIP obtains a polynomial improvement over state of the art for more general domain queries and the estimators are simple, flexible, unbiased, and elegant. The ADS generalizes Min-Hash sketches, used for approximating cardinality (distinct count) on data streams. We obtain lower bounds on Min-Hash cardinality estimation using classic estimation theory. We illustrate the power of HIP, both in terms of ease of application and estimation quality, by comparing it to the HyperLogLog algorithm (Flajolet et al. 2007), demonstrating a significant improvement over this state-of-the-art practical algorithm. We also study the quality of ADS estimation of distance ranges, generalizing the near-linear time factor-2 approximation of the diameter.
Starting Page	88
Ending Page	99
Page Count	12
File Format	PDF
ISBN	9781450323758
DOI	10.1145/2594538.2594546
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-06-18
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Approximate distinct counters Estimation Closeness centrality Min-hash sketch All-distances sketch
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in