NDLI: Approximative distance computation by random hashing

Content Provider	Springer Nature Link
Author	Mimaroglu, Selim Yagci, Murat Simovici, Dan A.
Copyright Year	2011
Abstract	We propose an approximate computation technique for inter-object distances of binary data sets. Our approach is based on locality sensitive hashing. We randomly select a number of projections of the data set and group objects into buckets based on the hash values of these projections. For each pair of objects, occurrences in the same bucket are counted and the exact Hamming distance is approximated based on the number of co-occurrences in all buckets. We parallelize the computation using mainly two schemes. The first assigns each random subspace to a processor for calculating the local co-occurrence matrix, where all the local co-occurrence matrices are combined into the final co-occurrence matrix. The second method provides the same distance approximation in longer runtimes by limiting the total message size in a parallel computing environment, which is especially useful for very large data sets generating immense message traffic. Our methods produce very accurate results, scale up well with the number of objects, and tolerate processor failures. Experimental evaluations on supercomputers and workstations with several processors demonstrate the usefulness of our methods.
Starting Page	572
Ending Page	589
Page Count	18
File Format	PDF
ISSN	09208542
Journal	The Journal of Supercomputing
Volume Number	61
Issue Number	3
e-ISSN	15730484
Language	English
Publisher	Springer US
Publisher Date	2011-05-26
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Computer Science Programming Languages, Compilers, Interpreters Processor Architectures
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Information Systems Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Software Architectures and Coordination Models

Advances in multicore systems architectures

Green networks

H-BSP: A Hierarchical BSP Computation Model

Superlinear Performance in Real-Time Parallel Computation

On private Hamming distance computation

Design of Processor Arrays for Reconfigurable Architectures

Engineering of Reconfigurable Hardware/Software Objects

Embedded Fault-Tolerant Systems

Approximative distance computation by random hashing

Similar Documents

Software Architectures and Coordination Models

Advances in multicore systems architectures

Green networks

H-BSP: A Hierarchical BSP Computation Model

Superlinear Performance in Real-Time Parallel Computation

On private Hamming distance computation

Design of Processor Arrays for Reconfigurable Architectures

Engineering of Reconfigurable Hardware/Software Objects

Embedded Fault-Tolerant Systems

Approximative distance computation by random hashing