NDLI: Scalable Algorithms for Distribution Search

Content Provider	IEEE Xplore Digital Library
Author	Matsubara, Y. Sakurai, Y. Yoshikawa, M.
Copyright Year	2009
Abstract	Distribution data naturally arise in countless domains, such as meteorology, biology, geology, industry and economics. However, relatively little attention has been paid to data mining for large distribution sets. Given n distributions of multiple categories and a query distribution Q, we want to find similar clouds (i.e., distributions), to discover patterns, rules and outlier clouds. For example, consider the numerical case of sales of items, where, for each item sold, we record the unit price and quantity; then, each customer is represented as a distribution of 2-d points (one for each item he/she bought). We want to find similar users, e.g., for market segmentation, anomaly/fraud detection. We propose to address this problem and present D-Search, which includes fast and effective algorithms for similarity search in large distribution datasets. Our main contributions are (1) approximate KL divergence, which can speed up cloud-similarity computations, (2) multi-step sequential scan, which efficiently prunes a significant number of search candidates and leads to a direct reduction in the search cost. We also introduce an extended version of D-Search: (3) time-series distribution mining, which finds similar subsequences in time-series distribution datasets. Extensive experiments on real multi-dimensional datasets show that our solution achieves up to 2,300 faster wall-clock time over the naive implementation while it does not sacrifice accuracy.
Starting Page	347
Ending Page	356
File Size	679869
Page Count	10
File Format	PDF
ISBN	9781424452422
ISSN	15504786
DOI	10.1109/ICDM.2009.51
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-12-06
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Data mining Clouds Multimedia databases Biomedical imaging Meteorology Geology Industrial economics Power generation economics Kinetic energy Electroencephalography Singular value decomposition Distribution sets KL divergence
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

D-Search: an efficient and exact search algorithm for large distribution sets

CSVD: clustering and singular value decomposition for approximate similarity search in high-dimensional spaces

Solution for Search Result Clustering by Using Singular Value Decomposition

On continuous partial singular value decomposition algorithms

A manipulator workspace generation algorithm using a singular value decomposition

Incremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems

Removal of EMG artifacts from single channel EEG signal using singular spectrum analysis

Privacy-Preserving Data Mining Based on Sample Selection and Singular Value Decomposition

Inversion Of Imaging Spectrometry Data Using Singular Value Decomposition

Scalable Algorithms for Distribution Search

Similar Documents

D-Search: an efficient and exact search algorithm for large distribution sets

CSVD: clustering and singular value decomposition for approximate similarity search in high-dimensional spaces

Solution for Search Result Clustering by Using Singular Value Decomposition

On continuous partial singular value decomposition algorithms

A manipulator workspace generation algorithm using a singular value decomposition

Incremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems

Removal of EMG artifacts from single channel EEG signal using singular spectrum analysis

Privacy-Preserving Data Mining Based on Sample Selection and Singular Value Decomposition

Inversion Of Imaging Spectrometry Data Using Singular Value Decomposition

Scalable Algorithms for Distribution Search