NDLI: Clustering very large multi-dimensional datasets with MapReduce

Please wait, while we are loading the content...

Convex optimization: from embedded real-time to large-scale distributed

Internet scale data analysis

Cancer genomics

The mathematics of causal inference

Lessons learned from contests in data mining

CHIRP: a new classifier based on composite hypercubes on iterated random projections

Integrating low-rank and group-sparse structures for robust multi-task learning

Diversity in ranking via resistive graph centers

Scalable distributed inference of dynamic user interests for behavioral targeting

From bias to opinion: a transfer-learning approach to real-time sentiment analysis

Enabling fast prediction for ensemble models on data streams

Linear scale semantic mining algorithms in microsoft SQL server's semantic platform

Estimating the number of users behind ip addresses for combating abusive traffic

Applying data mining techniques to address disaster information management challenges on mobile devices

Driving with knowledge from the physical world

Experiences with mining temporal event sequences from electronic medical records: initial successes and some challenges

Matching unstructured product offers to structured product specifications

Beyond keyword search: discovering relevant scientific literature

Refining causality: who copied from whom?

Differentially private data release for data mining

On the semantic annotation of places in location-based social networks

Stackelberg games for adversarial prediction problems

Tell me what i need to know: succinctly summarizing data with itemsets

Latent topic feedback for information retrieval

Density estimation trees

Dual active feature and sample selection for graph classification

Fast clustering using MapReduce

Bounded coordinate-descent for biological sequence classification in high dimensional predictor space

A taxi business intelligence system

"Which half Is wasted?": controlled experiments to measure online-advertising effectiveness

2D-interval predictions for time series

Supervised learning for provenance-similarity of binaries

Model order selection for boolean matrix factorization

Collective graph identification

Multiple domain user personalization

User reputation in a comment rating environment

Online active inference and learning

Combining file content and file relations for cloud based malware detection

Data-driven multi-touch attribution models

Enhancing investment decisions in P2P lending: an investor composition perspective

Interactive learning for efficiently detecting errors in insurance claims

Understanding atrophy trajectories in alzheimer's disease using association rules on MRI images

Predictive client-side profiles for personalized advertising

Collaborative topic modeling for recommending scientific articles

Conditional topical coding: an efficient topic model conditioned on rich features

k-NN as an implementation of situation testing for discrimination discovery and prevention

Sparsification of influence networks

Leakage in data mining: formulation, detection, and avoidance

Direct local pattern sampling by efficient two-step random procedures

Localized factor models for multi-context recommendation

Unsupervised clustering of multidimensional distributions using earth mover distance

It's who you know: graph mining using recursive structural features

Clustering very large multi-dimensional datasets with MapReduce

Multi-source domain adaptation and its application to early detection of fatigue

Apolo: interactive large graph sensemaking by combining machine learning and visualization

Accelerating large-scale data mining using in-database analytics

A game theoretic framework for heterogenous information network clustering

Trading representability for scalability: adaptive multi-hyperplane machine for nonlinear classification

Rank aggregation via nuclear norm minimization

Semi-supervised ranking on very large graphs with rich metadata

Click shaping to optimize multiple objectives

Selecting a comprehensive set of reviews

Unbiased online active learning in data streams

High-precision phrase-based document classification on a modern scale

Bid landscape forecasting in online ad exchange marketplace

From market baskets to mole rats: using data mining techniques to analyze RFID data describing laboratory animal behavior

NIMBLE: a toolkit for the implementation of parallel data mining and machine learning algorithms on mapreduce

A case study in a recommender system based on purchase data

Smoothing techniques for adaptive online language models: topic tracking in tweet streams

Partially labeled topic models for interpretable text mining

Tracking trends: incorporating term volume into temporal topic models

Exploiting vulnerability to secure user privacy on a social networking site

Leveraging collaborative tagging for web item design

An information theoretic framework for data mining

Mining frequent closed graphs on evolving data streams

Latent aspect rating analysis without aspect keyword supervision

Online heterogeneous mixture modeling with marginal and copula selection

Triangle listing in massive networks and its applications

Selective block minimization for faster convergence of limited memory large-scale linear models

Two-locus association mapping in subquadratic time

Article clipper: a system for web article extraction

Applications of data mining and machine learning in online customer care

A GPU-tailored approach for training kernelized SVMs

An improved GLMNET for l1-regularized logistic regression

Large-scale matrix factorization with distributed stochastic gradient descent

Benefits of bias: towards better characterization of network sampling

Response prediction using collaborative filtering with hierarchies and side-information

Learning to trade off between exploration and exploitation in multiclass bandit prediction

Activity analysis based on low sample rate smart meters

Detecting adversarial advertisements in the wild

A pattern discovery approach to retail fraud detection

Classification of proxy labeled examples for marketing segment generation

Detecting bots via incremental LS-SVM learning with dynamic feature adaptation

Democrats, republicans and starbucks afficionados: user classification in twitter

Data intensive analysis on the gordon high performance data and compute system

Broad scale predictive modeling and marketing optimization in retail sales

A multi-task learning formulation for predicting disease progression

Ameliorating buyer's remorse

Toward personalized care management of patients at risk: the diabetes case study

Frontex real-time news event extraction framework

Knowledge discovery and data mining in pharmaceutical cancer research

A simple statistical model and association rule filtering for classification

LikeMiner: a system for mining the power of 'like' in social media networks

Operational security analytics: doing more with less

A time-dependent topic model for multiple text streams

MIME: a framework for interactive visual pattern mining

Real-time risk control system for CNP (card not present)

Active learning for node classification in assortative and disassortative networks

SIGKDD demo: sensors and software to allow computational entomology, an emerging application of data mining

The power of analysis and data

Active learning using on-line algorithms

Social flocks: a crowd simulation framework for social network generation, community detection, and collective behavior modeling

The practitioner's viewpoint to data mining: key lessons learned in the trenches and case studies

Algorithms for speeding up distance-based outlier detection

Topic-level social network search

Thriving as a data miner in the real world

An effective evaluation measure for clustering on evolving data streams

Video analytics solution for tracking customer locations in retail shopping malls

An iterated graph laplacian approach for ranking on manifolds

Anomaly localization for network data streams with graph joint sparse PCA

Approximate kernel k-means: solution to large scale kernel clustering

Ask me better questions: active learning queries based on rule induction

Automatically tagging email by leveraging other users' folders

Axiomatic ranking of network role similarity

Brain effective connectivity modeling for alzheimer's disease by sparse gaussian bayesian network

Classification of functional magnetic resonance imaging data using informative pattern features

Clustering with relative constraints

Common component analysis for multiple covariance matrices

Compression of weighted graphs

Content-driven trust propagation framework

Cost-aware travel tour recommendation

Discovering highly reliable subgraphs in uncertain graphs

Discovering shakers from evolving entities via cascading graph inference

Discovering spatio-temporal causal interactions in traffic data streams

Display advertising impact: search lift and social influence

Diversified ranking on large graphs: an optimization viewpoint

Entity disambiguation with hierarchical topic models

Exploiting place features in link prediction on location-based social networks

Fast approximate similarity search based on degree-reduced neighborhood graphs

Fast coordinate descent methods with variable selection for non-negative matrix factorization

Fast locality-sensitive hashing

Friendship and mobility: user movement in location-based social networks

GBASE: a scalable and general graph management system

Human mobility, social ties, and link prediction

I want to answer; who has a question?: Yahoo! answers recommender system

Improving predictions using aggregate information

INCONCO: interpretable clustering of numerical and categorical objects

Incorporating SAT solvers into hierarchical clustering algorithms: an efficient and flexible approach

Latent graphical models for quantifying and predicting patent quality

Logical-shapelets: an expressive primitive for time series classification

Meta optimization and its application to portfolio selection

Mining closed episodes with simultaneous events

Mining mobility data to minimise travellers' spending on public transport

Mining mobility user profiles for car pooling

Mining partially annotated images

Multi-view transfer learning with a large margin approach

MultiRank: co-ranking for objects and relations in multi-relational data

On dynamic data-driven selection of sensor streams

On the privacy of anonymized networks

Ontology enhancement and concept granularity learning: keeping yourself current and adaptive

Personal privacy vs population privacy: learning to attack anonymization

Privacy-preserving social network publication against friendship attacks

Probabilistic topic models with biased propagation on heterogeneous information networks

Prominent streak discovery in sequence data

Protecting location privacy using location semantics

Ranking-based classification of heterogeneous information networks

Real-time bidding algorithms for performance-based display ad allocation

Revisiting sequential pattern hiding to enhance utility

Sampling hidden objects using nearest-neighbor oracles

Scalable kNN search on vertically stored time series

Serendipitous learning: learning beyond the predefined label space

Spatially regularized logistic regression for disease mapping on large moving populations

Temporal multi-hierarchy smoothing for estimating rates of rare events

ThermoCast: a cyber-physical forecasting model for datacenters

Towards bounding sequential patterns

User-click modeling for understanding and predicting search-behavior

User-level sentiment analysis incorporating social networks

Web information extraction using markov logic networks

Clustering very large multi-dimensional datasets with MapReduce

Content Provider	ACM Digital Library
Author	López, Julio Kang, U. Traina, Caetano Machado Traina, Agma Juci Ferreira Cordeiro, Robson Leonardo Faloutsos, Christos
Abstract	Given a very large moderate-to-high dimensionality dataset, how could one cluster its points? For datasets that don't fit even on a single disk, parallelism is a first class option. In this paper we explore MapReduce for clustering this kind of data. The main questions are (a) how to minimize the I/O cost, taking into account the already existing data partition (e.g., on disks), and (b) how to minimize the network cost among processing nodes. Either of them may be a bottleneck. Thus, we propose the Best of both Worlds -- BoW method, that automatically spots the bottleneck and chooses a good strategy. Our main contributions are: (1) We propose BoW and carefully derive its cost functions, which dynamically choose the best strategy; (2) We show that BoW has numerous desirable features: it can work with most serial clustering methods as a plugged-in clustering subroutine, it balances the cost for disk accesses and network accesses, achieving a very good tradeoff between the two, it uses no user-defined parameters (thanks to our reasonable defaults), it matches the clustering quality of the serial algorithm, and it has near-linear scale-up; and finally, (3) We report experiments on real and synthetic data with billions of points, using up to 1,024 cores in parallel. To the best of our knowledge, our Yahoo! web is the largest real dataset ever reported in the database subspace clustering literature. Spanning 0.2 TB of multi-dimensional data, it took only 8 minutes to be clustered, using 128 cores.
Starting Page	690
Ending Page	698
Page Count	9
File Format	PDF
ISBN	9781450308137
DOI	10.1145/2020408.2020516
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2011-08-21
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Mapreduce Very large multi-dimensional datasets Clustering
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in