NDLI: The "DGX" distribution for mining massive, skewed data

Please wait, while we are loading the content...

Challenges for knowledge discovery in biology

Extracting targeted data from the web

Mass collaboration and data mining

Applications of generalized support vector machines to predictive modeling

Data mining: are we there yet?

Mining e-commerce data: the good, the bad, and the ugly

Data mining platform for database developers

Recommender systems in commerce and community

The "DGX" distribution for mining massive, skewed data

Data mining criteria for tree-based regression and classification

Probabilistic modeling of transaction data with applications to profiling, visualization, and prediction

GESS: a scalable similarity-join algorithm for mining large data sets in high dimensional spaces

Mining the network value of customers

Empirical bayes screening for multi-item associations

Proximal support vector machine classifiers

Data mining with sparse grids using simplicial basis functions

Mining time-changing data streams

Visualizing multi-dimensional clusters, trends, and outliers using star coordinates

Ensemble-index: a new approach to indexing large databases

Robust space transformations for distance-based operations

Molecular feature mining in HIV data

Discovering unexpected information from your competitors' web sites

Personalization from incomplete data: what you don't know can hurt

Probabilistic query models for transaction data

Extracting collective probabilistic forecasts from web games

Tri-plots: scalable tools for multidimensional data mining

Efficient discovery of error-tolerant frequent itemsets in high dimensions

Learning and making decisions when costs and probabilities are both unknown

Data mining case study: modeling the behavior of offenders who commit serious sexual assaults

A human-computer cooperative system for effective high dimensional clustering

Mining massively incomplete data sets by conceptual reconstruction

Evaluating the novelty of text-mined rules using lexical knowledge

Fast ordering of large categorical datasets for better visualization

Random projection in dimensionality reduction: applications to image and text data

Gaining insights into support vector machine pattern classifiers using projection-based tour methods

PVA: a self-adaptive personal view agent system

A robust and scalable clustering algorithm for mixed type attributes in large database environment

Co-clustering documents and words using bipartite spectral graph partitioning

A spectral method to separate disconnected and nearly-disconnected web graph components

Clustering spatial data using random walks

Solving regression problems with rule-based ensemble classifiers

Mining top-n local outliers in large databases

Generalized clustering, supervised learning, and data assignment

Mining a stream of transactions for customer patterns

The distributed boosting algorithm

Induction of semantic classes from natural language text

DIRT @SBT@discovery of inference rules from text

Identifying non-actionable association rules

Discovering the set of fundamental rule changes

Finding simple intensity descriptions from event sequence data

Data filtering for automatic classification of rocks from reflectance spectra

Mining frequent neighboring class sets in spatial databases

Experimental comparisons of online and batch versions of bagging and boosting

TreeDT: gene mapping by tree disequilibrium test

Detecting graph-based spatial outliers: algorithms and applications (a summary of results)

A streaming ensemble algorithm (SEA) for large-scale classification

Discovering associations with numeric variables

Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner

Infominer: mining surprising periodic patterns

Real world performance of association rule algorithms

Segmentation-based modeling for advanced targeted marketing

Interactive path analysis of web site traffic

Estimating business targets

Magical thinking in data mining: lessons from CoIL challenge 2000

REVI-MINER, a KDD-environment for deviation detection and analysis of warranty and goodwill cost statements in automotive industry

Data mining techniques to improve forecast accuracy in airline business

Mining from open answers in questionnaire data

Funnel report mining for the MSN network

Evaluation of prediction models for marketing campaigns

Knowledge base maintenance using knowledge gap analysis

Mining user session data to facilitate user interaction with a customer service knowledge base in RightNow Web

Mining web logs for prediction models in WWW caching and prefetching

The "DGX" distribution for mining massive, skewed data

Content Provider	ACM Digital Library
Author	Korn, Flip Bi, Zhiqiang Faloutsos, Christos
Abstract	Skewed distributions appear very often in practice. Unfortunately, the traditional Zipf distribution often fails to model them well. In this paper, we propose a new probability distribution, the Discrete Gaussian Exponential (DGX), to achieve excellent fits in a wide variety of settings; our new distribution includes the Zipf distribution as a special case. We present a statistically sound method for estimating the DGX parameters based on maximum likelihood estimation (MLE). We applied DGX to a wide variety of real world data sets, such as sales data from a large retailer chain, us-age data from AT&T, and Internet clickstream data; in all cases, DGX fits these distributions very well, with almost a 99% correlation coefficient in quantile-quantile plots. Our algorithm also scales very well because it requires only a single pass over the data. Finally, we illustrate the power of DGX as a new tool for data mining tasks, such as outlier detection.
Starting Page	17
Ending Page	26
Page Count	10
File Format	PDF
ISBN	158113391X
DOI	10.1145/502512.502521
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2001-08-26
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Maximum likelihood estimation Dgx Zipf's law Frequency-count plot Outlier detection Rank-frequency plot Lognormal distribution
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in