NDLI: Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner

Please wait, while we are loading the content...

Challenges for knowledge discovery in biology

Extracting targeted data from the web

Mass collaboration and data mining

Applications of generalized support vector machines to predictive modeling

Data mining: are we there yet?

Mining e-commerce data: the good, the bad, and the ugly

Data mining platform for database developers

Recommender systems in commerce and community

The "DGX" distribution for mining massive, skewed data

Data mining criteria for tree-based regression and classification

Probabilistic modeling of transaction data with applications to profiling, visualization, and prediction

GESS: a scalable similarity-join algorithm for mining large data sets in high dimensional spaces

Mining the network value of customers

Empirical bayes screening for multi-item associations

Proximal support vector machine classifiers

Data mining with sparse grids using simplicial basis functions

Mining time-changing data streams

Visualizing multi-dimensional clusters, trends, and outliers using star coordinates

Ensemble-index: a new approach to indexing large databases

Robust space transformations for distance-based operations

Molecular feature mining in HIV data

Discovering unexpected information from your competitors' web sites

Personalization from incomplete data: what you don't know can hurt

Probabilistic query models for transaction data

Extracting collective probabilistic forecasts from web games

Tri-plots: scalable tools for multidimensional data mining

Efficient discovery of error-tolerant frequent itemsets in high dimensions

Learning and making decisions when costs and probabilities are both unknown

Data mining case study: modeling the behavior of offenders who commit serious sexual assaults

A human-computer cooperative system for effective high dimensional clustering

Mining massively incomplete data sets by conceptual reconstruction

Evaluating the novelty of text-mined rules using lexical knowledge

Fast ordering of large categorical datasets for better visualization

Random projection in dimensionality reduction: applications to image and text data

Gaining insights into support vector machine pattern classifiers using projection-based tour methods

PVA: a self-adaptive personal view agent system

A robust and scalable clustering algorithm for mixed type attributes in large database environment

Co-clustering documents and words using bipartite spectral graph partitioning

A spectral method to separate disconnected and nearly-disconnected web graph components

Clustering spatial data using random walks

Solving regression problems with rule-based ensemble classifiers

Mining top-n local outliers in large databases

Generalized clustering, supervised learning, and data assignment

Mining a stream of transactions for customer patterns

The distributed boosting algorithm

Induction of semantic classes from natural language text

DIRT @SBT@discovery of inference rules from text

Identifying non-actionable association rules

Discovering the set of fundamental rule changes

Finding simple intensity descriptions from event sequence data

Data filtering for automatic classification of rocks from reflectance spectra

Mining frequent neighboring class sets in spatial databases

Experimental comparisons of online and batch versions of bagging and boosting

TreeDT: gene mapping by tree disequilibrium test

Detecting graph-based spatial outliers: algorithms and applications (a summary of results)

A streaming ensemble algorithm (SEA) for large-scale classification

Discovering associations with numeric variables

Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner

Infominer: mining surprising periodic patterns

Real world performance of association rule algorithms

Segmentation-based modeling for advanced targeted marketing

Interactive path analysis of web site traffic

Estimating business targets

Magical thinking in data mining: lessons from CoIL challenge 2000

REVI-MINER, a KDD-environment for deviation detection and analysis of warranty and goodwill cost statements in automotive industry

Data mining techniques to improve forecast accuracy in airline business

Mining from open answers in questionnaire data

Funnel report mining for the MSN network

Evaluation of prediction models for marketing campaigns

Knowledge base maintenance using knowledge gap analysis

Mining user session data to facilitate user interaction with a customer service knowledge base in RightNow Web

Mining web logs for prediction models in WWW caching and prefetching

Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner

Content Provider	ACM Digital Library
Author	Yamanishi, Kenji Takeuchi, Jun-ichi
Abstract	This paper is concerned with the problem of detecting outliers from unlabeled data. In prior work we have developed SmartSifter, which is an on-line outlier detection algorithm based on unsupervised learning from data. On the basis of SmartSifter this paper yields a new framework for outlier filtering using both supervised and unsupervised learning techniques iteratively in order to make the detection process more effective and more understandable. The outline of the framework is as follows: In the first round, for an initial dataset, we run SmartSifter to give each data a score, with a high score indicating a high possibility of being an outlier. Next, giving positive labels to a number of higher scored data and negative labels to a number of lower scored data, we create labeled examples. Then we construct an outlier filtering rule by supervised learning from them. Here the rule is generated based on the principle of minimizing extended stochastic complexity. In the second round, for a new dataset, we filter the data using the constructed rule, then among the filtered data, we run SmartSifter again to evaluate the data in order to update the filtering rule. Applying of our framework to the network intrusion detection, we demonstrate that 1) it can significantly improve the accuracy of SmartSifter, and 2) outlier filtering rules can help the user to discover a general pattern of an outlier group.
Starting Page	389
Ending Page	394
Page Count	6
File Format	PDF
ISBN	158113391X
DOI	10.1145/502512.502570
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2001-08-26
Publisher Place	New York
Access Restriction	Subscribed
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in