NDLI: Concept Clustering of Evolving Data

Content Provider	IEEE Xplore Digital Library
Author	Shixi Chen Haixun Wang Shuigeng Zhou
Copyright Year	2009
Abstract	In Web search, a user refines his search several times before he finds the information he needs. It is very likely that, in the search log, similar sequences of searches appear many times, as many users had searched the Web with the same intent. Precisely interpreting the intent of the user is difficult, even with the help of the search log: there might be numerous instances of such intent scattering in small pieces in the log, but none of them is comprehensive enough to describe the concept precisely. This scenario occurs in many applications. For example, patterns in Web search, Internet traffic, program execution traces, network events, etc., are often non-stationary, yet the same patterns recur over time. In this paper, we argue that visible patterns are generated by hidden intent or hidden concepts, and precisely characterizing such concepts is only possible if we cluster as much data generated by such concepts as possible and learn from the clustered data as a whole, instead of learning from a single episode of such concept. The benefits is obvious as it enables us not only to better understand the underlying system that generates the data, but also to recognize future instance of a concept as soon as it occurs. To achieve this, we introduce a clustering based approach, where we adopt a novel clustering criterion, validation error minimization, to ensure that the found concepts are unique and precise. We propose a two step algorithm, which uses enhanced dynamic programming and EM like methods for clustering. Experiments show that in benchmark datasets, our approach achieves the highest accuracy with lowest cost in comparison with the current best approaches.
Starting Page	1327
Ending Page	1330
File Size	235051
Page Count	4
File Format	PDF
ISBN	9781424434220
ISSN	10844627
DOI	10.1109/ICDE.2009.232
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-03-29
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	History Web search Venture capital Scattering Supervised learning Data engineering USA Councils Training data Unsupervised learning Euclidean distance
Content Type	Text
Resource Type	Article
Subject	Signal Processing Information Systems Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Online Evaluation of Patterns from Evolving Web Data Streams

Using Anonymized Data for Classification

WEST: Modern Technologies for Web People Search

A clustering algorithm combine the FCM algorithm with supervised learning normal mixture model

A new clustering evaluation function using Renyi's information potential

Efficient Private Record Linkage

Similarity Group-By

OPAQUE: Protecting Path Privacy in Directions Search

Shared Winner Determination in Sponsored Search Auctions

Concept Clustering of Evolving Data

Similar Documents

Online Evaluation of Patterns from Evolving Web Data Streams

Using Anonymized Data for Classification

WEST: Modern Technologies for Web People Search

A clustering algorithm combine the FCM algorithm with supervised learning normal mixture model

A new clustering evaluation function using Renyi's information potential

Efficient Private Record Linkage

Similarity Group-By

OPAQUE: Protecting Path Privacy in Directions Search

Shared Winner Determination in Sponsored Search Auctions

Concept Clustering of Evolving Data