NDLI: Near-duplicate web page detection: A comparative study of two contrary approaches

Content Provider	IEEE Xplore Digital Library
Author	Narayana, V.A. Govardhan, A. Premchand, P.
Copyright Year	2011
Description	Author affiliation: Sch. of IT, JNTU, Hyderabad, India (Govardhan, A.) \|\| Dept. of CSE, Osmania Univ., Hyderabad, India (Premchand, P.) \|\| Dept. of CSE, CMR Coll. of Eng. & Tech, Hyderabad, India (Narayana, V.A.)
Abstract	Detection of duplicate and near-duplicate web pages has attracted voluminous research among the web crawling research community. There have been a considerable number of significant researches available in the literature for near-duplicate detection, but, none has been accepted as a universal solution. G.S. Manku et al.'s fingerprint based approach proposed in 2007 was considered as one of the “state-of-the-art" algorithms for finding near-duplicate web pages. In our earlier work, we had devised an efficient similarity score based approach for near-duplicate web page detection. The experimentation on the proposed approach has showed that it has achieved almost detection accuracy identical to G.S. Manku et al.'s fingerprint based approach. Hence, in this paper, we conduct an extensive comparative study between our similarity score based approach and G.S. Manku et al.'s fingerprint based approach in terms of the computational factors namely: 1) Time and 2) Storage space. The performances of the two approaches were considered to be ideally the same, and so, we take up complexity measures namely time and memory space to determine the better approach of the two. The comparison study clearly portrays the better (less complex) of the two approaches based on the factors considered.
Sponsorship	IEEE
Starting Page	769
Ending Page	776
File Size	230978
Page Count	8
File Format	PDF
ISBN	9781457704727
e-ISBN	9788988678558
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2011-11-29
Publisher Place	South Korea
Access Restriction	Subscribed
Rights Holder	AICIT
Subject Keyword	Fingerprint Web crawling Near-duplicate Permutation Storage space Time Similarity score
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Performance and comparative analysis of the two contrary approaches for detecting near duplicate web documents in web crawling.

Performance and Comparative Analysis of the Two Contrary Approaches for Detecting Near Duplicate Web Documents in Web Crawling

Detecting Near-Duplicates for Web Crawling (2007)

To create a confusion matrix in respect of threshold being fixed for effective detection of near duplicate web documents in Web Crawling

Detecting near-duplicates for web crawling

Near-Duplicate Web Page Detection: An Efficient Approach Using Clustering, Sentence Feature and Fingerprinting

1.1 detection of near duplicate web documents-an overview.

A Novel and Efficient Approach For Near Duplicate Page Detection in Web Crawling

Achieving both high precision and high recall in near-duplicate detection

Near-duplicate web page detection: A comparative study of two contrary approaches

Similar Documents

Performance and comparative analysis of the two contrary approaches for detecting near duplicate web documents in web crawling.

Performance and Comparative Analysis of the Two Contrary Approaches for Detecting Near Duplicate Web Documents in Web Crawling

Detecting Near-Duplicates for Web Crawling (2007)

To create a confusion matrix in respect of threshold being fixed for effective detection of near duplicate web documents in Web Crawling

Detecting near-duplicates for web crawling

Near-Duplicate Web Page Detection: An Efficient Approach Using Clustering, Sentence Feature and Fingerprinting

1.1 detection of near duplicate web documents-an overview.

A Novel and Efficient Approach For Near Duplicate Page Detection in Web Crawling

Achieving both high precision and high recall in near-duplicate detection

Near-duplicate web page detection: A comparative study of two contrary approaches