NDLI: Crawling Result Pages for Data Extraction Based on URL Classification

Content Provider	IEEE Xplore Digital Library
Author	Tiezheng Nie Zhenhua Wang Yue Kou Rui Zhang
Copyright Year	2010
Abstract	In Web database integration, crawling data pages is important for data extraction. The fact that data are contained by multiple result pages increases the difficulty of accessing data for integration. Thus, it is necessary to accurately and automatically crawl query result pages from Web database. To address this problem, we propose a novel approach based on URL classification to effectively identify result pages. In our approach, we compute the similarity between URLs of hyperlinks in result pages and classify them into four categories. Each category maps to a set of similar web pages, which separate result pages from others. Then, we use the page probing method to verify the correctness of classification and improve the accuracy of crawled result pages. The experimental result demonstrates that our approach is effective for identifying the collection of result pages in Web database, and can improve the quality and efficiency of data extraction.
Starting Page	79
Ending Page	84
File Size	248699
Page Count	6
File Format	PDF
ISBN	9781424484409
DOI	10.1109/WISA.2010.14
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-08-20
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Accuracy Databases Web pages Clustering algorithms Classification Classification algorithms Result pages Data mining Component URL Data extraction
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

UPCA: An efficient URL-Pattern based algorithm for accurate web page classification

Analyzing web layout structures using graph mining

Data Extraction from Semi-structured Web Pages by Clustering

Web site classification based on URL and content: Algerian vs. non-Algerian case

Phishing URL Detection Using URL Ranking

A New Framework for Domain-Specific Hidden Web Crawling Based on Data Extraction Techniques

Customized Category Based Clustering of URLs

Web Usage Mining Based on WAN Users' Behaviors

Bottom-up region extractor for semi-structured web pages

Crawling Result Pages for Data Extraction Based on URL Classification

Similar Documents

UPCA: An efficient URL-Pattern based algorithm for accurate web page classification

Analyzing web layout structures using graph mining

Data Extraction from Semi-structured Web Pages by Clustering

Web site classification based on URL and content: Algerian vs. non-Algerian case

Phishing URL Detection Using URL Ranking

A New Framework for Domain-Specific Hidden Web Crawling Based on Data Extraction Techniques

Customized Category Based Clustering of URLs

Web Usage Mining Based on WAN Users' Behaviors

Bottom-up region extractor for semi-structured web pages

Crawling Result Pages for Data Extraction Based on URL Classification