NDLI: Practical assessment of remotely communicated end-point servers of leading web crawlers

Content Provider	IEEE Xplore Digital Library
Author	Shoukat, I.A. Ansari, W.S. Iftikhar, M. Taheri, J. Zomaya, A.Y.
Copyright Year	2012
Description	Author affiliation: College of Computer & Information Sciences, King Saud University, Riyadh, Kingdom of Saudi Arabia (Shoukat, I.A.; Ansari, W.S.; Iftikhar, M.; Zomaya, A.Y.) \|\| Centre for Distributed and High Performance Computing, University of Sydney, Australia (Taheri, J.)
Abstract	Exploration of future dreams inclined us in information quest over the global literature. Web crawlers enriched with technological phenomena for such purpose to millions of seekers. The usage of crawling tools implicates justification for their consequences as searching technology that facilitates initiative passion to target new information. Several web crawlers are available with different technological mechanism of remotely communicated servers; but their efficiency in terms of different performance metrics has been an open question. Previous studies provide inadequate comparison with limited analysis factors against these servers. Here we put an effort and provided new results by considering several performance metrics under practical estimation of technical factors with maximum capacity to evaluate foremost crawling technology based end-point servers. We extensively studied remote servers running behind Google, Yahoo, MSN, Gigablast, Mamma and Excite. We reflected their performance in terms of response time, load, health status, down time, uptime, availability and reliability. These new results would provide a deep insight to every researcher regarding to searching usages and related technological enhancement in future.
Starting Page	219
Ending Page	224
File Size	943053
Page Count	6
File Format	PDF
ISBN	9781467319201
e-ISBN	9781467319218
DOI	10.1109/JCSSE.2012.6261955
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-05-30
Publisher Place	Thailand
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Measurement Availability Google Performance Evaluation Search Engines QoS Web Crawlers Crawlers Search engines Information Retrieval and Filtering Servers Time factors
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A performance evaluation of semantic based search engines and keyword based search engines

WSExpress: A QoS-aware Search Engine for Web Services

Aggregate discovery and retrieval of contents from authenticated interface

"Google by reformulation": modeling search as successive refinement

Retrieval of Volatile Information from World Wide Web Using Migrants

Similarity score for information filtering thresholds

Leveraging Google Web Search Technology to Find Web-Based Learning Objects

Exploring relevance assessment using crowdsourcing for faceted and ambiguous queries

Filtering noisy parallel corpora of web pages

Practical assessment of remotely communicated end-point servers of leading web crawlers

Similar Documents

A performance evaluation of semantic based search engines and keyword based search engines

WSExpress: A QoS-aware Search Engine for Web Services

Aggregate discovery and retrieval of contents from authenticated interface

"Google by reformulation": modeling search as successive refinement

Retrieval of Volatile Information from World Wide Web Using Migrants

Similarity score for information filtering thresholds

Leveraging Google Web Search Technology to Find Web-Based Learning Objects

Exploring relevance assessment using crowdsourcing for faceted and ambiguous queries

Filtering noisy parallel corpora of web pages

Practical assessment of remotely communicated end-point servers of leading web crawlers