NDLI: Adaptation of Apriori to MapReduce to Build a Warehouse of Relations between Named Entities across the Web

Content Provider	IEEE Xplore Digital Library
Author	Cryans, J.-D. Ratté, S. Champagne, R.
Copyright Year	2010
Abstract	The Semantic Web has made possible the use of the Internet to extract useful content, a task that could necessitate an infrastructure across the Web. With Hadoop, a free implementation of the MapReduce programming paradigm created by Google, we can treat these data reliably over hundreds of servers. This article describes how the Apriori algorithm was adapted to MapReduce in the search for relations between entities to deal with thousands of Web pages coming from RSS feeds daily. First, every feed is looked up five times per day and each entry is registered in a database with MapReduce. Second, the entries are read and their content sent to the Web service OpenCalais for the detection of named entities. For each Web page, the set of all itemsets found is generated and stored in the database. Third, all generated sets, from first to last, are counted and their support is registered. Finally, various analytical tasks are executed to present the relationships found. Our tests show that the third step, executed over 3,000,000 sets, was 4.5 times faster using five servers than using a single machine. This approach allows us to easily and automatically distribute treatments on as many machines as are available, and be able to process datasets that one server, even a very powerful one, would not be able to manage alone. We believe that this work is a step forward in processing semantic Web data efficiently and effectively.
Starting Page	185
Ending Page	189
File Size	348053
Page Count	5
File Format	PDF
ISBN	9781424460816
e-ISBN	9781424460823
DOI	10.1109/DBKDA.2010.34
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-04-11
Publisher Place	France
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	web mining association rules Data mining Semantic Web Databases Web services Itemsets Web pages MapReduce paradigm Apriori algorithm Internet Feeds Web server Testing
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Incremental FP_Growth Mining Algorithm Based on Web Information Extraction

MR-Apriori: Association Rules algorithm based on MapReduce

MapReduce as a programming model for association rules algorithm on Hadoop

The Research of Improved Apriori Algorithm for Mining Association Rules

The research of improved apriori algorithm for mining association rules

A new improvement of Apriori Algorithm for mining association rules

Linguistic Object-Oriented Web Mining

An efficient algorithm for web recommendation systems

A Method of Improvement and Optimization on Association Rules Apriori Algorithm

Adaptation of Apriori to MapReduce to Build a Warehouse of Relations between Named Entities across the Web

Similar Documents

Incremental FP_Growth Mining Algorithm Based on Web Information Extraction

MR-Apriori: Association Rules algorithm based on MapReduce

MapReduce as a programming model for association rules algorithm on Hadoop

The Research of Improved Apriori Algorithm for Mining Association Rules

The research of improved apriori algorithm for mining association rules

A new improvement of Apriori Algorithm for mining association rules

Linguistic Object-Oriented Web Mining

An efficient algorithm for web recommendation systems

A Method of Improvement and Optimization on Association Rules Apriori Algorithm

Adaptation of Apriori to MapReduce to Build a Warehouse of Relations between Named Entities across the Web