NDLI: MOMC: Multi-objective and Multi-constrained Scheduling Algorithm of Many Tasks in Hadoop

Content Provider	IEEE Xplore Digital Library
Author	Voicu, Cristiana Pop, Florin Dobre, Ciprian Xhafa, Fatos
Copyright Year	2014
Abstract	Even though scheduling in a distributed system was debated for many years, the platforms and the job types are changing everyday. This is why we need special algorithms based on new applications requirements, especially when a application is deployed in a Cloud environment. One of the most important framework used for large-scale data processing in Clouds is Hadoop and its extensions. Hadoop framework comes with default algorithms like FIFO, Fair Scheduler or Capacity Scheduler, and Hadoop on Demand. These scheduling algorithms are focused on a different and single constraint. It is hard to satisfy multiple constraints and to have a lot of objectives in the same time. After summarizing the most common schedulers, showing the need of each one in the moment it appeared on the market, this paper presents MOMC, a multi-objective and multi-constrained scheduling algorithm of many tasks in Hadoop. MOMC implementation focuses on two objectives: avoiding resource contention and having an optimal workload of the cluster, and two constraints: deadline and budget. To compare the algorithms based on different metrics, we use Scheduling Load Simulator, which is integrated in Hadoop framework and helps the developers to spend less time on testing. As killer application that generate many tasks we have chosen processing task for the Million Song Dataset, which is a set of data contains metadata for one million commercially-available songs.
Starting Page	89
Ending Page	96
File Size	684179
Page Count	8
File Format	PDF
ISBN	9781479941711
DOI	10.1109/3PGCIC.2014.40
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-11-08
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Measurement Scheduling algorithms Cloud Computing Clustering algorithms Map Reduce Task Scheduling Hadoop Containers Big Data Scheduling History
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Multi-threading based Map Reduce tasks scheduling

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

Task Scheduling Algorithm with Fault Tolerance for Cloud

BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

Cloud Computing—Task scheduling based on genetic algorithms

Efficient Map/Reduce-Based DBSCAN Algorithm with Optimized Data Partition

In-Map/In-Reduce: Concurrent Job Execution in MapReduce

Research of Cloud Computing Based on the Hadoop Platform

A Deadline-Constrained Multi-Objective Task Scheduling Algorithm in Mobile Cloud Environments

MOMC: Multi-objective and Multi-constrained Scheduling Algorithm of Many Tasks in Hadoop

Similar Documents

Multi-threading based Map Reduce tasks scheduling

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

Task Scheduling Algorithm with Fault Tolerance for Cloud

BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

Cloud Computing—Task scheduling based on genetic algorithms

Efficient Map/Reduce-Based DBSCAN Algorithm with Optimized Data Partition

In-Map/In-Reduce: Concurrent Job Execution in MapReduce

Research of Cloud Computing Based on the Hadoop Platform

A Deadline-Constrained Multi-Objective Task Scheduling Algorithm in Mobile Cloud Environments

MOMC: Multi-objective and Multi-constrained Scheduling Algorithm of Many Tasks in Hadoop