Loading...
Please wait, while we are loading the content...
Similar Documents
Natjam: design and evaluation of eviction policies for supporting priorities and deadlines in mapreduce clusters.
| Content Provider | CiteSeerX |
|---|---|
| Author | Cho, Brian Rahman, Muntasir Chajed, Tej Gupta, Indranil Abad, Cristina Roberts, Nathan Lin, Philbert |
| Abstract | This paper presents Natjam, a system that supports arbitrary job priorities, hard real-time scheduling, and efficient preemption for Mapreduce clusters that are resource-constrained. Our contributions include: i) exploration and evaluation of smart eviction policies for jobs and for tasks, based on resource usage, task runtime, and job deadlines; and ii) a work-conserving task preemption mechanism for Mapreduce. We incorporated Natjam into the Hadoop YARN scheduler framework (in Hadoop 0.23). We present experiments from deployments on a test cluster, Emulab and a Yahoo! commercial cluster, using both synthetic workloads as well as Hadoop cluster traces from Yahoo!. Our results reveal that Natjam incurs overheads as low as 7%, and is preferable to existing approaches. |
| File Format | |
| Access Restriction | Open |
| Subject Keyword | Mapreduce Cluster Eviction Policy Hard Real-time Scheduling Hadoop Cluster Trace Arbitrary Job Priority Hadoop Yarn Scheduler Framework Smart Eviction Policy Task Runtime Work-conserving Task Preemption Mechanism Job Deadline Yahoo Commercial Cluster Synthetic Workload Present Experiment Efficient Preemption Test Cluster Resource Usage |
| Content Type | Text |
| Resource Type | Article |
