Scheduling tasks close to their data and optimising resources utilisation are both crucial for the efficiency of MapReduce system. On the other hand, there is a conflict between fairness and efficiency. In this study, an efficient and dominant resource held time fairness (EHTF) scheduler is presented, in which the efficient utilisation of resources, data locality and fairness are addressed simultaneously. In EHTF scheduler, the authors introduce the concept of `coarse-grained fairness' to improve the efficiency of MapReduce system. For each scheduling, several tasks from different jobs can be assigned to the free slot without violating the coarse-grained fairness doctrine. To determine the best task from these several tasks in each scheduling step, a score model is proposed by taking into consideration both resources utilisation and data locality. The authors describe the design and implementation of EHTF scheduler. The authors' experimental results show that EHTF achieves more fairness and better throughput than Fair and Quincy schedulers.

+More

free slot throughput resource held time fairness ehtf scheduler mapreduce resources data locality score model fair and quincy schedulers coarsegrained fairness

Tonghong Li, Qi Qi, Lei Zhang, Jingyu Wang,Jianxin Liao,.Efficient and fair scheduler of multiple resources for MapReduce system. 10 (6),182-188.

Tonghong Li, et al."Efficient and fair scheduler of multiple resources for MapReduce system" 10

Tonghong Li, Qi Qi, Lei Zhang, Jingyu Wang,Jianxin Liao,"Efficient and fair scheduler of multiple resources for MapReduce system"