Inicio  /  Algorithms  /  Vol: 12 Par: 1 (2019)  /  Artículo
ARTÍCULO
TITULO

MAPSkew: Metaheuristic Approaches for Partitioning Skew in MapReduce

Matheus H. M. Pericini    
Lucas G. M. Leite    
Francisco H. De Carvalho-Junior    
Javam C. Machado and Cenez A. Rezende    

Resumen

MapReduce is a parallel computing model in which a large dataset is split into smaller parts and executed on multiple machines. Due to its simplicity, MapReduce has been widely used in various applications domains. MapReduce can significantly reduce the processing time of a large amount of data by dividing the dataset into smaller parts and processing them in parallel in multiple machines. However, when data are not uniformly distributed, we have the so called partitioning skew, where the allocation of tasks to machines becomes unbalanced, either by the distribution function splitting the dataset unevenly or because a part of the data is more complex and requires greater computational effort. To solve this problem, we propose an approach based on metaheuristics. For evaluating purposes, three metaheuristics were implemented: Simulated Annealing, Local Beam Search and Stochastic Beam Search. Our experimental evaluation, using a MapReduce implementation of the Bron-Kerbosch Clique Algorithm, shows that the proposed method can find good partitionings while better balancing data among machines.

 Artículos similares

       
 
Sani. I. Abba, Jamilu Usman, Ismail Abdulazeez, Dahiru U. Lawal, Nadeem Baig, A. G. Usman and Isam H. Aljundi    
The need for reliable, state-of-the-art environmental investigations and pioneering approaches to address pressing ecological dilemmas and to nurture the sustainable development goals (SDGs) cannot be overstated. With the power to revolutionize desalinat... ver más
Revista: Water

 
Mohammad H. Nadimi-Shahraki, Zahra Asghari Varzaneh, Hoda Zamani and Seyedali Mirjalili    
Feature selection is an NP-hard problem to remove irrelevant and redundant features with no predictive information to increase the performance of machine learning algorithms. Many wrapper-based methods using metaheuristic algorithms have been proposed to... ver más
Revista: Applied Sciences

 
Kübra Kiziloglu and Ümit Sami Sakalli    
Airlines face the imperative of resource management to curtail costs, necessitating the solution of several optimization problems such as flight planning, fleet assignment, aircraft routing, and crew scheduling. These problems present some challenges. Th... ver más
Revista: Aerospace

 
Jeffrey O. Agushaka and Absalom E. Ezugwu    
A situation where the set of initial solutions lies near the position of the true optimality (most favourable or desirable solution) by chance can increase the probability of finding the true optimality and significantly reduce the search efforts. In opt... ver más
Revista: Applied Sciences

 
Manar Ahmed Hamza, Hamed Alqahtani, Dalia H. Elkamchouchi, Hussain Alshahrani, Jaber S. Alzahrani, Mohammed Maray, Mohamed Ahmed Elfaki and Amira Sayed A. Aziz    
Unmanned aerial vehicles (UAVs) have significant abilities for automatic detection and mapping of urban surface materials due to their high resolution. It requires a massive quantity of data to understand the ground material properties. In recent days, c... ver más
Revista: Applied Sciences