2019/11/15 by Ali Mohammed, Aurélien Cavelan, Mohammed, Ali +7
Computer Science · #Parallel Computing and Optimization Techniques #Distributed and Parallel Computing Systems #Advanced Data Storage Technologies
paper · pdf · doi:10.48550/arxiv.1911.06714
Scientific applications are often complex, irregular, and\ncomputationally-intensive. To accommodate the ever-increasing computational\ndemands of scientific applications, high-performance computing (HPC) systems\nhave become larger and more complex, offering parallelism at multiple levels\n(e.g., nodes, cores per node, threads per core). Scientific applications need\nto exploit all the available multilevel hardware parallelism to harness the\navailable computational power. The performance of applications executing on\nsuch HPC systems may adversely be affected by load imbalance at multiple\nlevels, caused by problem, algorithmic, and systemic characteristics.\nNevertheless, most existing load balancing methods do not simultaneously\naddress load imbalance at multiple levels. This work investigates the impact of\nload imbalance on the performance of three scientific applications at the\nthread and process levels. We jointly apply and evaluate selected dynamic loop\nself-scheduling (DLS) techniques to both levels. Specifically, we employ the\nextended LaPeSD OpenMP runtime library at the thread level and extend the\nDLS4LB MPI-based dynamic load balancing library at the process level. This\napproach is generic and applicable to any multiprocess-multithreaded\ncomputationally-intensive application (programmed using MPI and OpenMP). We\nconduct an exhaustive set of experiments to assess and compare six DLS\ntechniques at the thread level and eleven at the process level. The results\nshow that improved application performance, by up to 21%, can only be achieved\nby jointly addressing load imbalance at the two levels. We offer insights into\nthe performance of the selected DLS techniques and discuss the interplay of\nload balancing at the thread level and process level.\n