2022/10/22 by Farid Zakaria, Zakaria, Farid, Thomas R. W. Scogland +6 · 1 voice
Computer Science · #Advanced Data Storage Technologies #Cloud Computing and Resource Management #Distributed and Parallel Computing Systems #cs.MS #cs.SE
paper · pdf · doi:10.48550/arxiv.2211.05118
openalex publication_date 2022/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
High Performance Computing~(HPC) software stacks have become complex, with the dependencies of some applications numbering in the hundreds. Packaging, distributing, and administering software stacks of that scale is a complex undertaking anywhere. HPC systems deal with esoteric compilers, hardware, and a panoply of uncommon combinations. In this paper, we explore the mechanisms available for packaging software to find its own dependencies in the context of a taxonomy of software distribution, and discuss their benefits and pitfalls. We discuss workarounds for some common problems caused by using these composed stacks and introduce Shrinkwrap: A solution to producing binaries that directly load their dependencies from precise locations and in a precise order. Beyond simplifying the use of the binaries, this approach also speeds up loading as much as 7x for a large dynamically-linked MPI application in our evaluation.