Engineering a large-scale data analytics and array computing library for research: Heat

2025/03/11 by Hoppe, Fabian, Gutiérrez Hermosillo Muriedas, Juan Pedro, Tarnawa, Michael +6
#Big Data #Data Science #Data analytics #GPUs #High-Performance Computing #Machine learning #Multi-dimensional Arrays #Parallel Computing #Research Software

paper · doi:10.14279/eceasst.v83.2626

Abstract

Heat is a Python library for massively-parallel and GPU-accelerated array computing and machine learning. It is developed by researchers for researchers, with the ultimate goal to make multi-dimensional array processing and machine learning for scientists (almost) as easy on a supercomputer as it is on a workstation with NumPy or scikit-learn. This paper highlights the relevance of this project to the research software engineering community by giving a short, but illustrative overview of Heat and discusses its role in the context of related libraries with a specific focus on its research software aspects.

Related

• ds-array: A Distributed Data Structure for Large Scale Machine Learning
• Landscape of High-performance Python to Develop Data Science and Machine Learning Applications
• Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence
• Linking Scientific Instruments and HPC: Patterns, Technologies, Experiences
• Mojo: MLIR-Based Performance-Portable HPC Science Kernels on GPUs for the Python Ecosystem