2026/05/21 by Philipp Thamm, Somayeh Mohammadi, Kathleen West +3 · 1 voice · 1 citation
Computer Science · Decision Sciences · Materials Science · #Cloud Computing and Resource Management #Energy (signal processing) #Energy consumption #Host (biology) #Machine Learning in Materials Science #Process (computing) #Scientific Computing and Data Management #Task (project management) #Workflow #Workflow management system #Workload #cs.DC
paper · pdf · doi:10.48550/arxiv.2605.22393
openalex publication_date 2026/05/21 · arxiv published 2026/05/21 · arxiv updated 2026/05/21 · openalex created_date 2026/05/23 · openalex updated_date 2026/07/29
Scientific workflows are pipelines of interdependent tasks. They are increasingly executed on shared Kubernetes clusters via workflow engines such as Nextflow. Their energy consumption matters for both cost and sustainability. It is necessary to examine and optimize workflow tasks individually, because they can be very heterogeneous. However, estimating task-level energy on clusters is difficult: Intel RAPL counters report only node-level energy, access to counters and host process information is typically restricted, and concurrent workloads introduce resource contention and measurement noise. We present Nf-PEAK, a containerized method to attribute CPU-package and DRAM energy to individual processes and Nextflow tasks. Nf-PEAK (i) identifies workflow pods, (ii) maps pods to host processes via cgroup metadata, (iii) samples RAPL and per-process performance counters, and (iv) applies a non-linear energy-credit model before aggregating results at task level. On a Kubernetes cluster, we evaluate three nf-core workflows under controlled co-located CPU load. Nf-PEAK reaches an average Mean Absolute Percentage Error of 6.6% in isolated runs and 10.9% when an unrelated workload saturates 8 of 32 hardware threads per node, and remains stable across 2, 3, 4, and 8 nodes. Compared to the state-of-the-art Kubernetes tool Kepler, Nf-PEAK yields lower error on average, particularly under co-located load.