Seemless Utilization of Heterogeneous XSede Resources to Accelerate Processing of a High Value Functional Neuroimaging Dataset

2018/04/04 by Krieger, Don, Shepard, Paul, Zusman, Ben +2
#FOS: Biological sciences #Neurons and Cognition (q-bio.NC)

paper · doi:10.48550/arxiv.1805.00786

Abstract

We describe the technical effort used to process a voluminous high value human neuroimaging dataset on the Open Science Grid with opportunistic use of idle HPC resources to boost computing capacity more than 5-fold. With minimal software development effort and no discernable competitive interference with other HPC users, this effort delivered 15,000,000 core hours over 7 months.

Related

• A Community-Developed Open-Source Computational Ecosystem for Big Neuro Data
• Hierarchical storage management in user space for neuroimaging applications
• Shared High Value Research Resources: The CamCAN Human Lifespan Neuroimaging Dataset Processed on the Open Science Grid
• Performance Evaluation of Big Data Processing Strategies for\n Neuroimaging
• A performance comparison of Dask and Apache Spark for data-intensive neuroimaging pipelines