2021/05/22 by Mark Asch, François Bodin, Asch, Mark +9
Computer Science · Decision Sciences · #--- #Advanced Data Storage Technologies #Distributed #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Networking and Internet Architecture (cs.NI) #Parallel #Scientific Computing and Data Management #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2105.10680
openalex publication_date 2021/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Scientific communities naturally tend to organize around data ecosystems created by the combination of their observational devices, their data repositories, and the workflows essential to carry their research from observation to discovery. However, these legacy data ecosystems are now breaking down under the pressure of the exponential growth in the volume and velocity of these workflows, which are further complicated by the need to integrate the highly data intensive methods of the Artificial Intelligence revolution. Enabling ground breaking science that makes full use of this new, data saturated research environment will require distributed systems that support dramatically improved resource sharing, workflow portability and composability, and data ecosystem convergence. The Cybercosm vision presented in this white paper describes a radically different approach to the architecture of distributed systems for data-intensive science and its application workflows. As opposed to traditional models that restrict interoperability by hiving off storage, networking, and computing resources in separate technology silos, Cybercosm defines a minimally sufficient hypervisor as a spanning layer for its data plane that virtualizes and converges the local resources of the system's nodes in a fully interoperable manner. By building on a common, universal interface into which the problems that infect today's data-intensive workflows can be decomposed and attacked, Cybercosm aims to support scalable, portable and composable workflows that span and merge the distributed data ecosystems that characterize leading edge research communities today.