vix.ing · top · new · best · stats · spec

Using script generators for pipeline prototyping

2021/12/19 by D. Petry, Petry, Dirk
Engineering · Physics and Astronomy · #Astronomical Observations and Instrumentation #Astronomy and Astrophysical Research #FOS: Physical sciences #Instrumentation and Methods for Astrophysics (astro-ph.IM)

paper · pdf · doi:10.48550/arxiv.2112.10050

openalex publication_date 2021/12/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Fully automated astronomical data calibration and imaging pipelines are difficult to develop without a good prototyping method which permits to bridge the time between observatory commissioning and the moment when the special features and possible problems of the data and their processing are fully understood. In this paper I present a method which has worked well for the ALMA observatory and which is sufficiently general to be transferable to most other projects. In short, the idea is to use a three-level data analysis software design (scriptable toolkit, script generator, automated pipeline) and a corresponding timing of the software development which is ramping up the effort in three stages starting at the beginning of construction, at the beginning of commissioning, and at the end of commissioning respectively. The important design pattern which I would like to underline here is the use of script generators as prototypes for the automated pipeline.

Related