\documentclass[11pt,twoside]{article} % Leave intact \usepackage{adassconf} \begin{document} % Leave intact \paperID{P4-8} \title{Planck/LFI DPC Software Integration plan} \author{A.\ Zacchei\altaffilmark{1}, C.\ Vuerli\altaffilmark{1}, N.\ Lama\altaffilmark{1}, F.\ Pasian\altaffilmark{1}} \affil{INAF-Osservatorio Astronomico di Trieste,Via Tiepolo 11 34131 Trieste Italy, Email: zacchei@ts.astro.it} \contact{Andrea Zacchei} \email{zacchei@ts.astro.it} \paindex{Zacchei, A.} \aindex{Vuerli, C.} \aindex{Lama, N.} \aindex{Pasian, F.} \authormark{Zacchei, Vuerli, Lama, \& Pasian} \keywords{Planck: software: integration, software: engineering} \begin{abstract} % Leave intact A very spread software project needs to be well defined through software integration and development plan to avoid extra work in the pipeline creation phase. Here we will describe the rationale in the case of the {\sc Planck}/LFI DPC project and what was designed and developed to build the integration environment. \end{abstract} \section{Introduction} LFI is one of the two instruments installed on board \setcounter{footnote}{0} \htmladdnormallinkfoot{{\sc Planck}}{http://astro.estec.esa.nl/Planck/}, the M3 mission of ESA's Horizon 2000+ programme. Data reduction and analysis will be performed in pipeline mode at the Data Processing Center (DPC). The development of the DPC software is being performed in a collaborative way across a consortium spread across over 20 institutes in a dozen countries. Individual scientists belonging to a Software Prototyping Team develop prototype code, which is then delivered to the LFI DPC team. The latter is responsible to integrate the code, so as to produce the pipeline software to be used during operations. Integrated source code is fed back to the originators. This development takes advantage of tools defined within the {\sc Planck} IDIS~\footnote{Integrated Data an Information System} collaboration. A software policy has been defined, with the aim of allowing the DPC to run the best possible algorithms within its pipeline, while fostering collaboration inside the LFI Consortium and across {\sc Planck}, and preserving at the same time the intellectual property of the code authors on the processing algorithms devised. \section{The Data Processing Center (DPC)} The {\sc Planck} DPCs are responsible for the archiving and the delivery of the following scientific data products: \begin{itemize} \item calibrated time series data, after removal of systematic features and attitude reconstruction; \item photometrically and astrometrically calibrated maps of the sky in the observed bands; \item sky maps of the main astrophysical components; \item catalogs of sources detected; \item CMB power spectrum. \end{itemize} DPC processing can be logically divided in five levels: simulation (level S), telemetry processing and interface with the MOC~\footnote{Mission Operation Center} (level 1), data reduction and calibration (level 2), component separation and optimization (level 3), generation of final products (level 4). DPC processing is harmonically arranged in a way that a data pipeline, driven by a process coordinator, is set up. The LFI pipeline is operated at OAT for levels S, 1, 2 and 3. Level 4 is hosted at MPI (Garching). To harmonize inter-consortia and intra-consortium activities, a common set of tools called IDIS for the sharing of information, documents, common software and data is being designed and built. \section{Development/Integration/Release General Scheme} In figure 1 the general structure of the {\sc Planck} LFI development cycle is shown. Scientists' contributions to algorithms development stems from data processing requirements. When they produce a frozen version (prototype), this code will be engineered, optimized and integrated at the DPC as a module to be harmonized into the pipeline; thus a first release will be issued. Scientific tests on the final product (release) could modify the main requirements, so this implies that we must take into account the pipeline changes that will produce different pipeline issues. \begin{figure} [h] \centering \epsscale{.50} \plotone{P4-8_1.eps} \caption{ LFI Development Cycle} \label{P4-8:Fig 1} \end{figure} \section{Software Integration Cycle} Prototype S/W developed by scientists is not expected to have the robustness, documentation and maintainability that the DPC pipeline S/W is expected to guarantee. For this purpose, an integration team (the DPC team) engineers the prototypes and pipelines elements in operative robust DPC S/W. Efforts will be also specifically concentrated on the possibility of correct recovery in the case of DPC hardware failures, and on the concept of providing ``warm'' or ``cold'' backup capabilities at operations time. The initial delivery of prototype software (code sources, scripts, make files, etc.) from both the software prototyping team and the modeling and simulations team to the DPC is accompanied by documentation forming the basis of the URDs for the scientific pipeline (one for each level). The LFI DPC integration team is responsible for integrating the code, so as to produce pipeline software satisfying all requirements defined in the URDs. The pipeline integration phase follows the ESA PSS-05 software development standards, including Product and Quality Assurance. The performance results and testing results of the integrated software are provided to the software prototyping team and the simulations and modelling team at the time of the official release. Integrated source code is provided back to the originators, and not intended for general distribution. \begin{figure} [h] \epsscale{.60} \plottwo{P4-8_2.eps}{P4-8_3.eps} \caption{ a) LFI Software Integration Cycle, b) LFI Software Verification and Validation Cycle} \label{P4-8:Fig 2} \end{figure} \section{Software Verification and Validation} Each module produced by scientists, before the integration phase, must pass the validation plan. The scheme of the validation (Fig 2b) at acceptance point applied corresponds to the following list of operations: \begin{itemize} \item a set of one or more tests is applied to the code, \item the set of tests generates a set of test-reports to be synthesized in a global-test-report, \item if a failure occurs to one of the tests, the block is rejected and sent to debugging; after debugging the full test will be repeated, \item if the validation report is positive the block is accepted and the validation-set is stored in the validation-repository; a final validation-report is added to the validation-document. \end{itemize} \section{Software Repository} The DPC software developed within LFI, having either prototype or integrated status, is subject to an access policy. This policy has the aim of allowing the DPC to run the best possible algorithms within its pipeline, while fostering collaboration inside the LFI Consortium and across {\sc Planck}, and preserving at the same time the intellectual property of the code authors on the processing algorithms devised. The access policy can be summarized as follows: \begin{itemize} \item LFI software can be physically accessed through the LFI software repository, where the access policy is enforced; the LFI Software Repository is handled by the Concurrent Versions System (CVS) and the Configuration Management Tool (CMT), chosen as the software configuration tool within the IDIS collaboration; \item prototype software delivered to the DPC team for integration is not necessarily available to the rest of the Consortium; the definition of the access list is at the discretion of the originators and of the DPC team; \item once integrated, the software is fed back to the originators but not to others; \item access to LFI software integrated in the LFI DPC is always read-only; \item a log of user access (both read-only and read/write) to the LFI software repository is kept; \item to allow evolution of both prototype and integrated code, originators are encouraged to share their codes with other {\sc Planck} scientists; these are encouraged to feed back their changes and improvements to the DPC through the developers of the original code to avoid divergences; \item collaboration among code developers is encouraged: access to repository directories is granted through project-based groups. \end{itemize} \section{Conclusions} The entire software development and integration cycle was already tested and applied to the production of the first LFI pipeline (the Bread-Board Model). The principles used to develop, integrate and engineer the code necessary for the {\sc Planck} LFI pipeline can easily be applied to any big software development project. \acknowledgments The LFI is funded by the national space agencies of the Institutes of the Consortium (ASI for the Italian participation). \begin{references} \reference Mandolesi N., et al., 1998, {\em {\sc Planck}/LFI, A Proposal Submitted to the ESA}. \reference Pasian F., (2002), Mem S. A. It. 2003, Vol 74, 502,{\em {\sc Planck}/LFI: the Data Processing Centre}. \end{references} \end{document} % Leave intact