Status of Computing at PDR

1. Status of Computing at PDR B.E. Glendenning (NRAO), G. Raffi (ESO)

2. Phase 2 Scope Scope: Observing preparation and support through archival access to automatically produced (pipeline) images Computing IPT

3. Management Planning • Computing Plan (framework) and 15 subsystem agreements prepared • Computing Plan submitted to document approval process (comments from JAO to be accounted for) • Subsystem agreements approved by Computing IPT • Management model: • Agreement “contracts” followed by subsystem scientists (scope) and software management (schedule, cost) • Initial PDR, yearly CDRs and release planning • Synchronized 6-month releases across all subsystems • SSR requirements mapped to releases for progress tracking Computing IPT

4. Package Feature name Short description SSR requirement Numbers To be completed at Release (R1.0-3.0) Status at Milestone T0 (N,P,C) ObsProject CheckObsDatabase Check observation database for conflict 3.0R10 R3.0 N ObsProject Local save Save/recover programs to/from local disk in human readable form 3.0-R11, 3.1R3, 3.1R12 R1.0 N Management Planning (2) • Readiness reviews, acceptance tests, and delivery to interim operations in 2007 • “Data flow” items get a second development period during interim operations • review requirements after first experience with operations • Not subject to current agreements – institutional responsibility could shift Computing IPT

5. AIPS++ Evaluation • AIPS++ (along with the ESO Next Generation Archive System) is a major package proposed for use by ALMA • Both to ensure at least one data reduction package is available to users and in implementing ALMA systems (notably the Pipeline) • AIPS++ is a very controversial package (long development period, has not yet received wide acceptance) • ALMA Computing jointly with SSR committee, has arranged several evaluations (next presentation by R.Lucas) • Audit of capabilities based on documentation • AIPS++/IRAM tests to test suitability for millimeter data • Benchmarking tests • NRAO/AIPS++ Consortium have set up a technical review Computing IPT

6. AIPS++ technical review • Held at the National Radio Astronomy Observatory in Socorro on March 5-6, 2003 • Review panel: Roger Brissenden (CfA), Hilton Lewis (Keck), Chair, Andrew Lumsdaine (U. Indiana), Dave McConnell (ATNF), Steve Scott (OVRO), David Silva (ESO), Doug Tody (NRAO) and Rick White (STSCI). • Report submitted to NRAO Director and circulated to AIPS++ Executive Committee. Computing IPT

7. AIPS++ review summary - major recommendations (1/2) Note: All here given recommendations are a short summary of the full text • Change the focus of AIPS++ • from a system aiming to be a general-purpose radio astronomy package to a system that is developed to meet the strategic goals of the Consortium. • Modify the Development Process • The AIPS++ master schedule should be consistent with the major project schedules (e.g., GBT, ALMA, EVLA). • Strengthen the Project Management and the Project Team • Further Strengthen the Project Scientist Role Computing IPT

8. AIPS++ review summary - major recommendations (2/2) • Short-Term Priorities (next 12 months) • Significant improvements in reliability and stability • Performance improvements approaching the performance of AIPS • Synthesis imaging capability covering many common use cases of the VLA • Reduced emphasis on graphical user interface • Proceed With the Proposed Technology Changes • The Panel finds that the case for changing the technology to support future development of AIPS++ is compelling. The proposed architectural change to the framework has technical merit and a proof of concept should be pursued at this time, in parallel with current development. However, it is critically important that this not be allowed to distract the AIPS++ team from completing the current development. Computing IPT

9. Current AIPS++ Architecture Computing IPT

10. Proposed AIPS++ Architecture Computing IPT

11. ALMA Proof of Concept • AIPS++ proposes to use ALMA Common Software (ACS) technology and interoperability designs (exact technical relation being defined) • CORBA, Python, Container/Component, Java • To be confirmed by AIPS++ Executive Committee. • This should allow AIPS++ to interoperate much more readily with other software packages and (particularly) ALMA subsystems • In particular it becomes easier in principle for the Pipeline to adopt engines from other packages • provided some data model interoperability exists Computing IPT

12. Preliminary Design Review • March 18-20, Tucson • Panel • R. Doxsey (CHAIR) (STScI, Head HST Mission Office) • P. Quinn (ESO, Head of Data Management and Operations Division) • N. Radziwill (NRAO, Head of GBT Software Development) • J. Richer (Cambridge, ALMA/UK Project Scientist, Science Advisory Committee) • D. Sramek (NRAO, ALMA Systems Engineering IPT Lead) • S. Wampler (NSO, Software Architect) • A. Wootten (NRAO, ALMA/North America Project Scientist)  Computing IPT

13. Hot Topics presented at PDR (1/2) • Missing operational requirements Science committee is being set up to answer operational questions (Lucas and Schwarz are Computing representatives there) • Conflict of priority at ATFIf the issue of operators is solved, we do not see here a conflict in the long run, although in 2003 we have quite some pressure. • Splitting of Archive and Pipeline operation between OSF and SOC Long term baseline concept (not for Interim Science) : • Archive will be in two copies: OSF and SOC. • Calibration and Quick look running at the OSF (due to their timing) while Science Pipeline will run at the SOC. Computing IPT

14. Hot Topics presented at PDR (2/2) • Only OSF in 5 years planSantiago (SOC) and RSCs commissioning is not included. • Santiago Center (SOC) commissioned in 2008possibly with a limited bandwidth (e.g. 10 Mb/s -> 100 Mb/s) to check operations model and reliability asap. • Santiago link in 2007 Good to monitor OSF and gain experience with link. Could have limited bandwidth. • RSC installation (once or twice?) Baselineplan: prototype system to run the array, delivering the final (pipeline and archive) systems as late in the construction period as possible: one in Chile and one to each RSC. Computing IPT

15. Possible issues • Staffing (especially at NRAO) not yet at plan, although full recruitment now in progress • Data rate increase sought by Science Software Requirements group, based on assumption that Base Correlator change will be approved. • Correlator enhancements need corresponding increase in data rates to be of most importance • Cost implications need to be determined Computing IPT

16. PDR Recommendations (1/2) • Unofficial top-level results, based on exit debriefing by panel to IPT Management and M. Rafal • Official report and IPT response by mid-April • Well prepared PDR; at or ahead of where similar projects have been at this stage • The ALMA Project needs to develop an understanding of operations, and the Computing IPT needs to fold this in to their planning and priorities. This might result in reprioritization of SSR requirements • ALMA project needs to define clearly the steps necessary for getting to, and implementing, Interim Operations and the consequent requirements for computing support Computing IPT

17. PDR Recommendations (2/2) • The interaction with AIPS++ requires careful management. Operational (Pipeline) requirements on AIPS++ need further development. • Testing effort seems satisfactory; management will need to follow-up to ensure subsystem unit tests are in fact carried out Computing IPT

18. Computing Goals for 2003 • Support prototype antenna tests • Use AIPS++ for ALMA Computing work (Pipeline, Off-line, DRUI) while completing feasibility phase: - AIPS++ benchmarking - Support use of ACS in AIPS++ Proof of Concept  Review progress on AIPS++ in 1 year (CDR2) • Produce 1st integrated ALMA software Release • Waiting for ALMA Operations Plan (software aspects) 2004-01-01 Computing IPT