DECam Community Pipeline Review

1. DECam Community Pipeline Review Closeout Presentation DES Council of Directors’ Review August 30-31, 2010 NCSA, Urbana IL

2. Executive Summary - 1 • CP Implementation Plan matched well and adequately traced to Requirements • Strongly endorse structured management plan and progress tracking approach. Encourage adopting it for DESDM as soon as possible • Define, run and verify more “CP-like” data through DESDM as soon as possible to identify modifications that haven’t been anticipated • Only 4-5% of tasks are plausible descopes. Consider phased deployment approach matched to testing. Postpone lower priority items until post-commissioning phase.

3. Executive Summary - 2 • Recommend early involvement and integration of NOAO team into CP testing and deployment to minimize “surprises” at time of delivery and focus documentation effort • Testing and Acceptance plans are in early stage of development. Develop plan for getting to state of fully populated acceptance test matrix including test data sets and pass/fail criteria (including data quality) • Concern about level of resources (both workforce and hardware) at NOAO

4. Charge topic 1:Does the CP implementation plan address the requirements described the Community Pipeline Requirements and Specifications documents? • The detailed Implementation plan is explicitly constructed to address the listed requirements, and it appears to be basically adequate for the task. • The CP implementation team should look at a prioritization of the CP requirements in consultation with NOAO to ensure that the most urgently needed capabilities are in place by the time of commissioning. The resources and schedule are currently a best-case scenario which runs the risk of not providing a fully functional CP by Oct, 2011 for commissioning. We suggest a phased project approach to give the present timeline a greater likelihood of success. Furthermore, the project should give high priority to completing the remaining work items which are required as part of both the DESDM and the CP. There appear to be a few individual requirements that could be relaxed while still enabling a fully functional CP (e.g., the12hr reduction requirement, asteroid removal, etc).

5. Charge topic 1:Does the CP implementation plan address the requirements described the Community Pipeline Requirements and Specifications documents? • The requirement that the CP operate exclusively on single-day data does not adequately reflect the processing needs from the community. The CP pipeline should be designed to work on designated short blocks of nights. Most observing runs with DECam will be >1 night long. The current CP budget envelope does not take into account the extra cost of adapting the pipeline to deal with small blocks of nights rather than a single night. Estimates of the cost of this are small (~5% additional cost), and should be done. • The community needs document should be accepted by the DES project after discussion of the more flexible “requirements”. This will provide a firm basis for fixing the implementation plan. The community pipeline requirements document has been largely stable—with the exceptions of the requirements identified in the possible descope options, the document should be accepted by DES.

6. Charge Topics 2&3: Is the CP scope of work, budget, and contingencies accurate, and is the cost model is accurate and credible; Comment on the management plan for the CP project and the means of tracking progress • For the first time in the 3 DESDM reviews I have participated in, CP feels like a real part of the project, and appears to be on a feasible path. • The scope of work is well-defined, and the schedule appears adequate for the work defined. • Staffing seems adequate in most areas, but CP testing is a concern. • The management plan defines a formal process for development that is appropriate to the CP project and is a major improvement in how software is being developed. • The requirement for processing a full night of data in 12 hours is aggressive and performance tests to date do not indicate the ability to do so with the currently sized/budgeted hardware. Do not devote excessive time to optimization at the 10-20% level.. • Capabilities needed for campaign processing are not yet in the implementation plan.

7. Charge Topic 4: Comment on the algorithmic and structural modifications of the DES Data Management System that will be needed to implement the CP • Run more “CP-like” data through current DESDM to identify any other areas where modifications that will be necessary

8. Charge Topic 5: Comment on whether the issues involved in transitioning from an Oracle database (used in DES DM) to postgres (used in the NOAO E2E system) are sufficiently understood and worked out • Yes. The key issues appear to have been identified and should not pose significant problems.

9. Charge Topic 7&8) Comment on the plan and schedule for CP data challenges; also on the plans for on-sky acceptance tests of the CP. • The CP “acceptance testing” procedure in DC6a is not yet well defined. The definition of tasks to be tested and data quality goals to be achieved is urgently needed. The schedule for DC6a calls for it to run at the end of 2010, and to be specifically geared to testing the CP. However, only the “CP1” phase of the project will have been completed by the time. At the moment, there is no consensus as to what datasets should or will be used to perform this testing. Although the DES simulations can be adapted to take into account some of the types of data the CP will have to deal with, other types (cf. large galaxies or nebulae) will have to be tested with real data. • Involvement of the NOAO data management team already at the time of CD6a is highly desirable. The NOAO DM team has the ultimate responsibility for maintenance and operation of the CP. It makes sense to let them gain experience with the system as soon as possible. In addition, the NOAO team has the experience to help identify the types of data and tests that are unique to the CP and that will serve as tests of the CP that differ from those already handled during the previous (DES-centric) data challenges.

10. 7&8) Comment on the plan and schedule for CP data challenges; also on the plans for on-sky acceptance tests of the CP. 3) The plans for the CP acceptance tests by NOAO are just being formulated. Coordination between the data modes used for DC6a and for the NOAO acceptance of the CP is required. NOAO is just beginning to determine the requirements and tests that will adequately test the CP to verify that it is ready for general NOAO user science. To avoid unpleasant surprises, the tests and the data that will be needed for the tests must be specified in time for inclusion of some of the data in DC6a (or at worst to inform the ordering of work in the transition between “CP1” and “CP2” phases. • The CP acceptance testing must be coordinated in both schedule and execution with the testing of the camera done at CTIO. Appointing Alistair Walker as CP scientist goes some way towards enforcing this coordination, given his role in DECam commissioning. • The plan for CP testing presupposes that the CP is installed and ready to run on NOAO computers. A potential schedule extension in which the on-sky testing is done with a CP running on a cluster setup at NCSA might be considered as an option.

11. Charge Topic 9: Comment on the roles of NCSA vs. NOAO personnel in Community Pipeline maintenance during operations • Long-term involvement by DES personnel is desirable for maintenance and upgrade of CP. However, it was not clear what resources are available for this during the operations period.

12. Charge Topic 10: General Comments on Descoping and Prioritization of CP Tasks • At most ~10% of tasks could be descoped or deferred • Most of work is documentation and conversion, which cannot be avoided. Early involvement of NOAO staff in deployment and testing can reduce documentation burden • Consider phased development that adds functionality and increasing quality following commissioning • User community can expect less than optimal performance during science verification phase • Reduce expectations on source “catalogs”