1 / 14

VISTA Data Flow System

VISTA Data Flow System. for WFCAM & VISTA data 2nd overview for AGOC. Some events since 1 st AGOC visit. Mar 28 1 st AGOC visit Apr 04 VISTA Project Board moved pipeline PDR coincident with camera s/w FDR (Nov) Apr 15/16 v1/2 VDFS Archive CDR (wfcam)

garin
Download Presentation

VISTA Data Flow System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VISTA Data Flow System for WFCAM & VISTA data2nd overview for AGOC 2nd VDFS Presentation to AGOC

  2. Some events since 1st AGOC visit • Mar 28 1st AGOC visit • Apr 04 VISTA Project Board moved pipeline PDR coincident with camera s/w FDR (Nov) • Apr 15/16 v1/2 VDFS Archive CDR (wfcam) • May 05 v1/2 VDFS Archive CDR panel report received • May 28 v1/2 VDFS Archive Software Architecture Design Document • Jun 06 VISTA Project Board • Jun 12/13 Workshop on Observing Strategy (wfcam) • Jul 04 VDFS Q2 reviewed, Q3 plan set • Jul 04 Jim Gray & Alex Szalay visit archive • Aug 01 VISTA Project Board • Sep 17-19 VISTA Camera pre-FDR workshop • Oct 01 VDFS Q3 reviewed, Q4 plan set • Oct 02 VISTA Project Board 2nd VDFS Presentation to AGOC

  3. v1Archive CDR conclusions • VDFSv1 Science Archive CDR was held 15/16 Apr. The main task of the Panel was to determine whether the Science Archive design put forward by WFAU members met eight (seven!) criteria, their conclusions were: • 1 The end product is acceptable: Yes. • 2 The project as conceived is feasible: The Panel had major concerns regarding the project schedule and risk assessment. Both were far too optimistic, and timescales could easily be scaled up by a factor of two. • 3 Detailed plans for building the complete system are in place: A plan is in place, but important areas need to be better defined and planned. One important example is software architecture and software design procedures. 2nd VDFS Presentation to AGOC

  4. V1 Archive CDR conclusions 2 • 4 The budget and staffing resources are sufficient for the project and commensurate with the estimated time to completion: Staffing is a concern especially after deployment. Too little staff was allocated for v1 operations. • 5 A reasonable estimate of the risk has been made, and whether there are useful de-scope options: Risk assessment needs to be planned in greater detail. No de-scoping options were presented although the Panel strongly felt that the hardware and software needed to produce satisfactory pixel data and source catalogs and curate them should be in place before any resources were allocated to the ingestion of external catalogs for example. • 6 External interfaces are well controlled: Yes. • 7 The path from v1 to v2 is understood and feasible: The path from v1 to v2 will be easier if sufficient attention is paid to proper software architecture and design procedures that will maintain portable code and clean interfaces between subsystems. 2nd VDFS Presentation to AGOC

  5. AGOC visit 28 Mar 2003 – pt1/5 • Conclusions • 1.     The Committee recommended the following: • a.      Further VISTA DFS Advisory Committee members should be appointed who were potential data users who were not linked to the delivery of the project. • On further reflection it was felt there was no need for a formal AC as individual contacts served the advice functions needed by the team. • The question of a future role for a Users Committee remains open with PPARC for the future. 2nd VDFS Presentation to AGOC

  6. 1st AGOC visit 28 Mar 2003 – pt2/5 • b.     Appropriate procedures appeared to be in place for the proper engagement with ESO over the ESO deliverable aspects of the project. • Apr 04 VISTA Project Board moved pipeline PDR to be coincident with camera s/w FDR (Nov) • Sep 11 ESO asked for DFS PDR after camera FDR 2nd VDFS Presentation to AGOC

  7. 1st AGOC visit 28 Mar 2003 –pt3/5 • c.     A regular external oversight activity was required. At present this function fell to the VISTA Board. The VISTA Board appeared to be focusing most of its efforts on other aspects of the VISTA project, to the extent that the DFS project was not getting the level of scrutiny and oversight that it needed. PPARC should institute a more effective oversight structure for the DFS project. • PPARC wish to continue current oversight by VISTA Board supplemented 6 monthly this panel, at least until next tranche of e-science money. 2nd VDFS Presentation to AGOC

  8. 1st AGOC visit 28 Mar 2003 –pt4/5 • d.      The unfunded future effort issue would need to be resolved through appropriate bids to the current PPARC e-science call. • May 31 VEGA proposal submitted • Jul 14/15 PPRP passes £2.3m VDFS part of VEGA • Sep 30 GSC letter re VDFS/VEGA • Oct 14 telecon Oct 28 discussion meeting • Jan 2004 GSC 2nd VDFS Presentation to AGOC

  9. 1st AGOC visit 28 Mar 2003 –pt5/5 • e.     Project management procedures appeared to be emerging but these had only been instituted recently. A specific set of milestones and interim deliverables was still required, as was a process for measuring progress against them. Future oversight would need to ensure that these procedures became fully operational and effective. 2nd VDFS Presentation to AGOC

  10. Changes to External Milestones (top level) • WFCAM DFS • ‘04 Jan Commissioning - no data ~ camera hardware problems • ‘04 Apr Operations - no data ~ camera hardware problems • ‘05 Jan v2 pipeline & archive • ‘05 Apr 1st year science out – but won’t be a year of it by then! • VISTA DFS • ‘03 July Preliminary Design Review – moved by Board, then by ESO -- new date being negotiated • ‘03 Nov Final Design Review – corresponding slippage • ‘05 Dec Instrument in Chile • ‘06 May Instrument and onsite DFS Commissioning • ‘06 Aug Instrument Acceptance => Operations 2nd VDFS Presentation to AGOC

  11. Milestone Scheduled Date Actual Date VDFS Phase A funding starts 01 Apr 2002 2002 VDFS-v1(WFCAM) Pipeline Critical Design Review (CDR) 21/22 Oct 2002 21/22 Oct 2002 VDFS-v1(WFCAM) Science Archive CDR (Reviews in Oct'02 and Apr'03) 21/22 Oct 2002 15/16 April 2003 VDFS-v4(ESO deliverables) Preliminary Design Review by ESO 19 Dec 2003 2003 Deliver VDFS-v1(WFCAM) Pipeline & Science Archive 31 Dec 2003 Start VDFSv1(WFCAM) Pipeline Commissioning (Hawaii and UK) 01 Jan 2004 2004 WFCAM Commissioned and Science Observations Start 01 Apr 2004 Start VDFSv1(WFCAM) Science Operations (Hawaii and UK) 01 Apr 2004 Part VDFS e-science2 funding starts if this bid is successful 01 Apr 2004 VDFS-v4(ESO deliverables) Critical Design Review by ESO 13 Aug 2004 VDFS’s e-science1 funding ends 31 Sep 2004 Main VDFS e-science2 funding starts 01 Oct 2004 Deliver VDFS-v2(WFCAM) Pipeline & Science Archive 23 Dec 2004 Astrogrid tools released 31 Dec 2004 Top Level Milestones to end 2004 2nd VDFS Presentation to AGOC

  12. Top Level Milestones 2005-8 Milestone Scheduled Date Actual Date Complete 1st year WFCAM data taking & processing (Hawaii and UK) 01 Apr 2005 2005 Deliver VDFS-v4(ESO deliverables) for European Instrument Integration 26 Aug 2005 Deliver VDFS-v3(WFCAM) Pipeline & Science Archive 31 Dec 2005 VDFS-v4(ESO deliverables) Exposure Time calculator to ESO 06 Feb 2006 2006 Complete 2nd year WFCAM data taking & processing (Hawaii and UK) 01 Apr 2006 VISTA ready for on-sky commissioning 07 Jul 2006 Start VDFS-v4(ESO deliverables) commissioning at ESO, Chile 07 Jul 2006 Target date for handover to ESO 16 Aug 2006 Estimated VDFS-v4(ESO deliverable) s/w final delivery; ESO acceptance 22 Sep 2006 Start VDFS-v4(UK) Pipeline & Science Archive Commissioning in UK 16 Oct 2006 Complete VDFS-v4(UK) Pipeline and Archive End-to-End Commissioning 31 Dec 2006 Start VDFS-v4(UK) Science Operations 01 Jan 2007 2007 Complete third year of WFCAM data taking & processing 01 Apr 2007 Complete first 18 months of VISTA data taking & processing 31 Jun 2008 2008 Release VDFS-v5(UK) Pipeline & Science Archive (no further development) 31 Jun 2008 VDFS Routine Operations Phase starts 01 Jul 2008 2nd VDFS Presentation to AGOC

  13. Working of Management Process • Project Wide Mgt team + Local Mgt added Hambly & Hodgkin to monthly review team • Deliverables detailed • Work Packages (& their managers)  • Sub tasks defined (+ implementers) • Long range plan • Risk analysis  but not yet reviewed • Quarterly detailed plan & deliverables • Report against in future • Review past and make next • Monthly Mgt team (tele) progress meetings  • Weekly/fortnightly (local) team progress meetings.  Minutes & actions distributed.  2nd VDFS Presentation to AGOC

  14. Deliverables - metrics *Key 100% = complete 50% = almost finished 10% = started 0% = not started 2nd VDFS Presentation to AGOC

More Related