1 / 13

GridPP Storage Future

GridPP Storage Future. Jens Jensen GridPP workshop RHUL, April 2010. Roadmap. <1 year of GridPP3 Then GridPP4 Hopefully full steam ahead A test of the Grid’s core data model guid:931b5c63-3209-4f14-8b80-63aa31141686 guid:c769954c-1ba6-4cd9-8716-b2a4d496dab5

avel
Download Presentation

GridPP Storage Future

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridPP Storage Future Jens Jensen GridPP workshop RHUL, April 2010

  2. Roadmap • <1 year of GridPP3 • Then GridPP4 • Hopefully full steam ahead • A test of the Grid’s core data model • guid:931b5c63-3209-4f14-8b80-63aa31141686 • guid:c769954c-1ba6-4cd9-8716-b2a4d496dab5 • guid:55323b16-2e2e-49fa-9bfd-41ab8099a88c • guid:f64ec9ab-f03f-46bb-9e43-22539502c3a3 • guid:81aa5bed-febd-40ef-8b72-38960720f16f • guid:0d4e652e-38ef-41db-8d05-372f2f5bbf6b

  3. Metapoints • For now: • Ongoing operations • “Issues” • New milestones • For GridPP4: • Revisit strategy • Revisit metrics • Revisit risks (last updated Feb 07)

  4. Strategic View “The GridPP storage group will provide and support storage middleware necessary and sufficient to Grid-enable storage at every GridPP site.” Strategic view needs to change for GridPP4

  5. (GridPP3) Goals • Storage Element interfaces (SRM+GridFTP+IS) • GridFTP performance • IS accuracy • Deployment/upgrade/maintenance/ debugging • Standards compliance, interoperable • Usable storage

  6. Dealing with Change

  7. Middleware Support • dCache – supported by DESY and FNAL • DPM – supported by CERN • StoRM – supported by INFN • EGI, EMI, IGE • More TLAs • Cloud computing

  8. Operations Issues • Upgrades • Coping with change • Core work area • Storage middleware • Scope • Optimisation and tuning, operations, troubleshooting, monitoring,

  9. Operations Issues • Upgrades • Coping with change • Expanding range of work • Catalogues • Data management • Storage middleware • Distributed filesystems • Filesystems • Hardware • Networks • Scope • Optimisation and tuning, operations, troubleshooting, monitoring,

  10. Issues • Specific frequent problems • Maybe need a better name • Difficult to solve • Require some sort of planning to address correctly • Deliverables

  11. Issues • File Integrity Testing • Planning: www.gridpp.ac.uk/wiki/File_Integrity_Testing • Outcome: discussed in weekly EVO • Status: closed(ish) • Access Patterns • Planning: loose set of pages in wiki • Outcome (so far): see all the nice presentations • Status: ongoing • file:// • Related to Access Patterns • Testing • Planning: www.gridpp.ac.uk/wiki/Storage_and_Data_Management_Testing

  12. Issues • Middleware • {StoRM,BeStMan} on {Hadoop,Lustre,GPFS} • Re-evaluating dCache • Development priorities for dCache • Information Systems • Fully compliant with WLCG Installed Capacity • GLUE 2.0 • Accurate accounting data • SSD Evaluation • Syncat – coping with orphaned files • Use of virtual machines • Filesystems evaluations (ext4, ZFS) • Distributed filesystems evaluations (NFS4, Lustre, GPFS)

  13. Conclusion • Lots of challenges, still • Storage is hard • Adapting to changes • Have learned caution • Good people • Combined expertise covers a lot • Friendly group • Seventy something members of group, in 9 countries • Some more active than others (~20)

More Related