The Developing Needs for e-infrastructures

1. The Developing Needs for e-infrastructures Professor John Wood, Chair, JISC Committee for the Support of Research

2. efficient large solid angle detectors... 4 orders of magnitude in 30 years …fast electronics NEUTRON MOORE’S LAW Neutron powder diffraction data rates (1950-2010) (4 orders of magnitude gain with ILL/ISIS alone) GEM(ISIS)  D20(ILL) 1998 D1a(ILL) 1983 Complex detectors with advanced electronics • detectors & advanced data acquisition • unique synergy within CCLRC • e-technologies (e-science) • bringing the central facilities into the universities

3. In practice: Its all about scale • Creation: • Examining the detector arrays on the MAPs spectrometer at ISIS

4. A One off experiment • Collection: • An ATSR image of Sicily with Mount Etna eruption; taken 24 July 2001

5. Data Deluge! • Capacity: • eg at RAL • 20PB by 2010 • 1PB = 1015 Bytes • Billions of Floppys • Millions of CDs • Thousands of PCs (today’s)

6. Curation – who is responsible? • Curation: • Some STFC based Repositories • The Atlas Datastore • The British Atmospheric Data centre • The CCLRC Data Portal • The CCLRC Publications Archive • The CCPs (Collaborative Computational Projects) • The Chemical Database Service • The Digital Curation Centre • The EUROPRACTICE Software service • The HPCx Supercomputer • The JISCmail service • The NERC Datagrid • The NERC Earth Observation Data Centre • The Starlink Software suite • The UK Grid Support Centre • The UK Grid for Particle Physics Tier 1A • The World Data Centre for Solar-Terrestrial Physics Atlas Datastore Tape Robot

7. The problem will grow • New large scale facilities are being planned and built around the world. • They will be run remotely and have to interact in real time with HPC simulations, each informing the other. What will be the role of the researcher once the experiment starts? • Data storage etc needs to be planned right at the start. • An example: XFEL in Hamburg

8. Schematic layout of a single pass XFEL A new X-ray source is needed for studies ofnew,ofnon-equilibrium states of matter at atomic resolution in space and time

9. X-Ray FELs Initial Future ERLs Future Peak brightness of pulsed X-ray sources Ultrafast x-ray sources will probe space and time with atomic resolution. Peak Brightness [Phot./(s · mrad2 · mm2 · 0.1%bandw.)] 3rd Gen. SR SPPS what do we do today and what tomorrow? 2nd Gen. SR Initial Laser Slicing FWHM X-Ray Pulse Duration [ps] H.-D. Nuhn, H. Winick

10. Fascination - FELs for hard X-rays The X-ray free-electron lasers will provide coherent radiation of the proper wavelength and the proper time structure, so that materials and the changes of their properties can be portrayed at atomic resolution in four dimensions, in space and time. Diffraction pattern of 10 x 10 x 10 Au cluster

11. Take a movie of chemical reactions Schematic presentation of transition states in a chemical reaction

12. Imaging of a single bio-molecule Lysozym with atomic resolution crystal single molecule Oversampling: J. Miao, K.O. Hodgson and D. Sayre,PNAS 98 (2001) 6641-6645

13. Coulomb Explosion von Lyzosym t=0 t=50 fsec t=100 fsec R. Neutze, R. Wouts, D. van der Spoerl, E. Weckert, J. Hajdu:Nature 406 (2000) 752-757

14. The VUV-FEL user facility at DESY

15. RF gun M2 M3 M4 M5 M6 M7 M1 undulators bunch compressor bunch compressor FEL experimental area collimator bypass Laser 1000 MeV 4 MeV 150 MeV 450 MeV 250 m VUV-FEL

16. European XFEL Facility in Hamburg phase II HERA phase I PETRA XFEL Length ca. 3.3 km

17. XFEL: Office and Laboratory Building

18. 1k x 1k system with 4 x 4 super modules

19. DAQ Challenge: 2D X-Ray Detector Systems •  106 pixels per frame for one detector • O(400-500) frames per train (goal, likely will start with less) • 10 trains per second (machine allows up to 30 Hz…) • With 2 Byte/pixel  average rate10 Gbyte/sec for one 2D detector! • Time between frames as short as 200ns  buffering needed 100 ms 100 ms 600 ms 99.4 ms LPD 200 ns

20. Data Storage Issues • Assume: • 3 x 1 Megapixel 2D Detector Systems • 2 Byte/pixel • 500 frames per train are read taken and read out • 10 trains per second • Running year: 200 days = 4800 hours • Running efficiency: 10% • Good-frame efficiency/compression: 25% (is this realistic?) •  ~13 Pbyte/year (1 Petabyte = 106 Gigabyte =1015 Byte)

21. Today’ Situation at DESY • SUN SL8500 Tape Robot • Installed at DESY in Jan 07 • Up to 10,000 Cartridges • Multi library capability • Lifetime of about 5 to 10 years (matter of running costs) • Up to 64 drives possible • Currently 30 drives LTO3 24 data/6 backup • LTO3 400 GB/Cart, 120 MB/s 10,000 LTO3 Cartridges: 4 Petabyte V.Gülzow et al.

22. Terabyte Year DESY Storage Capacity Planning w/o XFEL V.Gülzow et al.

23. Technology Forecast – Storage at DESY • not a technology problem • money and manpower issues • to be determined: • user behaviour • compression and accept/reject algorithms • potentially critical: access to data!

24. Exciting times ahead for JSR! • Raising awareness • Training • New interrogation tools and approaches • Infrastructure • Integrating information from many sources • Waking up decision makers • A new way of doing research for many communities • Who is going to be responsible for the long term storage, curation and authentication of data?