150 likes | 283 Views
ESRF Status. Long Term Strategy Computing infrastructure Control systems. Long term strategy. Nano science center 10 120m long beamlines (down to 20nm beam size) Extreme condition science (5Mbar,50Teslas,5Kk) Enlarge exph by 8000m² + 10000m² labs
E N D
ESRF Status Long Term Strategy Computing infrastructure Control systems JM Chaize, ESRF TANGO collaboration meeting
Long term strategy • Nano science center • 10 120m long beamlines (down to 20nm beam size) • Extreme condition science (5Mbar,50Teslas,5Kk) • Enlarge exph by 8000m² + 10000m² labs • new detectors (larger, faster data, larger data) • Topping-up operation ->new injector design • Higher intensity ->new absorbers,chambers,RF cav • Canted undulators, new lattice, more beamlines JM Chaize, ESRF TANGO collaboration meeting
Long term strategy JM Chaize, ESRF TANGO collaboration meeting
LTS consequences on computing • Data rate x 1000 for new detectors • Need new network infrastructure • Need new storage facility (Petabyte range) • Need new compute clusters (Petaflop…) • Grid computing, Parallel software development • Online data analysis becomes necessary • Virtualization of data storage, databank with metadata. • New 500m² data center JM Chaize, ESRF TANGO collaboration meeting
LTS roadmap • Presented at council spring 2006 • Detailed study report is spring 2007 • Detailed technical analysis • Costs and planning • Human resources • First phase starting in spring 2008 JM Chaize, ESRF TANGO collaboration meeting
Experiment data statistics • 2005 summary • 197 TBytes generated • 62 Millions of files • 2006 evaluation • 280 TBytes (1TB/day!) • 100 Millions of files • Present storage facility about 100TB JM Chaize, ESRF TANGO collaboration meeting
Internet 1 Gbps 1 Gbps 10 Gbps 4 Gbps Compute Clusters 1 Gbps FC attachments Frelon 2kx2k pixels 8-32 MB/s Network Attached RAID Storage Tape Robot Beamlines Central Building Computer Room 150 m2, 75 kW CTRM Computer Room 150 m2, 75 kW Topology…putting it together Q315 6kx6k pixels 35 MB/s JM Chaize, ESRF TANGO collaboration meeting
Comments on data statistics • x 10 since 2001 • We expect x 10 in the next few years (10 TB/days …) • May be 99% of 0 and 1% of real data. • Need to do online analysis! • Eliminate wrong data, extract the information as early as possible. JM Chaize, ESRF TANGO collaboration meeting
Control systems JM Chaize, ESRF TANGO collaboration meeting
Operating system convergence OS9 10 Workstations Solaris servers Solaris Debian30 60 VMEs Suse 7.2 10 PCs 6 CompactPci PCI bridge 40 PCs Suse 8.2 JM Chaize, ESRF TANGO collaboration meeting
Libera devices @ ESRF • 3 for Bpm for transfer line in operation • 3 for Bunch per Bunch feedback • Possibly many other in the future • (224 SR + beamlines BPM + other projects…) • Low quality ethernet interface • (weak, unreliable, non standard) • Release policy unclear, technical and commercial support difficult. • SOLEIL, are you happy with it? • We need to make pressure together! JM Chaize, ESRF TANGO collaboration meeting
TACO 8500 devices (defined) 170000 properties TANGO 2000 devices (defined) 18000 device properties 500 server instances 97 hosts (67 on events) TACO vs TANGO JM Chaize, ESRF TANGO collaboration meeting
TANGO HDB config Config server Archive events HDB_query Taco servers HDB_config HDB architecture Archive event handler Tango servers TACO HDB Filler DC cache Polling JM Chaize, ESRF TANGO collaboration meeting
VacGauge VacGauge VacGauge VacGauge VacGauge VacGauge TACO command TACO command TACO command TACO command TACO command TACO command PgBalzers PgBalzers PgBalzers VIP VIP VIP Serial line Serial line Serial line Serial line Serial line Serial line Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Gauge Vacuum system History SrGauges Tango Spectrum Attributes 32 Servers VacCellGauge VacCellGauge VacCellGauge Tango Scalar Attributes To HDB 90 Servers 500 Devices JM Chaize, ESRF TANGO collaboration meeting
Pb of dynamic attributes • Residual Gas Analyzer (100 dynamic attributes) • Potentially 64 devices x 100 attributes • Thermocouple controller (64 dynamic attributes) fully dynamic! • Need automatic way to configure storage JM Chaize, ESRF TANGO collaboration meeting