140 likes | 155 Views
CyberShake Study 2.3 Readiness Review. Study 2.3 Overview. Compare codes and velocity models RWG V3.0.3 vs AWP-ODC-SGT CVM-S 4 vs CVM-H 11.9 Different version of CVM-H than previous runs Adds San Bernardino, Santa Maria basins 286 Southern California sites 0.5 Hz
E N D
Study 2.3 Overview • Compare codes and velocity models • RWG V3.0.3 vs AWP-ODC-SGT • CVM-S 4 vs CVM-H 11.9 • Different version of CVM-H than previous runs • Adds San Bernardino, Santa Maria basins • 286 Southern California sites • 0.5 Hz • Deterministic post-processing only
SGT Computational Requirements • SGTs on Blue Waters • Computational time: 8.4 M SUs • RWG: 16k SUs/site x 286 sites = 4.6 M SUs • AWP: 13.5k Sus/site x 286 sites = 3.8 M SUs • 22.35 M SU allocation, 22 M SUs remaining • Storage: 44.7 TB • 160 GB/site x 286 sites = 44.7 TB
PP computational requirements • Post-processing on Stampede • Computational time: • 4000 SUs/site x 286 sites = 1.1 M SUs • 4.1 M SU allocation, 3.9 M remaining • Storage: 44.7 TB input, 13 TB output • 44.7 TB of SGT inputs; will need to rotate out • Seismograms: 46 GB/site x 286 sites = 12.8 TB • PSA files: 0.8 GB/site x 286 sites = 0.2 TB
Long-term storage • 44.7 TB SGTs: • To be archived to tape (NCSA? TACC? Somewhere else?) • 13 TB Seismograms, PSA data • Have been using SCEC storage - scec-04? • 5.5 TB workflow logs • Can compress after mining for stats • CyberShake database • 1.4 B entries, 330 GB data (scaling issues?)
Verification work • 4 sites (WNGC, USC, PAS, SBSM) • RWG V3.0.3, CVM-S • RWG V3.0.3, CVM-H • AWP, CVM-S • AWP, CVM-H • Plotted with previously calculated RWG V3 • Expect RWG V3 slightly higher than the others
WNGC CVM-S CVM-H RWG V3.0.3 - Green AWP - Purple RWG V3 - Orange
USC CVM-S CVM-H RWG V3.0.3 - Green AWP - Purple RWG V3 - Orange
PAS CVM-S CVM-H RWG V3.0.3 - Green AWP - Purple RWG V3 - Orange
SBSM CVM-S CVM-H RWG V3.0.3 - Green AWP - Purple RWG V3 - Orange
Estimated Duration • Limiting factors: • Blue Waters queue time • Uncertain how many sites in parallel • Blue Waters → Stampede transfer • 100 MB/sec seems sustainable from tests, but could get much worse • 50 sites/day; unlikely to reach • Estimated completion by end of June
Risks • Stampede becomes busier • Post-processing still probably shorter than SGTs • CyberShake database unable to handle data • Would need to create other DBs, distributed DB, change technologies