200 likes | 318 Views
Overview of Broadband Platform Software as used in SWUS Project. SCEC software responsibilities on project include: Integration of scientific methods into Broadband Platform Added Irikura , Composite Source Model, EXSIM in 2013 Added multiple Part A and Part B reports
E N D
Overview of Broadband Platform Software as used in SWUS Project
SCEC software responsibilities on project include: • Integration of scientific methods into Broadband Platform • Added Irikura, Composite Source Model, EXSIM in 2013 • Added multiple Part A and Part B reports • Added GMPE codes and utilities (e.g. rotD50, pynga) • Running BBP simulations using SCEC computing resources and posting simulation results • Ran full Part A and Part B (BBP v13.5) for most methods • Ran portions of Part A and Part B (BBP v13.6) for most methods SCEC Software Activities on BBP
The SCEC BBP Platform (BBP) integrates: • scientific software (e.g. C and Fortran) • formatting scripts (e.g. python) • seismic software utilities (e.g. rotd50, pynga) • post-processing plotting routines • observational data (e.g. seismograms) • expected results (e.g. seismograms, Peak SA values) Define the BBP Platform
Standard Inputs: • Earthquake Source File (.src file) • Station List(.stl file) • Velocity model specification (region) • Computational Method • Each method has region specific input files(e.g. Greens functions) • Standard outputs: • Velocity seismograms • Acceleration seismograms • Standard Rupture Format (4 methods), • Metadata list of programs and input files used • Metadata compute environment information Standard BBP Inputs and Outputs
'/home/epicenter/fsilva/bbp/13.6.0/comps/run_bbp.py', '-x', '/home/epicenter-03/fsilva/june13/gp-nps-50r/Xml/northpalmsprings-gp-0000.xml', '-s', '10000000', '-l', '/home/epicenter-03/fsilva/june13/gp-nps-50r/Sims/logs/10000000/10000000_northpalmsprings-gp.log' Running the BBP Platform
Sim Specification FileName: Simulation_ID.xml (e.g. 10000049.xml) Example Simulation Spec for Part A Validation Simulation: Genslip$BBP_INSTALL_GF/Mojave/gp/mojave_generic1d-urs01.vmod /home/epicenter-03/fsilva/june13/gp-nps-50r/Src/northpalmsprings-gp-0049.src mojave_generic1d-urs01.vmod northpalmsprings-gp-0049.src metadata.txt northpalmsprings-gp-0049.srf Mojave Jbsim$BBP_INSTALL_GF/Mojave/gp/mojave_generic1d-urs01.vmod /home/epicenter-03/ Hfsims$BBP_INSTALL_GF/Mojave/gp/mojave_generic1d-urs01.vmod /home/epicenter-03 Plot_Map/home/epicenter-03/fsilva/june13/gp-nps-50r/Src/northpalmsprings-gp-0049.src PlotSeis$BBP_INSTALL_VAL/NORTHPS/gp/northps_v13_3_1.stl northps_v13_3_1.stl True RotD50 $BBP_INSTALL_VAL/NORTHPS/gp/northps_v13_3_1.stl northps_v13_3_1.stl GMPEComparison$BBP_INSTALL_VAL/NORTHPS/gp/northps_v13_3_1.stl /home/ GenPlots$BBP_INSTALL_VAL/NORTHPS/gp/northps_v13_3_1.stl northps_v13_3_1.stl AccPEER3 acc 6.12 NORTHPS 90 False GenHTML $BBP_INSTALL_VAL/NORTHPS/gp/
As a general principle, the newest version of the broadband platform is better than any previous version and users should upgrade at first opportunity. • Older versions of the code will be archived to support reproducibility of previously obtained results. • Any use of an older version of the platform should be examined carefully to understand the scientific goals of using an older version. BBP Software Version Strategy
BBP results should be attributed to specific BBP software and configuration versions because results may change as methods improve. • For a Part A Validation Result, changes to any of these three versions that may impact the results: • Broadband Version 13.6.0-30-May-2013 • This is the source code version • Velocity model version LABasin - 29-May-2013-1.07 • This is the region specific input parameters used in the simulation • Validation package version NR - 29-May-2013-1.05 • - This is the set of observational data and associated corrections used in the GOF plots for this validation simulation • These are found in the index.html file in the outdata directory for each simulation. Versioning BBP Results
Plans to create 2 BBP Distributions SWUS (Complete) Standard (no obs data): SWUS Distribution Broadband Version 13.6.0-30-May-2013 Velocity model version LABasin - 29-May-2013-1.07 Validation package version NR - 29-May-2013-1.05 Standard Distribution:. Broadband Version 13.6.0-30-May-2013 Velocity model version LABasin - 29-May-2013-1.07 Planned BBP Distributions
When comparing BBP results, all results should be produced with the same version of the broadband platform. Standard problems should be re-run when new versions of the platform are released. • We assume that: • Post processing changes affect results for all methods. • Changes to any method’s source code, including python wrappers, may affect results from other methods • Changes to one method parameters do not affect results from another method • We make this assumption to simplify our analysis of results: • We do not have the resources to detect all possible sources of coupling between scientific programs in broadband platform • This assumption can be relaxed, if additional analysis is done to confirm results are equivalent between BBP software releases. Comparing Results Between BBP Software Versions
We have posted results. These should be considered results from BBP v13.6.0 • Results are posted under two studies: • http://bbpvault.usc.edu/bbp/BBP_June2013_Study/ • http://bbpvault.usc.edu/bbp/BBP_May2013_Study/ • When accessing these results: • Use v13.6 results for any simulation if the results exist. • If no v13.6 results exist, use v13.5 results. • If no v13.5 results exist, simulation was not completed. Results From Two Software Versions
Posted Results are Organized by Release, Method, Sim Type • http://bbpvault.usc.edu/bbp/BBP_June2013_Study/ • http://bbpvault.usc.edu/bbp/BBP_May2013_Study/ • List of Methods: Within Method (GP): • csm/ gmpe/ • exsim/ landers/ • gp/ lomap/ • irikura/ niigata/ • sdsu/ nps/ • ucsb/ nr/ • whittier/ • tottori/ BBP Simulation Results
Results Organized in two Base Directory: • http//bbpvault.usc.edu/bbp/BBP_June_2013/ • http//bbpvault.usc.edu/bbp/BBP_June_2013/ • Part A Results: • Method Name: [csm,exsim,gp,irikura,sdsu,ucsb] • Event Name: [landers,lomap,niigata,nps,nr,whittier,tottori] • Simulation ID: [10000000, 10000049] • http://bbpvault.usc.edu/bbp/BBP_June2013_Study/gp/landers/outdata/10000049/ • Absolute pathname carries information about what method was used and what event was used to produce the simulation results contained in a directory. In each simulation directory, a simulation specification file (sim_id.xml) provides detailed information about the programs run to produce the results in the directory. BBP Results Directory
Part A Results Directory Listing: http://bbpvault.usc.edu/bbp/BBP_June2013_Study/gp/landers/outdata/10000049/index-10000049.html
Results Organized in two Base Directories: • http//bbpvault.usc.edu/bbp/BBP_June_2013/ • http//bbpvault.usc.edu/bbp/BBP_June_2013/ • Part B Results: • Base File location: Study Name/Method Name/gmpe/ • http://bbpvault.usc.edu/bbp/BBP_June2013_Study/gp/gmpe • Filename formed from: [method, mag, mechanism, region, dist] • bp-gmpe-ucsb-62-ss-nocal-20km.pdf • bp-gmpe-ucsb-66-ss-socal-50km_w_gmpe.pdf BBP Output (Results) Directory
Simulation Directories Metadata Files: • index-10000049.html • 10000049.xml • system_status-10000049.txt • software_status-10000049.txt • Example: software_status-10000049.txt • Python version: 2.7 (r27:82500, Sep 16 2010, 18:02:00) • GCC Version: gcc (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4) • ICC Version: icc (ICC) 12.0.4 20110427 • IFORT Version: ifort (IFORT) 12.0.4 20110427 • G77 Version: GNU Fortran (GCC) 3.4.6 20060404 (Red Hat 3.4.6-20.fc14) • GFORTRAN Version: GNU Fortran (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4) • NumPyVersion: 1.4.1 • SciPy Version: 0.7.2 • Matplotlib Version: 1.0.1 BBP Metadata Files
EXSIM: BBP file with 2 header lines. dt: 0.005000 (200 sps) npts: 32768 (163.84 secs) CSM: BBP file with 2 header lines: dt: 0.020000 (50 sps) npts: 4096 (81.92 secs) GP: BBP file with 1 header lines. dt: 0.025000 (40 sps) npts: 4096 (102.4 secs) Irikura: BBP file with 1 header lines. dt: 0.008333 (120.004 sps) npts: 14400 (119.9952 secs) UCSB: BBP file with 1 header lines. dt: 0.020000 (50 sps) npts: 8192 (163.84 secs) SDSU: BBP file with 1 header lines. dt: 0.003124 (320 sps) npts: 32770 (102.373 secs) BBP Seismogram Files
Estimates for Part A and Part B SimulationsMethod • Compute HoursStorage (GB) Outdata Only (GB) • GP 50 620 • SDSU 150 620 • EXSIM 36 500 • CSM 71 390 • Irikura 52 472 • UCSB 150 620 • 609 * 32 CPUs • 19,488 CPU Hrs 3222 GB 300 GB • Estimate per Part A and Part B runs 20k SU - 3.2 TB (Total) - 0.3 TB (Outdata) BBP Compute and Storage Estimates
Estimates for Fling StudyMethod Compute Hours Storage (GB) Outdata Only (GB) GP 1416 ??? 160 1416* 16 CPUs = 22656 Assume all 6 methods run Fling 22656 * 6 = 135936 160* 6 = 960 (GB) Cumulative Totals (Part A & Part B + Fling) For 6 Methods (Outdata Only): (20k + 135k) = 155k CPU hours (300 + 960) = 1.1 TB Storage BBP Compute and Storage Estimates