1 / 24

SDSC TG RP Report September 07

SDSC TG RP Report September 07. Mark Sheddon SDSC TG RP PI sheddon@sdsc.edu. SDSC RP Quarterly Meeting Report. HPCOPS Plans FY08 RP Plans CY08 New and Cool Draft SDSC YR3 TG RP Milestones Quarterly RP Accomplishments/Milestone Status Wrap-up. HPCOPS FY08. Operate DataStar and BlueGene

dalia
Download Presentation

SDSC TG RP Report September 07

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SDSC TG RP ReportSeptember 07 Mark Sheddon SDSC TG RP PI sheddon@sdsc.edu

  2. SDSC RP Quarterly Meeting Report • HPCOPS Plans FY08 • RP Plans CY08 • New and Cool • Draft SDSC YR3 TG RP Milestones • Quarterly RP Accomplishments/Milestone Status • Wrap-up

  3. HPCOPS FY08 • Operate DataStar and BlueGene • Provide archival storage for HPC users • Support advanced consulting and porting of applications codes • Support GPFS-WAN • Continue Technical Collaborations with NCSA • Middleware (Metascheduling) • Visualization Pipeline • Training • Conduct EOT activities • Summer intern program • HPC Summer Institute • TeacherTECH • Develop online training • …

  4. New and CoolMeta-scheduling Themes from TG futures user WS’s Managing scheduling flexibility and allocations, such as throughput on demand or opportunistic scheduling Take meta-scheduling seriously, not as a future dream—allocate funding for development (11) Standardization that would make TeraGrid a real grid that could support the effective use of allocations and meta-scheduling • Co-scheduling • User set-able reservations • Automatic resource selection • On demand • Ensemble scheduling • Workflow

  5. Co-schedulingUser Set-able Reservations • GUR/HARC  Local Schedulers • GUR and HARC make reservations on local schedulers (Catalina, Moab, etc.) • No communication between GUR and HARC is required.   • Current Installations • GUR (from SDSC) • NCSA (IA-64) • SDSC (IA-64, DataStar) • HARC (from LSU) • NCSA (All machines, except Abe) • SDSC (IA-64) GUR (SDSC) HARC (LSU) Reservation S/W Catalina (SDSC) Moab Local Schedulers

  6. New and CoolCo-schedulingUK e-science All-Hands Demo (Sep07) SC’07 Demo (Nov07) • SDSC/NCSA/UK • Working w/Peter Coveney • GUR, HARC • Catalina, Moab • Clinical Application • HemeLB used in the GENIUS project to characterize blood flow in arteries and vessels of the brain (cerebral haemodynamics). • 4 sites = 2+2 • SDSC, NCSA, 2 UK - Manchester, Oxford (?)

  7. New and CoolSC’07 Demo: GPFS/HPSS GPFS • SDSC and IBM • GPFS <=> HPSS integration • Largest archive system (# of files) • 1,000,000,000 file archive • Auto migration • GPFS' ILM policy scan performance -- how long it takes to find (pre-migration and migration) candidates • HPSS' archive performance using GHI and file aggregation -- how fast can HPSS update GPFS' extended attributes and store data into HPSS • HPSS' backup performance -- how long it takes HPSS to capture a file system snapshot • HPSS' restore performance -- how long it takes for HPSS to rebuild the GPFS namespace after a disaster HPSS

  8. New and CoolSC’07 Demo: pNFSSCinet Bandwidth Challenge pNFS Client • TG partners and IBM • TG Global Filesystem • No GPFS-WAN license required TeraGrid Network pNFS pNFS Client GPFS-WAN Server SDSC pNFS Client

  9. SDSC Q-Update: ResourcesGPFS-WAN “2.0” (milestone complete) • GPFS-WAN 2.0 “Test Day” March 21, 07 • Sites participating: ANL, PSC, NCAR, NCSA, TACC, SDSC • GPFS-WAN 2.0 Production April 2, 07 • Production sites: ANL, NCSA, NCAR (Frost front-end), SDSC • Successfully migrated 150 TB from old to new system • Successfully tested GPFS-WAN w/network outage and fail-over to Abilene on June 12 • GPFS-WAN 2.0 serving from SDSC • New IBM p575 (16, 8-way) servers in fully redundant, high-availability configuration • Purged “scratch” (63TB) • “Projects” space (404TB) • “Collections” space (105TB)

  10. SDSC Q-Update: Resources • Systems • Implement CTSSv4 kits as they become available on DataStar, BlueGene, and the IA-64 clusters (milestone complete to date) • CTSSv4 kits implemented on DataStar, BlueGene, and IA-64 clusters • Archive • Expand disk cache for HPSS to provide increased performance (milestone complete) • HPSS disk cache expanded from 33TB to 200TB • Put SDSC remote silo at PSC into production (milestone complete) • Developing usage policies • Provide high-speed GridFTP access to HPSS (milestone in progress) • Dependent on HPSS 6.2 upgrade, production target by year end

  11. SDSC Q-Update: Data Collections • Add collections approved by the allocation committee (ongoing milestone compete to date) • Multiphase Flow Simulations • PI: Josette Bellan, Caltech/JPL • The Next Generation Biology Workbench: A Resource for Biological Data Analysis • PI: Mark Miller, SDSC • Projects in Astrophysical and Cosmological Structure Formation • PI: Michael Norman, UCSD • Employment Responses to Global Markets • PI: Marc Muendler, UCSD • Insight into biomolecular structure, function and interactions from simulation • PI: Thomas Cheatham, Utah • Water Infrastructure Information System • PI: Sunil Sinha, Penn State • Dynamic Astronomical Image Services for the Virtual Observatory • PI: Roy Williams, Caltech • Expand online disk available to collections from 230TB to 400TB (milestone complete)

  12. SDSC Q-Updates: ASTA • ASTA - Current • Southern California Earthquake Center (SCEC) • PI Tom Jordan (USC), et. al. - SDSC Lead Yifeng Cui • The TeraShake-3 project is currently generating very-large scale 1-Hz 100m simulations on both DataStar and Lonestar, eight times larger than the original TeraShake case, requiring 80 hours run time on DataStar. (SDSC staff worked with Tommy Minyard at TACC to port the TeraShake code to Lonestar). • The Anelastic Wave Model (AWM) CVS was updated to version 4.6.4 and new features were added to AWM for portability. • Support was provided to the CyberShake project, managing the execution of more the 100K jobs on SDSC’s IA-64 cluster. • Support was provided to the DynaShake project, implementing new AWN MPI I/O features for the staggered-grid split node method. • ASTA - Complete • Demographic Patterns in Small-Scale Population Groups • PI Steve Lancing, U. Arizona - SDSC Lead Amit Majumdar • Implemented a checkpointing feature and automatic job submission script

  13. SDSC Q-Updates: User Services • Consulting/Training • Participate in TG’07 training activities by contributing to “Making Efficient Use of TeraGrid” workshop (milestone complete) • Krishna Muriki worked with Sergiu in planning the workshop, deciding on content, and coordinating speakers. Dave Hart from SDSC presented an overview of NSF allocations. • Simplify and promote data management across the TeraGrid through improvement of SRB configuration for users. (milestone in progress) • Krishna Muriki developed a script through which all TG users can setup their SRB accounts. Along with George Kremenek , he’s working on adding this script to the CTSS SRB distribution. • Documentation • Create robust database-backed User News application and integrate with Inca for system outages. (milestone in progress - delayed temporarily, redirected to CTSSv4 documentation)

  14. SDSC Q-Updates: Networking • Support proposed 2nd 10Gb/s link from LA to Chi (milestone revised and completed) • Shared10Gb/s Framenet link between Chicago and LA now in place

  15. SDSC Q-Update: EOTInstitutes/Workshops April – August 07 13 TeacherTECH, 1 StudentTECH, 9 Other = 23 Total April • Introduction to Parallel Programming for Community College Students; SC-07 Education Program • Blue Gene Applications Workshop • TeacherTECH Science - Sensors and the Environment May • TeacherTECH Science - Sensing the Environment in San Diego County • TeacherTECH Tools - Creating a Successful Science Webquest for Your Students • TeacherTECH Tools - SMART™ Interactive User Group June • StudentTECH, Introduction to Maya and 3D Modeling for High School Students July • TeacherTECH Tools - Podcasting for the Science Educator • IT – Engineering and Environmental Education Tools (with UCSD Jacobs School of Engineering) • TeacherTECH Science - Computer Mapping and GIS for Educators • NSF CI-TEAM – Current Projects and Future Directions(NSF OCI - Washington, D.C.) • TeacherTECH Tools - Beginning Podcasting for the Science Educator, SDSC

  16. SDSC Q-Update: EOTInstitutes/Workshops April – August 07 July( continued) • Reducing Time to Solution – SDSC HPC Summer Institute • Introduction to Grid Computing and CI Science; Navajo Technical College; SC-07 Education Program • TeacherTECH Math - Using Technology to Enhance the Math Classroom • TeacherTECH Tools - Advanced SMART Board Technology for the Classroom • TeacherTECH Tools - Advanced Podcasting for the Science Classroom • Computational Biology Institute; NCSA; SC-07 Education Program • TeacherTECH Tools - iMovie for the 21st Century Educator August • Cyberinfrastructure for Humanities, Arts, and Social Sciences, SDSC; SC-07 Education Program • Society of American Archivists CI Summer Camp • TeacherTECH Tools - Advanced SMART Technology for the Science Classroom • TeacherTECH Science - Molecular Modeling for Biology and Chemistry Teachers

  17. SDSC Q-Update: Misc. • Data Collections • New working group formed - Natasha Balac (SDSC) lead • TG’07 Program Committee • Nancy Wilkins-Diehr, Amit Majumdar, David Hart, Mark Sheddon

  18. SDSC Q-Update: New Building(beginning of April, 07) • 80K GSF • (50K ASF) • Summer 08 completion

  19. SDSC Q-Update: New Building(beginning of June, 07) • 80K GSF • (50K ASF) • Summer 08 completion

  20. SDSC Q-Update: New Building(beginning of September, 07) • 80K GSF • (50K ASF) • Summer 08 completion

  21. Dr. PhilTeraGrid RoamingHPC Counseling and Heavy Lifting…

More Related