1 / 80

OSCER: State of the Center

OSCER: State of the Center. Henry Neeman, OSCER Director hneeman@ou.edu OU Supercomputing Center for Education & Research A Division of OU Information Technology. Tuesday October 7 2008 University of Oklahoma. Preregistration Profile. Organizations

fergus
Download Presentation

OSCER: State of the Center

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OSCER:State of the Center Henry Neeman, OSCER Director hneeman@ou.edu OU Supercomputing Center for Education & Research A Division of OU Information Technology Tuesday October 7 2008 University of Oklahoma

  2. Preregistration Profile • Organizations • Academic: registered 62 institutions in 17 states, DC and 1 foreign country (AR, AZ, CO, FL, IA, IL, IN, KS, LA, MO, NC, ND, OK, PA, SD, TN, TX; DC; Costa Rica) • Industry: registered 29 firms • Government: registered 13 agencies (federal, state) • Non-governmental: registered 6 organizations • Demographics • 42% OU, 58% non-OU • 84% from EPSCoR states, 16% non-EPSCoR • Speed • 151 registrations in the first 24 hours • 202 registrations in the first 7 days OSCER State of the Center Address Tuesday October 7 2008

  3. This Year’s Big Accomplishments • Deployed new cluster • Oklahoma Cyberinfrastructure Initiative OSCER State of the Center Address Tuesday October 7 2008

  4. Outline • Who, What, Where, When, Why, How • What Does OSCER Do? • Resources • Education • Research • Dissemination • OSCER’s Future OSCER State of the Center Address Tuesday October 7 2008

  5. OSCER:Who, What, Where, When, Why, How

  6. What is OSCER? • Division of OU Information Technology • Multidisciplinary center • Provides: • Supercomputing education • Supercomputing expertise • Supercomputing resources: hardware, storage, software • For: • Undergrad students • Grad students • Staff • Faculty • Their collaborators (including off campus) OSCER State of the Center Address Tuesday October 7 2008

  7. Aerospace & Mechanical Engr NEW! Anthropology Biochemistry & Molecular Biology Biological Survey Botany & Microbiology Chemical, Biological & Materials Engr Chemistry & Biochemistry Civil Engr & Environmental Science Computer Science Economics Electrical & Computer Engr Finance Health & Sport Sciences History of Science Industrial Engr Geography Geology & Geophysics Library & Information Studies Mathematics Meteorology Petroleum & Geological Engr Physics & Astronomy NEW! Psychology Radiological Sciences Surgery Zoology E E E E Who is OSCER? Academic Depts More than 150 faculty & staff in 26 depts in Colleges of Arts & Sciences, Atmospheric & Geographic Sciences, Business, Earth & Energy, Engineering, and Medicine – with more to come! OSCER State of the Center Address Tuesday October 7 2008

  8. Advanced Center for Genome Technology Center for Analysis & Prediction of Storms Center for Aircraft & Systems/Support Infrastructure Cooperative Institute for Mesoscale Meteorological Studies Center for Engineering Optimization Fears Structural Engineering Laboratory Human Technology Interaction Center Institute of Exploration & Development Geosciences Instructional Development Program Interaction, Discovery, Exploration, Adaptation Laboratory Microarray Core Facility OU Information Technology OU Office of the VP for Research Oklahoma Center for High Energy Physics Robotics, Evolution, Adaptation, and Learning Laboratory Sasaki Applied Meteorology Research Institute Symbiotic Computing Laboratory E E E E Who is OSCER? OU Groups OSCER State of the Center Address Tuesday October 7 2008

  9. California State Polytechnic University Pomona (minority-serving, masters) Colorado State University Contra Costa College (CA, minority-serving, 2-year) Delaware State University (EPSCoR, masters) Earlham College (IN, bachelors) East Central University (OK, EPSCoR, masters) Emporia State University (KS, EPSCoR, masters) Great Plains Network Harvard University (MA) Kansas State University (EPSCoR) Langston University (OK, minority-serving, EPSCoR, masters) Longwood University (VA, masters) Marshall University (WV, EPSCoR, masters) Navajo Technical College (NM, tribal, EPSCoR, 2-year) NOAA National Severe Storms Laboratory (EPSCoR) NOAA Storm Prediction Center (EPSCoR) Oklahoma Baptist University (EPSCoR, bachelors) Oklahoma Climatological Survey (EPSCoR) Oklahoma Medical Research Foundation (EPSCoR) Oklahoma School of Science & Mathematics (EPSCoR, high school) Purdue University (IN) Riverside Community College (CA, 2-year) St. Cloud State University (MN, masters) St. Gregory’s University (OK, EPSCoR, bachelors) Southwestern Oklahoma State University (tribal, EPSCoR, masters) Syracuse University (NY) Texas A&M University-Corpus Christi (masters) University of Arkansas (EPSCoR) University of Arkansas Little Rock (EPSCoR) University of Central Oklahoma (EPSCoR) University of Illinois at Urbana-Champaign University of Kansas (EPSCoR) University of Nebraska-Lincoln (EPSCoR) University of North Dakota (EPSCoR) University of Northern Iowa (masters) YOU COULD BE HERE! E E E E Who? External Collaborators OSCER State of the Center Address Tuesday October 7 2008

  10. Who? OSCER Personnel • Director: Henry Neeman • Associate Director for Remote & Heterogeneous Computing: Horst Severini • Manager of Operations: Brandon George • System Administrator: David Akin (hired Jan 2005) • System Administrator: Brett Zimmerman (hired July 2006) • NEW! HPC Application Software Specialist: Josh Alexander (hired July 2008) • A little bit of OU IT sysadmin Chris Franklin to run the Condor pool OSCER State of the Center Address Tuesday October 7 2008

  11. Who is OSCER? Interns OSCER has been attracting interns from French universities • 2008: 2 from Limoges, 3 from Clermont-Ferrand • 2007: 3 from Limoges, 3 from Clermont-Ferrand • 2006: 3 from Limoges, 10 from Clermont-Ferrand • 2005: 2 from Limoges, 1 from Clermont-Ferrand OSCER State of the Center Address Tuesday October 7 2008

  12. Who Are the Users? Almost 450 users so far, including: • Roughly equal split between students vs faculty/staff; • many off campus users; • … more being added every month. Comparison: The TeraGrid, a national supercomputing metacenter consisting of 11 resource provide sites across the US, has ~4500 unique users. OSCER State of the Center Address Tuesday October 7 2008

  13. Biggest Consumers • Center for Analysis & Prediction of Storms: daily real time weather forecasting • Oklahoma Center for High Energy Physics: simulation and data analysis of banging tiny particles together at unbelievably high speeds OSCER State of the Center Address Tuesday October 7 2008

  14. What Does OSCER Do?

  15. What Does OSCER Do? • Resources • Teaching • Research • Dissemination OSCER State of the Center Address Tuesday October 7 2008

  16. OSCER Resources(and a little history)

  17. 2002 OSCER Hardware • TOTAL:1220.8 GFLOPs*, 302 CPU cores, 302 GB RAM • Aspen Systems Pentium4 Xeon 32-bit Linux Cluster (Boomer) • 270 Pentium4 Xeon CPUs, 270 GB RAM, 1080 GFLOPs • IBM Regatta p690 Symmetric Multiprocessor (Sooner) • 32 POWER4 CPUs, 32 GB RAM, 140.8 GFLOPs • IBM FAStT500 FiberChannel-1 Disk Server • Qualstar TLS-412300 Tape Library • Internet2 * GFLOPs: billions of calculations per second OSCER State of the Center Address Tuesday October 7 2008

  18. 2005 OSCER Hardware • TOTAL: 8009 GFLOPs*, 1288 CPU cores, 2504 GB RAM • Dell Pentium4 Xeon 64-bit Linux Cluster (Topdawg) • 1024 Pentium4 Xeon CPUs, 2176 GB RAM, 6553.6 GFLOPs • Aspen Systems Itanium2 cluster (Schooner) • 64 Itanium2 CPUs, 128 GB RAM, 256 GFLOPs • Condor Pool: 200 student lab PCs, 1200 GFLOPs • National Lambda Rail(10 Gbps network), Internet2 • Storage library: Qualstar (10 TB, AIT-3) * GFLOPs: billions of calculations per second OSCER State of the Center Address Tuesday October 7 2008

  19. 2008 OSCER Hardware • TOTAL:47,651.68 GFLOPs, 5651 cores, 8768 GB RAM • NEW! Dell Pentium4 Xeon Quad Core Linux Cluster (Sooner) • 529 Xeon 2.0 GHz Harpertown dual socket quad core, 16 GB RAM • 3 Xeon 2.33 GHz Clovertown dual socket quad core, 16 GB RAM • 2 Xeon 2.4 GHz quad socket quad core nodes, 128 GB RAM each • 34,386.88 GFLOPs • Coming: ~30 NVIDIA Tesla C1060 cards (933/78 GFLOPs each) • Condor Pool: 773 lab PCs, 13,264.8 GFLOPs, 2543 GB RAM • 183 x Intel Pentium4 32-bit 2.8 GHz with 1 GB RAM each • 400 x Intel Core2 Duo 2.4 GHz with 4 GB RAM each • NEW! 190 x Intel Core2 Duo 3.0 GHz with 4 GB RAM each • National Lambda Rail, Internet2(10 Gbps networks) • Storage library: Overland Storage NEO 8000 (100 TB, LTO) OSCER State of the Center Address Tuesday October 7 2008

  20. Improvement in OSCER Hardware GFLOPs: 2008 = 39 x 2002 RAM: 2008 = 29 x 2002 CPU cores: 2008 = 19 x 2002 Moore’s Law: 2008 = 16 x 2002 OSCER State of the Center Address Tuesday October 7 2008

  21. Dell Intel Xeon Linux Cluster 1,072 Intel Xeon CPU chips/4288 cores: • 529 x dual socket/quad core Harpertown 2.0 GHz, 16 GB • 3 x dual socket/quad core Clovertown 2.33 GHz, 16 GB • 2 x quad socket/quad core, 2.4 GHz, 128 GB each 8,768 GB RAM ~130,000 GB disk QLogic Infiniband Force10 Networks Gigabit Ethernet Platform LSF HPC Red Hat Enterprise Linux 5 Peak speed: 34,386.88 GFLOPs* *GFLOPs: billions of calculations per second sooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  22. Dell Intel Xeon Linux Cluster First friendly user: Aug 15 HPL benchmarked Sep 30-Oct 1: 27.11 TFLOPs (78.8% of peak) (hoping for 28 TFLOPs later this week) In production: Thu Oct 2 80% of cores in use Fri Oct 3 sooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  23. Dell Intel Xeon Linux Cluster Transition: first 6 weeks of usage (porting, tuning, production runs) • Topdawg • 5480 jobs • Job failure rate: 55% • Sooner • 48,000+ jobs • Job failure rate: 24% sooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  24. Dell Intel Xeon Linux Cluster Deployment • OSCER operations staff worked ridiculously long hours nonstop for three months. • They all went above and beyond, under extremely difficult circumstances. • We’re extraordinarily fortunate to have such an amazing crew. • Thank them every chance you get! sooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  25. Decommissioned 1,024 Pentium4 Xeon CPUs 2,176 GB RAM 23,000 GB disk Infiniband & Gigabit Ethernet OS: Red Hat Linux Enterp 4 Peak speed: 6,553 GFLOPs* *GFLOPs: billions of calculations per second topdawg.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  26. Decommissioned 1,024 Pentium4 Xeon CPUs 2,176 GB RAM 23,000 GB disk Infiniband & Gigabit Ethernet OS: Red Hat Linux Enterp 4 Peak speed: 6,553 GFLOPs* *GFLOPs: billions of calculations per second Goodbye! topdawg.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  27. About to be Decommissioned 64 Itanium2 1.0 GHz CPUs 128 GB RAM 5,774 GB disk SilverStorm Infiniband Gigabit Ethernet Red Hat Linux Enterprise 4 Peak speed: 256 GFLOPs* *GFLOPs: billions of calculations per second Purchased with NSF Major Research Instrumentation grant schooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  28. About to be Decommissioned 64 Itanium2 1.0 GHz CPUs 128 GB RAM 5,774 GB disk SilverStorm Infiniband Gigabit Ethernet Red Hat Linux Enterprise 4 Peak speed: 256 GFLOPs* *GFLOPs: billions of calculations per second Purchased with NSF Major Research Instrumentation grant Goodbye! schooner.oscer.ou.edu OSCER State of the Center Address Tuesday October 7 2008

  29. Condor Pool Condor is a software package that allows number crunching jobs to run on idle desktop PCs. OU IT has deployed a large Condor pool (773 desktop PCs in IT student labs all over campus). It provides a huge amount of additional computing power – more than was available in all of OSCER in 2005. And, the cost is very very low – almost literally free. Also, we’ve been seeing empirically that Condor gets about 80% of each PC’s time. OSCER State of the Center Address Tuesday October 7 2008

  30. Current Status at OU • Deployed to 773 machines in OU IT PC labs • Submit/management from old 32-bit Xeon nodes • Fully utilized • Some machines are burping, but will be fixed shortly • COMING: 2 submit/management nodes, 2.5 TB RAID OSCER State of the Center Address Tuesday October 7 2008

  31. Tape Library Overland Storage NEO 8000 LTO-3/LTO-4 Current capacity 100 TB raw Expandable to 400 TB raw EMC DiskXtender OSCER State of the Center Address Tuesday October 7 2008

  32. National Lambda Rail OSCER State of the Center Address Tuesday October 7 2008

  33. Internet2 www.internet2.edu OSCER State of the Center Address Tuesday October 7 2008

  34. OSCER Teaching

  35. What Does OSCER Do? Teaching Science and engineering faculty from all over America learn supercomputing at OU by playing with a jigsaw puzzle (NCSI @ OU 2004). OSCER State of the Center Address Tuesday October 7 2008

  36. What Does OSCER Do? Rounds OU undergrads, grad students, staff and faculty learn how to use supercomputing in their specific research. OSCER State of the Center Address Tuesday October 7 2008

  37. OSCER’s Education Strategy • “Supercomputing in Plain English” workshops • Supercomputing tours (like last night) • Q&A • Rounds OSCER State of the Center Address Tuesday October 7 2008

  38. Supercomputing in Plain English Supercomputing in Plain English workshops target not only people who are sophisticated about computing, but especially students and researchers with strong science or engineering backgrounds but modest computing experience. Prerequisite: 1 semester of Fortran, C, C++ or Java Taught by analogy, storytelling and play, with minimal use of jargon, and assuming very little computing background. Streaming video: http://www.oscer.ou.edu/education.php Registrations: almost 400 from 2001 to 2007 OSCER State of the Center Address Tuesday October 7 2008

  39. Workshop Topics • Overview • The Storage Hierarchy • Instruction Level Parallelism • High Performance Compilers • Shared Memory Parallelism • Distributed Parallelism • Multicore • High Throughput Computing • Grab Bag: Scientific Libraries, I/O libraries, Visualization OSCER State of the Center Address Tuesday October 7 2008

  40. Teaching: Workshops • Supercomputing in Plain English • Fall 2001: 87 registered, 40 – 60 attended each time • Fall 2002: 66 registered, c. 30 – 60 attended each time • Fall 2004: 47 registered, c. 30-40 attend each time • Fall 2007: 41 @ OU, 80 at 28 other institutions • NCSI Parallel & Cluster Computing workshop (summer 2004, summer 2005) • Linux Clusters Institute workshop (June 2005, Feb 2007) • Co-taught at NCSI Parallel & Cluster Computing workshop at Houston Community College (May 2006) • NEW! SC07 Education Committee Parallel Programming & Cluster Computing workshop Tue Oct 2 (the day before the 2007 Symposium) • NEW! SC08 Education Committee Parallel Programming & Cluster Computing workshop Aug 10-16 • NEW! SC08 Education Committee Parallel Programming & Cluster Computing workshop Mon Oct 6 … and more to come. OU is the only institution in the world to host and co-instruct multiple workshops sponsored by each of NCSI, LCI and the SC education program. OSCER State of the Center Address Tuesday October 7 2008

  41. Teaching: Academic Coursework • CS: Scientific Computing (S. Lakshmivarahan) • CS: Computer Networks & Distributed Processing (S. Lakshmivarahan) • Meteorology: Computational Fluid Dynamics (M. Xue) • Chemistry: Molecular Modeling (R. Wheeler) • Electrical Engr: Computational Bioengineering (T. Ibrahim) • Chem Engr: Nanotechnology & HPC (L. Lee, G. Newman, H. Neeman) OSCER State of the Center Address Tuesday October 7 2008

  42. Courses at OU Chem Engr: Industrial & Environmental Transport Processes (D. Papavassiliou) Engineering Numerical Methods (U. Nollert) Math: Advanced Numerical Methods (R. Landes) Electrical Engr: Computational Bioengineering (T. Ibrahim) Research Experience for Undergraduates at OU Ind Engr: Metrology REU (T. Reed Rhoads) Ind Engr: Human Technology Interaction Center REU (R. Shehab) Meteorology REU (D. Zaras) External American Society of Mechanical Engineers, OKC Chapter Oklahoma State Chamber of Commerce National Educational Computing Conference 2006 (virtual tour via videoconference) Norman (OK) Lions Club NEW! Society for Information Technology & Teacher Education conference 2008 NEW! Acxiom Conference on Applied Research in Information Technology 2008 NEW! Shawnee (OK) Lions Club Other Universities SUNY Binghamton (NY) Bradley University (IL) REPEAT! Cameron University (OK) NEW! DeVry University (OK) NEW! East Central University (OK) El Bosque University (Colombia) Southwestern University (TX) Louisiana State University Midwestern State University (TX) Northwestern Oklahoma State University Oklahoma Baptist University Oklahoma City University Oklahoma State University – OKC Oral Roberts University (OK) St. Gregory’s University (OK) Southeastern Oklahoma State University (TORUS) Southwestern Oklahoma State University Texas A&M-Commerce University of Arkansas Fayetteville University of Arkansas at Little Rock University of Central Oklahoma High Schools and High School Programs Oklahoma School of Science & Mathematics Oklahoma Christian University’s Opportunity Bytes Summer Academy Dept of Energy National Scholarship Finalists Ardmore High School (OK) E E E E Teaching: Presentations & Tours OSCER State of the Center Address Tuesday October 7 2008

  43. Teaching: Q & A OSCER has added a new element to our education program: When students take the Supercomputing in Plain English workshops, they then are required to ask 3 questions per person per video. Dr. Neeman meets with them in groups to discuss these questions. Result: A much better understanding of supercomputing. OSCER State of the Center Address Tuesday October 7 2008

  44. What Does OSCER Do? Rounds OU undergrads, grad students, staff and faculty learn how to use supercomputing in their specific research. OSCER State of the Center Address Tuesday October 7 2008

  45. Research & Teaching: Rounds Rounds: interacting regularly with several research groups • Brainstorm ideas for applying supercomputing to the group’s research • Code: design, develop, debug, test, benchmark • Learn new computing environments • Write papers and posters Has now evolved into supercomputing help sessions, where many different groups work at the same time. OSCER State of the Center Address Tuesday October 7 2008

  46. OSCER Research

  47. OSCER Research • OSCER’s Approach • Rounds • Grants • Upcoming Initiatives OSCER State of the Center Address Tuesday October 7 2008

  48. What Does OSCER Do? Rounds OU undergrads, grad students, staff and faculty learn how to use supercomputing in their specific research. OSCER State of the Center Address Tuesday October 7 2008

  49. Research: OSCER’s Approach • Typically, supercomputing centers provide resources and have in-house application groups, but most users are more or less on their own. • OSCER’s approach is unique: we partner directly with research teams, providing supercomputing expertise to help their research move forward faster (rounds). • This way, OSCER has a stake in each team’s success, and each team has a stake in OSCER’s success. OSCER State of the Center Address Tuesday October 7 2008

  50. Research & Teaching: Rounds Rounds: interacting regularly with several research groups • Brainstorm ideas for applying supercomputing to the group’s research • Code: design, develop, debug, test, benchmark • Learn new computing environments • Write papers and posters Has now evolved into supercomputing help sessions, where many different groups work at the same time. OSCER State of the Center Address Tuesday October 7 2008

More Related