High Performance Computing

2. High Performance Computing University of Alabama in Huntsville Alabama A&M University Alabama State University Athens State University Auburn University -Montgomery Bevill State College Jacksonville State University Troy University Tuskegee University University of West Alabama University of Montevallo U.S. Air Force U.S. Army NASA Intel Corporation Operon Biotechnologies Time Domain University of Alabama at Birmingham University of Alabama University of South Alabama Auburn University

3. Alabama Supercomputer Applications • Materials Science • Computational Fluid Dynamics • Computer Science • Medical • Social Science • Education • Electromagnetics • Computational Chemistry • Structural Dynamics • Physics • Earth Science

4. Compilers and Programming • Compilers • GNU C/C++ Fortran 77 (All) • Intel C/C++ Fortran 77/90/95 (All) • Portland Group Fortran 77/90/HP (XD1 Only) • Pathscale Fortran 77/90 (XD1 only) • Parallel Programming • OpenMP • MPI • Pthreads • Java threads • Math libraries; ACML, SLATEC, MKL, SCSL, IMSL.

5. Memory Architectures • Distributed Memory-A CPU can access its memory only • Example : A cluster made from Single CPU PC’s • Both the Cray XD1 and the SGI Altix 350 can run distributed memory programs. • Shared Memory-Multiple CPUs can access the same memory • Example: Dual CPU PC • The Cray XD1 can run shared memory programs on the two CPUs in a given node. • The SGI Altix 350 can run shared memory programs on the 16 CPUs in the compute nodes. • Hybrid – combines both approaches • Cray XD1 Example: Form a cluster out of Dual CPU Nodes (mostly distributed) • SGI Altix Example: Form a Cluster out of 16 CPU Nodes (mostly shared) (A Node = 1 copy of the operating system)

6. SGI Altix Supercomputer • • 216 Itanium 2 Processors • - 1186 GFLOPS Peak • • Shared Memory Architecture • - NUMAlink, Inifiniband, fiber channel and gigabit ethernet data networks. • • Memory • - 1040 Gigabytes • • Disk Storage • - 10.8 Terabytes

7. Altix Login 6P Compute 16P 1.5 GHz Metadata 2P Compute 16P 1.4 GHz FC Switch Compute 72,72P 1.6 GHz Scratch Disks Ethernet Switch Home Directories Applications Queue Server Firewall License and Password Server AREN Internet and Internet2 SGI Altix Backup Copy

8. Cray XD1 Supercomputer • • 132 AMD Opteron Processor System • - 580 GFLOPS Peak • • Distributed Memory, Rapid Array Network. • 6 Xilinx Virtex 4 LX160 FPGAs • • Memory • - 244 Gigabytes • Nodes with 2,4,8 Gigabytes • • Disk Storage • - 3.5 Terabytes

9. The AMD Opteron Processor 64KB Dedicated Memory Bus 64KB Native 32 & 64 bit x86 compatibility 1 MB Up to 19.2 GB/s I/O

10. Balanced Interconnect GigaBytes GFLOPS GigaBytes per Second Memory I/O Interconnect Processor 1 GB/s PCI-X 0.25 GB/s GigE Xeon Server 5.3 GB/s DDR 333 Cray XD1 6.4GB/s DDR 400 8 GB/s Removing the communications bottleneck

11. Dense Memory Cluster • • 168 Opteron Processors • - 1008 GFLOPS Peak • • Shared/Distributed Memory Architecture • - Inifiniband, and gigabit ethernet data networks. • • Memory • - 1312 Gigabytes • • Disk Storage • - 21.5 Terabytes

12. Performance Comparison

13. Torque/MOAB queue system • User commands • qsub – run a job • qstat – see status • checkjob – see job information • Queue Server Node • Server – keeps track of queues and jobs • Scheduler – chooses when/where to run jobs MOAB runs jobs to ensure maximum utilization of the system, without over-subscription, and ensures jobs get the requested amount of memory/CPUs. Compute node * MOM daemon – runs jobs and reports available CPUs/memory

14. 500 Mbs Internet AREN Backbone 50 Mbs Florence Huntsville 155 Mbs 10 Gbs Sep. Jacksonville • Highspeed Backbone • 1.5-100 Mb to Clients, driven by their requirements. 10Mb typical. • Multiple Internet Access Points (GigE): 1.6 Gbps 100 Mbs Birmingham 500 Mbs Internet 10 Gbs ATL Sep. 165 Mbs 155 Mbs Tuscaloosa Auburn 155 Mbs 10 Mbs 500 Mbs Internet Montgomery 155 Mbs 1 Gbs Troy 10 Mbs 155 Mbs Dothan Mobile ASA has the networking infrastructure to move very large datasets via high bandwidth. 100 Mbs Internet

15. Statewide Education Network Alabama Research & Education Network (AREN) Internet K-12 Schools Universities Central Offices SDE Postsecondary Colleges Public Libraries ASA’s high speed network is connected to the Supercomputer Center, Internet, and Internet2.

16. Statewide Network: A Reality! 1994 2008 56 Kb T1

17. Network Solutions and Services • Network Operations • Network Monitoring • Network Operations Centers • Helpdesk • Enterprise Management • Internet Usage Statistics for Clients • Network Design and Engineering • Internet Provider • LAN, MAN, WAN • Network Consulting, Analysis, Troubleshooting • Network Security • Custom Development • Linux, NT, DNS, Proxy, Content Filter / Firewall, Mail, Web, etc. • Server Hosting • Network Installation • Network Equipment • Integration and Test

18. Network Technologies WAN Technologies Protocols • Point to Point (DS1, DS3) • Frame Relay • ATM (DS1, DS3, OC-3, OC-12) • SONET • xDSL • Metro Ethernet • Wireless • TCP/IP • IPv6 • IPX • SNA • Legacy Protocols, e.g., AppleTalk LAN Technologies Network Operating Systems • Ethernet 10/100/1000 • Fiber Channel (Storage Area Network) • FDDI • HiPPI (800, 1600, GSN) • ATM (LANE, MPOA, MPLS) • UNIX (Linux, SCO, BSDi) • Windows XP, Server 2003, Vista • Mac OS X • Cisco IOS • Novell

19. Internet Services • Virtual Private Networks • Web Services • Internet • Intranet • Security - Firewalls, Content Filtering, Intrusion Detection, Vulnerability Assessment • Servers - Web, Email, Collaboration, DNS • Quality of Service Configuration • DNS Hosting and Management

20. Technology Support through Partnerships Alabama Virtual Library (AVL) – Over 614,000 Average Monthly Searches; 464,093 AVL Cards Issued through 9/03 Alabama Learning Exchange (ALEX) – Custom Software Development, Technical Support, and Server Hosting Data System Applications – Web-based Portals for Data Analysis & Reporting, Including Secure Data Transmission Content Filtering –Available For All AREN Internet Clients Through Statewide Telemate License

21. AREN Benefits • State Agency That is a Proven Internet Service Provider for Education in Alabama Since 1994 • 24x7 Network Operations – Network Is Monitored and Maintained By Networking Experts • Excellent Source for Technology Consulting, Assistance, and Training • Custom System Development • Internet2 – Very High Speed Connection of State Network Education Institutions to this Resource Dedicated to Research and Academics

22. Computer Facility and Operations Center • State of the Art Computer Facility in Cummings Research Park with Capacity for Expansion and Security • Computer room floor space of 3,065 square feet (50% open) with an additional 22,000 square feet of office, storage, and meeting space • Multiple levels of physical security to the building, office area, and the computer room are provided by controlled access magnetic keys • 24 x 7 Network and Computing Operations, Monitoring, and Helpdesk • Staffed by Network Engineers, Systems Analysts, System Developers, and Technicians Excellent Facility for Expansion and New Services such as Backup Operations for State Government Data Centers and Computing on Demand Resources

23. Diverse Clients and Partnerships • ASA Clients include: • Research Universities • Regional Universities • Community Colleges • K-12 Systems and Schools • State Government Agencies • Alabama Department of Transportation • Alabama Department of Finance – Information Services Division • Public Libraries • Federal Agencies • Local Governments • Private Industry (e.g., Intel) • ASA has Partnerships with: • State Department of Education • Department of Postsecondary Education • Alabama Public Library Service • Alabama Virtual Library • The University of Alabama • Gulf Central Gigapop/Internet2/State Regional Optical Network • NASA

24. Summary • The Alabama Supercomputer Authority provides a number of services for academic, government and economic development within the state of Alabama. • High Performance Computing facilities • State wide network • Support of distance learning initiatives • Web and email hosting • Custom development • Disaster recover hosting • Computer security services • Alabama Virtual Library

25. Alabama Supercomputer Authority State of Alabama Leader and Trusted Partner for Technology