1 / 20

SLAC computing Highlights Les Cottrell, SLAC

SLAC computing Highlights Les Cottrell, SLAC. After dinner talk to the Silicon Valley Tour group from Switzerland at Lakeside Restaurant, Shoreline, Mountain View California May 13 th , 2011 Personal viewpoint of highlights in computing at SLAC Nobel prize winning computing Web at SLAC

neena
Download Presentation

SLAC computing Highlights Les Cottrell, SLAC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SLAC computing HighlightsLes Cottrell, SLAC After dinner talk to the Silicon Valley Tour group from Switzerland at Lakeside Restaurant, Shoreline, Mountain View California May 13th, 2011 Personal viewpoint of highlights in computing at SLAC • Nobel prize winning computing • Web at SLAC • China’s Internet • Record breaking network speeds • The future and data centers

  2. 1967-1970 Nobel prize experimentDiscovery of Quark • Use of computers for control, data acquisition, & logging very new • Programming led to need for physicist -programmers • Acquired a computer 96kBytes, 500kHz, 400KB disk • 100,000 times less than today • Way less than today’s iPhone more like today’s watch • Took a large room

  3. Dedicated to experiment no fashionable (at the time) time sharing • “Like wife sharing it tends to work best in special cases” • Programmed in FORTRAN IV with real time extensions • Experiment would not have been possible without it. • Had a lovely spacewar game ported from MIT with joysticks (from bubble chamber experiment) • Much friendlier than IBM 360/91 supercomputer used for off line analysis “IBM makes good business machines” • Kept running for 15 years 1979 • Company SDS long gone, swallowed up by Xerox • Nobel prize awarded in 1990

  4. Web at SLAChttp://www.slac.stanford.edu/history/earlyweb/history.shtml • 1991 • Sep 1991 Paul Kunz (physicist at SLAC) brings word of the World Wide Web's existence to SLAC on his return from a meeting with Tim Berners-Lee at CERN. • Dec 12 1991 first WWW server at SLAC (and first server outside of Europe) is successfully installed.  • 1992 • SPIRES-HEP, information system originally designed to provide access to preprints, or scientific papers that have not yet been published in peer-reviewed journals. Timely access to preprints can prevent duplicated effort, confirm or discount the direction of a proposed experiment or provide a vital clue to a struggling researcher, but the publication process can take weeks, months—even years. • Only access via guest accounts, nightmare to manage • First killer app for Web is the SLAC HEP physics preprint data base. • No need for guest account at SLAC to access, demoed by Berners-Lee at CHEP in France in February 1992 • Formation of SLAC web wizards, Bebo was a leader/evangelist

  5. Explosion • Fall 1992 Tony Johnson of SLAC releases the MidasWWW browser. Based on X-windows, • allows viewing of PostScript files on the Web from Unix and VMS, and even handles compressed PostScript. • February 1993 NCSA release Mosaic for PC • November 1993 SLAC puts up official home page on the web with links to web pages developed by others across the laboratory. • 1994 IHEP in Beijing China becomes the first Chinese institution to have a fully operational world-wide Internet connection, when a dedicated link with SLAC () is initiated. • Use of WWW explodes to the world beyond physics...

  6. China’s Internet link: Early Historyhttp://www.slac.stanford.edu/pubs/slacpubs/6250/slac-pub-6478.pdf • 1987-1991 BEPC2 (VAX 785) @ IHEP linked to CERN via packet switched data network (PSDN), dial up, email only, expensive

  7. Invitation • Chinese scientists from IHEP, visit SLAC in April/May 1991 • They were interested in computing and networking and as assistant director of computing I was invited to attend a meeting with them • They were particularly interested in a network connection to SLAC to support the Beijing Electron Spectrometer collaboration between IHEP, SLAC and other US institutions • With a meeting in Tokyo on Computing in High Energy Physics starting 3 weeks later, I suggest extending my trip to a visit to IHEP in Beijing, see the world • IHEP were very supportive, invite me

  8. Be Careful what you ask for… • Not knowing what to expect in Beijing • Just 2 years after the Tianamen Square • What was the technology available in IHEP • I was worried… • I met with Pief Panofsky the Emeritus Director of SLAC • He was very encouraging • He had this vision of how excellent networking could make a worldwide physics collaboration really work well • But outbound IHEP international calls needed an operator • How to make this work with a digital network …

  9. Panofsky to the Rescue • Panofsky called Nobel laureate T. D. Lee of Columbia for help • Request top priority to installation of 3 phones with unattended international access. • Get me visa • 3 weeks later after the CHEP conference in Tokyo I was met at the old Beijing airport • Taken to the Friendship Hotel used to be for Russian experts

  10. Working with IHEP staff • I was amazed to find the phone lines in place and working • They were excited about working with western “experts” • Determined not to let knowledge of English impede things • I was flattered to be considered an “expert”, and by their attention, friendliness and enthusiasm • However, soon found that despite nods and smiles I was talking way too fast • So wrote everything down as I talked, this forced me to go slowly and provided a written record

  11. Accomplishment while there IHEP VAX • Hooked up a 9600bps modem between phone & VAX 11/785 • Used 2nd phone line to call Charley Granieri at SLAC (15 hours apart) • Set up asynchronous DECnet dial-up connection SLAC – IHEP • Effective 400 bits/sec, very noisy and hard to use • Often unable (no international line available) to make connections • Frequent disconnects in mid-session • $3.0/min SLAC

  12. Now it gets really hard • US downstation Point Reyes north of San Francisco • China downstation Beijing airport • From airport microwave to BTA bldg 801 35km away in center of Beijing • BTA 801 to 821 exchange bldg 2 blocks from IHEP • Tried infrared, the microwave but error rates to0 high, eventually got a fibre route • Last 2 blocks there was copper, but problems with converting fibre to copper • March 1st 1993 acceptable error rates, handed over to IHEP • Seconds later monitor program showed the SLAC DECrouter adjacent to IHEP • DoE/SLAC paid $50K/yr, similar from China

  13. How did it work & for what • Ten times better: • 42kps file transfer • Echo time < 1 second • Error rate 1 in 10 million • 1-2 unscheduled outages/month • Twice yearly occulted by sun directly in line with satellite • How was it used: • Transfer physics data: 200MB/day (equivalent to that era’s tape cartridge – IBM 3480) • 2500 emails/day 400 sites in 21 countries via SLAC gateway • News groups, collaboration coordination, code management • Copying files, remote login, real-time communication

  14. Connecting to the Internet • Once 64kbps link established many Chinese institutions wanted to connect to IHEP to get access to Internet • Jan 1994 meeting to recommend China domain naming • Node.ihep.ac.cn • Proposed replacing DEC routers with Cisco routers • Got export licenses from US DOC • Received in Beijing Feb 1994, installed in March • Worldwide HEPnet connected • Dec 1994 Visit by US congressman George Brown to IHEP increased US government interest • Agreement for Internet to carry Chinese traffic (DOC, DoD, DoE) • Required Internet wide-area email sent April 18, 1994, saying China connection April 25, 1995 • IHEP fully Internetted via US West Coast interconnection point

  15. High speed networking: Need • Until late 1990s all HEP data was transferred between Labs by tapes. • Pack a 747 full of 50 Gbyte tapes, takes 10 hours to Geneva or ~ 2-4 Tbits/s, however • Expensive • At SLAC alone, producing, packaging and shipping the tapes took the full-time effort of two people. • How long did it take for data from SLAC to reach a researcher at CERN? Two weeks so data rate actually ~ < 1Gbits/s. Assuming no errors • This century network speed began to achieve and pass 1Gbits/s and so make it possible to use the network to transfer data

  16. Breaking the Internet2 Land Speed Record twice 2002/3 • Internet2 Land speed record 2002 in Baltimore • http://www-iepm.slac.stanford.edu/lsr/faq.html • Big set up for SC002 to demonstrate high performance • with loaned equipment from Cisco (router), Level3 (circuits), Intel 10Gbps Ethernet interface, collaboration of SLAC, Caltech, CERN and others for SC2002 • First time Impromptu, hall discussion with student Antony Antony from Amsterdam University and almost a bit of a lark • 900 Gbps equivalent of 2 DVD’s/minute, Sunnyvale to Amsterdam (<6000km) • Repeated Sunnyvale to Geneva in Feb 2003 at 2.38Gbits/s or a Tbyte/hour or • heroic tuning of hosts, TCP stacks, team of about 20 people, but COTS hardware & software • Caught attention: reported in English, Spanish, Portuguese, French Dutch and Japanese, on CNN, BBC, Wired, Nature, Times of india, Tech TV and ABC radio • Guinness book of records 2004 • CENIC award, Internet2 record • http://www-iepm.slac.stanford.edu/lsr/ • http://www-iepm.slac.stanford.edu/lsr2/

  17. Supercomputing Nov 2003 - Phoenix • SLAC, Caltech, LANL, CERN, Manchester and Amsterdam • 23Gbits/sec multiple computers (6.6 Tbytes in 48 minutes = 2000 DVDs) • won SC2003 Bandwidth challenge • Congratulations from US congress woman Anna Eshoo • http://www-iepm.slac.stanford.edu/monitoring/bulk/sc2003/hiperf.html

  18. Repeated wins at SC 2004 and 2005 • 2004 (Pittsburgh): • 101.3Tbit/s 3 full length DVD movies/sec, library of congress in 14 minutes • Used 8 10Gbps waves from multiple providers • E.g. ~ 9Gbps from SLAC to Sunnyvale • Limitations in bus speeds • 2005 (Seattle): • Sun, Cisco, Ciena, Qlogic, 3Par, Internet2, Esnet, NLR, Surfnet, CANARIE, AMPATH • 22 * 10Gbps fibres (US, UK, Brazil, Korea, Switzerland (CERN) )… • > 150.7Gbps 130 DVDs/min, Lib. of Congress in < 9 mins, serve 10,000 MPEG HDTV movies in real time • http://www-iepm.slac.stanford.edu/monitoring/bulk/sc2004/hiperf.html

  19. Clustered Systemshttp://www.clusteredsystems.com/ • 50TFlops in a single rack • Using Intel Sandybridge systems, • NextGen => GPUs • 100kW/rack • Build showcase in fish bowl @ SLAC • Innovative cooling: • Conductive cold plates water cooled, fanless • Connected to board components by Gel packs • Can use 60 deg F (instead of normal 40 deg F) return cooling water for further cooling • 2.5 times less power/ TFlop and less space, more green • Disruptive technology • Demo later this year

  20. Stanford Research Computing Facility (SRCF) • Cost of upgrading existing computing centers designed in mainframe days is prohibitive while maintaining services • For each $ spent on computer need another for providing power & cooling • “create world in 6 days but no installed user base” • $43M computer center for SU located at SLAC • break ground end CY • Modular & flexible, high density power & cooling, low operational costs • Green: take advantage of California’s temperate climate with ambient air cooling • Initial 2.5-3.5MW eventually 6MW

More Related