1 / 21

TeraGrid Science Gateways

TeraGrid Science Gateways. Nancy Wilkins-Diehr Area Director for Science Gateways wilkinsn@sdsc.edu. You in this audience are the future of computational science You can tell me more about the value of gateways than I can tell you

aletta
Download Presentation

TeraGrid Science Gateways

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TeraGrid Science Gateways Nancy Wilkins-Diehr Area Director for Science Gateways wilkinsn@sdsc.edu TeraGrid 09 Education Program, June 22, 2009

  2. You in this audience are the future of computational science You can tell me more about the value of gateways than I can tell you First let me give you some background so we’re on the same page, then let’s begin a discussion TeraGrid 09 Education Program, June 22, 2009

  3. Today I hope to demonstrate • What are gateways? • Why are gateways worth the effort? • What do they allow scientists to do that they couldn't do without gateways? • What are some specific examples of this? • Why are these examples important? TeraGrid 09 Education Program, June 22, 2009

  4. GatewaysA natural result of the impact of the internet on worldwide communication and information retrieval • Implications on the conduct of science are still evolving • 1980’s, Early gateways, National Center for Biotechnology Information BLAST server, search results sent by email, still a working portal today • 1992 Mosaic web browser developed • 1995 “International Protein Data Bank Enhanced by Computer Browser” • 2004 TeraGrid project director Rick Stevens recognized growth in scientific portal development and proposed the Science Gateway Program • Today, Web 3.0 and programmatic exchange of data between web pages • Simultaneous explosion of digital information • Growing analysis needs in many, many scientific areas • Sensors, telescopes, satellites, digital images and video • #1 machine on Top500 today is 1000x more powerful than all combined entries on the first list in 1993 • Scientists developing their own tools to take advantage of the internet and help address the digital data problem Only 17 years since the release of Mosaic! TeraGrid 09 Education Program, June 22, 2009

  5. NSF understands the impact technology can have on science • “Virtual environments have the potential to enhance collaboration, education, and experimentation in ways that we are just beginning to explore.” • “In every discipline, we need new techniques that can help scientists and engineers uncover fresh knowledge from vast amounts of data generated by sensors, telescopes, satellites, or even the media and the Internet.” Gateways are a terrific example of interfaces that can support transformative science TeraGrid 09 Education Program, June 22, 2009

  6. Today, there are approximately 35 gateways using the TeraGrid TeraGrid 09 Education Program, June 22, 2009

  7. Why are gateways worth the effort? ======= # Full path to executable executable=/users/wilkinsn/tutorial/bin/mcell # Working directory, where Condor-G will write # its output and error files on the local machine. initialdir=/users/wilkinsn/tutorial/exercise_3 # To set the working directory of the remote job, we # specify it in this globus RSL, which will be appended # to the RSL that Condor-G generates globusrsl=(directory='/users/wilkinsn/tutorial/exercise_3') # Arguments to pass to executable. arguments=nmj_recon.main.mdl # Condor-G can stage the executable transfer_executable=false # Specify the globus resource to execute the job globusscheduler=tg-login1.sdsc.teragrid.org/jobmanager-pbs # Condor has multiple universes, but Condor-G always uses globus universe=globus # Files to receive sdout and stderr. output=condor.out error=condor.err # Specify the number of copies of the job to submit to the condor queue. queue 1 • Increasing range of expertise needed to tackle the most challenging scientific problems • How many details do you want each individual scientist to need to know? • PBS, RSL, Condor • Coupling multi-scale codes • Assembling data from multiple sources • Collaboration frameworks #! /bin/sh #PBS -q dque #PBS -l nodes=1:ppn=2 #PBS -l walltime=00:02:00 #PBS -o pbs.out #PBS -e pbs.err #PBS -V cd /users/wilkinsn/tutorial/exercise_3 ../bin/mcell nmj_recon.main.mdl +( &(resourceManagerContact="tg-login1.sdsc.teragrid.org/jobmanager-pbs") (executable="/users/birnbaum/tutorial/bin/mcell") (arguments=nmj_recon.main.mdl) (count=128) (hostCount=10) (maxtime=2) (directory="/users/birnbaum/tutorial/exercise_3") (stdout="/users/birnbaum/tutorial/exercise_3/globus.out") (stderr="/users/birnbaum/tutorial/exercise_3/globus.err") ) TeraGrid 09 Education Program, June 22, 2009

  8. Not just ease of useWhat can scientists do that they couldn’t do previously? • Linked Environments for Atmospheric Discovery (LEAD) - access to radar data • National Virtual Observatory (NVO) – access to sky surveys • Ocean Observing Initiative (OOI) – access to sensor data • PolarGrid – access to polar ice sheet data • SIDGrid – expensive datasets, analysis tools • GridChem –coupling multiscale codes • How would this have been done before gateways? TeraGrid 09 Education Program, June 22, 2009

  9. Gateways democratize access to high end resources • Almost anyone can investigate scientific questions using high end resources • Not just those in the research groups who log in to supercomputers • Gateways allow anyone with a web browser to explore • Opportunities can be uncovered via google • My 11-year-old son discovered nanoHUB.org himself while his class was studying Bucky Balls • Foster new ideas, cross-disciplinary approaches • Encourage students to experiment • But used in production too • Significant number of papers resulting from gateways including GridChem, nanoHUB • Scientists can focus on challenging science problems rather than challenging infrastructure problems TeraGrid 09 Education Program, June 22, 2009

  10. Tremendous Opportunities Using the Largest Shared Resources - Challenges too! • What’s different when the resource doesn’t belong just to me? • Resource discovery • Accounting • Security • Proposal-based requests for resources (peer-reviewed access) • Code scaling and performance numbers • Justification of resources • Gateway citations • Tremendous benefits at the high end, but even more work for the developers • Potential impact on science is huge • Small number of developers can impact thousands of scientists • But need a way to train and fund those developers and provide them with appropriate tools TeraGrid 09 Education Program, June 22, 2009

  11. Gateways in the marketplaceKids control telescopes and share images • “In seconds my computer screen was transformed into a live telescopic view” • “Slooh's users include newbies and professional astronomers in 70 countries” • Observatories in the Canary Islands and Chile, Australia coming soon • 5000 images/month since 2003 • Increases public support for investment in these facilities TeraGrid 09 Education Program, June 22, 2009

  12. Learn about the jet stream using LEADCan be done with live weather data • portal.leadproject.org • Education link • LEAD to learn modules • Polar jet stream TeraGrid 09 Education Program, June 22, 2009

  13. North American Mesoscale (NAM) model • Two jet streams in North America – polar and subtropical, polar is more prominent • Polar generally moves west to east, but sometimes forms large loops to the north and south • First image is a view of wind speed at 45, 55, 65, and 75 m/s.  • Also can see polar jet stream’s effect on surface temperatures TeraGrid 09 Education Program, June 22, 2009

  14. Move ahead to 6z (1 AM EST) on Jan 30th • Uncheck the box next to the 55 m/s wind speed and the 45 m/s wind speed • Looking at the 300 mb pressure and surface temperature, notice the upper-level ridge in the western part of the country and the trough in the eastern part of the country • Western states are experiencing warmer temperatures than the eastern states • Polar jet stream appears to be in the area of the largest surface temperature gradients and where the 300 mb geopotential height contours are tightest, making it more important to forecasters than the sub-tropical jet stream • During the winter months, the surface temperature gradients are strongest and farthest south, causing the polar jet stream to be stronger in the winter than the summer. TeraGrid 09 Education Program, June 22, 2009

  15. Move ahead to 18z (1 PM EST) on January 30th • Uncheck the box next to the 65 m/s wind speed and the Wind Barbs, check the box next to Upward Motion • Regions of upward motion at the entrance and exit of the high speed “jet streak” support the development of storm systems • Test your knowledge • http://www.atmos.millersville.edu/~lead/JS_Questions.html TeraGrid 09 Education Program, June 22, 2009

  16. Social Informatics Data GridCollaborative access to large, complex datasets • SIDGrid is unique among social science data archive projects • Streaming data which change over time • Voice, video, images (e.g. fMRI), text, numerical (e.g. heart rate, eye movement) • https://sidgrid.ci.uchicago.edu • Login and click portal • Search for "elan example" • Public project with video,audio and annotation data • Preview the data with Flash • Search • Export the data • Can visualize and annotate with SIDGrid Elan software • https://sidgrid.ci.uchicago.edu/about/getting_started • Run the analysis scripts. http://www.ci.uchicago.edu/research/files/sidgrid.mov TeraGrid 09 Education Program, June 22, 2009

  17. Gateways and Gadgets • Web technologies change fast • Must be able to adapt quickly • Gateways and gadgets • Gateway components incorporated into any social networking page • 75% of 18 to 24 year-olds have social networking websites • iPhone apps? • Web 3.0 • Beyond social networking and sharing content • Standards and querying interfaces to programmatically share data across sites • Resource Description Framework (RDF), SPARQL TeraGrid 09 Education Program, June 22, 2009

  18. Open Life Sciences Gateway • Add a BLAST query to your igoogle page and run on the TeraGrid • Google.com, add stuff • Gadget authorization • Click the link of “Personalize the gadget” • Use account/password scidac/scidac to login in • Here is a short test sequence: • >MRLFKTRKSTDTYSTLAAQQQQQQQQQQQHQAEGSNISHSSNSSSNKSHT PATCSNRLNKSIVSSTSISSSLPDLHDKSPVMILSCTTLASNGATATAAV TATATGTAATSGGSLQQQQQQHLQHQQQQQPLRTATPTCLLSGRQTPSAI • Check the blast type to "blastp" and leave the job submission site as "any site". Click OK, you will find a new blast job is created on Job History Portlet (Gadget). TeraGrid 09 Education Program, June 22, 2009

  19. Tremendous Potential for Gateways • In only 17 years, the Web has fundamentally changed human communication • Science Gateways can leverage this amazingly powerful tool to: • Transform the way scientists collaborate • Streamline conduct of science • Influence the public’s perception of science • Reliability, trust, continuity are fundamental to truly change the conduct of science through the use of gateways • High end resources can have a profound impact • We need you to help shape this very exciting future! TeraGrid 09 Education Program, June 22, 2009

  20. Now that you know more about gateways, what ideas come to mind for gateways in your area of interest? TeraGrid 09 Education Program, June 22, 2009

  21. Have fun at TG09 Contact me any time Nancy Wilkins-Diehr, wilkinsn@sdsc.edu TeraGrid 09 Education Program, June 22, 2009

More Related