1 / 43

QuakeSim Project: Portals and Web Services for Geo-Sciences

QuakeSim Project: Portals and Web Services for Geo-Sciences. Marlon Pierce Indiana University mpierce@cs.indiana.edu. QuakeSim Project Summary. Goal is to provide a distributed environment for connecting scientific computing and data resources with Web based user interfaces.

stoutw
Download Presentation

QuakeSim Project: Portals and Web Services for Geo-Sciences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QuakeSim Project: Portals and Web Services for Geo-Sciences Marlon Pierce Indiana University mpierce@cs.indiana.edu

  2. QuakeSim Project Summary • Goal is to provide a distributed environment for connecting scientific computing and data resources with Web based user interfaces. • QuakeSim’s IT development includes • Portals for user interfaces. • Web Services for running remote applications and accessing databases • Databases for semantic fault models and InSAR data (USC)

  3. Some QuakeSim Applications and Their Data • Disloc, Simplex • Fault models are used to calculate surface displacements (Disloc) using Okada method. • Simplex is the inverse. • These help researchers refine their fault models from observed displacements, or model displacements associated with faults. • GeoFEST • Finite element code for detailed modeling of fault stresses, seismic displacements, uses fault models as input. • Coupled to mesh generation tools • Can (for example) calculate post- and co-seismic displacements. • RDAHMM • Time series analysis code, can be applied to GPS and seismic archives. • Identifies signal components (possibly associated with underlying physical causes) with no fixed parameters.

  4. HTTP(S) Portlets + Client Stubs SOAP/HTTP WSDL WSDL WSDL WSDL WSDL WSDL WSDL WSDL DB Service Job Sub/Mon And File Services Visualization Or Map Service JDBC DB DB, etc Operating and Queuing Systems Host 1 (QT or GRWS) Host 2 (Comp Grid) Host 3 (GIS)

  5. Daily RDAHMM Updates Daily analysis and event classification of GPS data from REASoN’s GRWS.

  6. April 21, 2006: What Happened?

  7. We can also anlyze real-time GPS data from the CRTN.

  8. Real time state changes?

  9. Disloc model of Northridge fault. Disloc used in Gerry Simila’s geophysics classes (CSUN).

  10. Integrating QuakeSim and UAVSAR • July 29, 2008 M 5.4 Chino Hills Earthquake • Used QuakeSim to model expected surface displacements from the event • Passed on KML file to UAVSAR program/project • Overlaid displacements with UAVSAR image • Will continue to merge projects using the Los Angeles ShakeOut in mid–November as a testbed

  11. TeraGrid Supercomputing Resources (GPIR)

  12. GeoFEST Finite Element Modeling portlet and plotting tools

  13. QuakeSim and Web 2.0 • Export all observations and computational results as KML, GeoRSS. • Use Social Networks to share projects, results, papers, proposals, etc. • Facebook and OpenSocial have open APIs. • Use social (Google) gadgets to deliver your Web components to everyone. • Use Google’s APIs to integrate your services with Calendar, Blogspot, YouTube, etc.

  14. More Information • Email: mpierce@cs.indiana.edu • QuakeSim Web Site: • www.quakesim.org • Portal URL: • http://gf7.ucs.indiana.edu:8080/gridsphere • Portal SourceForge Page: • https://sourceforge.net/projects/crisisgrid • Code SVN: • http://crisisgrid.svn.sourceforge.net/viewvc/crisisgrid/

  15. Acknowledgments QuakeSim work is funded by NASA AIST (A. Donnellan, PI) and ACCESS (Y. Bock, PI) programs. Indiana University developers: Galip Aydin, Xiaoming Gao, Zhigang Qi Robert Granat (JPL), Jay Parker (JPL), Maggi Glasscoe (JPL), John Rundle (UC-Davis), Harout Nazerian (JPL), Rami Al-Ghanmi (USC), Dennis Mcleod (USC), Paul Jamason (Scripps), Ruey-Juin Chang (Scripps), Gerry Simila (CSUN)

  16. Stop Talking Now, Champ

  17. Q: What Is Web 2.0? • A: IT for everyone • Too much of “Enterprise” computing requires specialized knowledge and specialized tools. • Result: specialization of tasks within teams like QuakeSim. • Waste of talent: scientists can write code, just don’t have time to waste on difficult operating environments. • What then is Web 2.0 in detail?

  18. What Is a Gadget? Simple gadgets for getting a Grid proxy credential and running remote commands. Both run on my own Web server.

  19. Google Reader and GeoRSS

  20. Google Maps and GeoRSS

  21. Google Earth and KML

  22. Cloud Computing and Gateways • Cloud computing is the combination of virtualization (Xen, VMWare, OpenVZ,…) with Web Services • Web Services control the life cycle of the virtual machines. • The virtual machines are under the control of the application developer. • UC-D can distribute the VC Service VM, for example • Examples include Amazon EC2, Eucalyptus (UCSD), and Virtual Workspace/Nimbus (UChicago) • Data clouds focus on data virtualization • Google’s BigTable, Facebook’s Cassandra • Apache’s Hadoop and related projects (HBase, HDFS) • Challenges • MPI on clouds • Mounting high performance file systems

  23. What Would You Want a Cloud? • Application Developers: Reproducible operating environments • Develop your application and be sure it will be deployed under the same conditions. • Distribute reproducible results. • Have control of your operating environment • Move applications closer to data. • Data replication built-in • Assume vast amounts of cheap diskspace

  24. Simplex refines fault models from GPS displacements

  25. UCSB’s Queue Prediction Service (QBETS) Forecasts time you will wait in the queue on various TG super computers. Inherited from OGCE project.

  26. OGCE’s XBaya Workflow Composer

  27. Some Design Choices • Build portals out of portlets (Java Standard) • Reuse capabilities from our Open Grid Computing Environments (OGCE) project, the REASoN GPS Explorer project, and many TeraGrid Science Gateways. • Decorate with Google Maps, Yahoo UI gadgets, etc. • Use Java Server Faces to build individual component portlets. • Build standalone tools, then convert to portlets at the very end. • Use simple Web Services for accessing codes and data. • Keep It Stateless … • Use Condor-G and Globus job and file management services for interacting with high performance computers. • TeraGrid • Favor Google Maps and Google Earth for their simplicity, interactivity and open APIs. • Generate KML and GeoRSS • Use Apache Maven based build and compile system, SVN on SourceForge

  28. Grid Job Submission • Globus provides a universal queuing system interface. • PBS, LoadLeveler, Sun Grid Engine, LSF • We chose Condor-G as our job management software for submitting jobs to HPC queuing systems. • University of Wisconsin • Works with Globus, Matlab DCE, Unicore, etc. • We co-locate Condor-G with our GeoFEST Web Service. • Communication is through Birdbath, Condor’s Web Service interface. • So GeoFEST service API is more or less the same, just now Grid enabled. • We also plan to release a general version of this service. • Condor command line and Birdbath have different names for job description parameters. • Big Easter Egg hunt to find this, but now we know.

  29. Portlet Summary

  30. Security Concerns They’ll see the Big Board!

  31. QuakeSimDistributed Environment for Modeling Observations

  32. Managing Real Time GPS Data Slides from Galip Aydin

  33. California Real Time Network Continuous GPS Stations (CGPS) are depicted as triangles while the Real-Time stations are represented as circles. Image is obtained from SOPAC GPS Explorer at http://sopac.ucsd.edu/projects/realtime How does one manage all the data generated by the 85 stations? How can you get just the data you want? Note this is fundamentally different from traditional request/response style Web Services.

  34. Processing Real-Time GPS Streams RYOPorts 7010 Scripps RTDServer Raw Data 7011 7012 NB Server GPS Networks Station Health Filter ryo2nb ascii2pos Single Station Displacement Filter RDAHMM Filter ryo2ascii ryo2nb ascii2pos Single Station RDAHMM Filter ascii2gml ryo2ascii /SOPAC/GPS/CRTN01/RYO Raw Data /SOPAC/GPS/CRTN01/ASCII /SOPAC/GPS/CRTN01/POS /SOPAC/GPS/CRTN01/DSME A Complete Sensor Message Processing Path, including a data analysis application.

  35. Application Integration with Real-Time Filters • Station Monitor Filter records real-time positions for 10 minutes and calculates position changes • Graph Plotter Application creates visual representation of the positions. • RDAHMM Filter records real-time positions for 10 minutes and invokes RDAHMM application which determines state changes in the XYZ signal. • Graph Plotter Application creates visual representation of the RDAHMM output.

  36. 2 – Multiple Publishers Test Topic 2 Topic 1A Topic n Topic 1B We add more GPS networks by running more publishers. The results show that 1000 publishers can be supported with no performance loss. This is an operating system limit.

  37. 4 – Multiple Brokers Test RYO Publisher RYO To ASCII Converter Topic 1A NB Server 1 Topic 1B Simple Filter 1 Simple Filter 2 Simple Filter 750 Simple Filter 751 Simple Filter 752 Simple Filter 1500 NB Server 2 NB Server 2 Topic 1B NaradaBrokering allows creation of Broker networks. We create a two-broker network. Messages published to first broker can be received from the second broker. We take timings on each broker. We connect 750 clients to each broker and run for 24 hours. We chose 750 clients to stay well below the saturation limit. The results show that the performance is very good and similar to single broker test.

More Related