1 / 14

Building and Testing OGCE Software on the NMI Build and Test Facility

Building and Testing OGCE Software on the NMI Build and Test Facility. Marlon Pierce Indiana University. OGCE’s portal release includes several Grid portlets built with Java COG, our Velocity Bridge, and GTLAB JSF libraries.

vondra
Download Presentation

Building and Testing OGCE Software on the NMI Build and Test Facility

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building and Testing OGCE Software on the NMI Build and Test Facility Marlon Pierce Indiana University

  2. OGCE’s portal release includes several Grid portlets built with Java COG, our Velocity Bridge, and GTLAB JSF libraries

  3. The OGCE Gadget Container allows you to build portals out of public and private Google Open Social gadgets. Supports HTTPS. Downloadable, packaged software.

  4. OGCE’s Cyberaide JavaScript provides a Grid abstraction layer for developing mashups and gadgets. Downloadable, packaged software.

  5. OGCE Tools for Science Workflows The XBaya workflow composer allows you to build scientific workflows from services running across the TeraGrid. This is part of our workflow suite.

  6. OGCE Packaging, Building and Testing • All component builds are designed to be self contained • Uses Apache Maven 2.x • Download includes everything you need. • Built with a single command: mvn clean install • Builds are modular • Code is managed by SourceForge’s SVN. • Apache JMeter test suite for the portal. • Need more than just unit tests • Run against your installation • Automated tests nightly

  7. Basic NMI Build and Test Script Structure • submitOGCE.sh is the master script. This is called as a cron job. • This calls the .submit script for each component • For each OGCE component, we have 3 metronome submit scripts • Nonmac (linux), mac-x86, mac-ppc • Example submit script on next slide • This stages in inputs to the destination machine, runs SVN • Finally, we run a build-specific shell script that does the actual Maven build. • You can get all of this from our SourceForge SVN: • http://ogce.svn.sourceforge.net/viewvc/ogce/metronome-scripts

  8. Example NMI submit file script project = OGCE component = Trunk Build component_version = 1.0.0 description = Automated test of OGCE build/runtime execution run_type = build inputs = ogcePortal.svn, buildOGCEPortal-nonmac.scp, OGCEJMeterTestFile.scp, OGCEJMeterResults.scp, jakarta-jmeter-2.scp, maven.scp remote_task = buildOGCEPortal-nonmac.sh remote_task_args = "" platforms = x86_64_fc_4, x86_64_fc_5, x86_64_rhap_5, x86_64_rhas_3, x86_64_rhas_4, x86_64_sles_8, x86_cent_4.2, x86_deb_3.1, x86_deb_4.0, x86_fc_2, x86_fc_3, x86_fc_4, x86_fc_5, x86_rh_7.2, x86_rh_8.0, x86_rh_9, x86_rhap_5, x86_rhas_3, x86_rhas_4, x86_slc_3, x86_sles_8, x86_sles_9, x86_slf_3, x86_suse_10.0, x86_suse_10.2, x86_ubuntu_5.10 prereqs = java-1.5.0_08 notify = riverma@indiana.edu remote_task_timeout = 120m

  9. Practical Lessons • Need to modify your build script for some platforms • JDK versions were different between Linux platforms, Macintosh x86, and Macintosh ppc platforms respectively • Made it necessary to use three different NMI scripts • Maven program had to be copied separately for Linux platforms because of NMI incompatibilities with OGCE’s bundled maven • Once again, be self-contained

  10. Practical Lessons • Environment variables need to be specifically assigned • $JAVA_HOME needed to be set and added to path • $HOME directory needed to be set to current NMI build directory because OGCE portal builds in the $HOME by default • Keep trying to build even if project fails the first time • OGCE svn download would occasionally fail, so we had each script try 'rebuilding' the portal up to three times every night

  11. Practical Lessons • Have your script "sleep" if runtime programs aren't loading • Apache Tomcat server would often fail to start • Issuing a "sleep 20" command gave Tomcat enough time to load

  12. Some Sample Timings • Linux platforms • Average build time per platform = 52.19 minutes • Average total time w/ queue wait (all platforms) = 223.2 minutes • Macintosh OSX 10.4 (PPC) • Average build time = 10.3 minutes • Average total time (including queue) = 25.29 minutes • Macintosh OSX 10.4 (x86) • Average build time = 6.2 minutes • Average total time (including queue) = 96.5 minutes

  13. Critique of the Build and Test System • Help support has always been great • Documentation should be improved. • Difficult to remember how to find out which Linux platforms are available. • Difficult to remember how to find out which Java versions are supported. • Need some “getting started” scripts • I really don’t care too much about specific OS and Java versions. It would be nice to have “use all linux”, “use newest Java” options. • Would also allow me to use the same scripts for all my platforms. • I would like to revisit our systems testing strategy

  14. More Information • Email: mpierce@cs.indiana.edu • OGCE Web Site: www.collab-ogce.org • Blog/News Feed: http://collab-ogce.blogspot.com/ • See our poster at the SDCI All Hands Meeting Thursday night • Rishi Verma did most of the original build and test work

More Related