1 / 62

Welcome ! DAY 1

Welcome ! DAY 1. Facility Overview. Entering and Exiting Rear entrance is closest to the auditorium Perimeter doors are not secure Do not exit through the auditorium rear emergency doors Restrooms Located near the 2 nd floor front entrance, left of the double doors In Case of Emergency

dutch
Download Presentation

Welcome ! DAY 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Welcome! DAY 1

  2. Facility Overview • Entering and Exiting • Rear entrance is closest to the auditorium • Perimeter doors are not secure • Do not exit through the auditorium rear emergency doors • Restrooms • Located near the 2nd floor front entrance, left of the double doors • In Case of Emergency • Please exist through the rear doors of the auditorium and proceed down the rear stairwell

  3. Facility Overview • Security • Be sure to sign in at the beginning of each day • Entrance into a Raytheon work area will require a Raytheon escort • Parking • No visitor parking registration is needed • Please park in unmarked spots to avoid being towed

  4. Workshop Overview • Food & Drink • Food and Drink are allowed in the auditorium. • Breaks & Lunches • Periodic breaks will be provided during the workshop • Lunch will be brought in for those who pre-registered • Wireless Access • Access to the RaytheonVAG wireless network requires a user name and password • Pre-registered attendees will have an account provided • Teleconferencing • The ECHO teleconference # (877.973.2179 Passcode 770291) will be used • A WebEx live session will also be available. Details provided separately. • Discussions • Questions should be asked as the presentations proceed. • Some discussions may be tabled for the designated Q&A periods

  5. Agenda Overview • Day One • 9:00am ~ 4:30pm • Focus entirely on Reverb discussion topics including: • Reverb Showcase • Reverb Functional Overview • Reverb Automation • Reverb Technical Topics • WIST2REVERB Transition • Reverb Future Changes • Day Two • 9:00am ~ 12:00pm • Focus shifted to ECHO engineering topics including: • Upcoming ECHO Changes • Upcoming Operational Activities

  6. ECHO Organization

  7. ECHO Team Introductions

  8. Reverb MVP Award • Honorable Mentions • Kelly Lemig – 42 NCRs • Jon Pals – 17 NCRs • Dawn Siemonsma – 15 NCRs • Sample Bug Report Peter Gibbons MVP

  9. Operations status

  10. ECHO Partner Status • Current Holdings • Collections 3,227 • Granules 94 million • Browse 35 million (internal) / 22 million (external) • Data Partners • Operational: 13 • ASF, GHRC, GSFC, LAADS, LARC (ECS), LARC (ANGe), LPDAAC, NSIDC (ECS), NSIDC (V0), PODAAC, ORNL, SEDAC, USGS_EROS • Test: 1 • CDDIS • Client Partners • Operational 5 • WIST, ESA, MTMGW, GMU, ECHO-ESIP • In evaluation or test 6 • MIRADOR (OpenSearch), YOTC (OpenSearch), Reverb, SCS3, Mercury, CWIC • Development/Design 3 • AVAPC, J-Earth, PO.DAAC

  11. ECHO Client Usage (1 of 2)

  12. ECHO Client Usage (2 of 2)

  13. Query Performance

  14. Order Metrics (1 of 2) *All orders submitted by WIST.

  15. Order Metrics (2 of 2)

  16. Ingest Activity < .25% Rejections

  17. ECHO Availability *ECHO HA solution deployed in March 2009 ** % Availability does not include PM (Up to 4 hours / week) outages *** Downtime (Hours) _does_ include PM outages

  18. Process Overview

  19. ONE GOAL

  20. BUILD THE RIGHT SOFTWARE

  21. Original Approach • Testbed • Alpha Release every 2 weeks • Open to the public • Early feedback • 20% Dev Time Reserved for feature request and prioritization • Partner Test (Beta) • Early beta – approx Feb 2011 • Open to the public with WIST announcement • Solicit public input on Reverb prioritization • Operational • May 2011 • Already have a base of users from beta testing • Smooth (ideally non-existent) transition from beta testing • Allow for continued evolution of Reverb

  22. Development Lessons Learned • Deployment overhead still too high to sustain 2 week alpha pushes • Nuances of new technologies contributed to costly build process • Feedback focused more on features not-yet implemented vs. testable functionality • Needed preview release in Partner Test sooner than later to test end-to-end ordering outside of Ops • 20% rework was sometimes difficult to plan for, but was very useful to address feedback concerns • “Top 5” requests were helpful, but under-utilized

  23. Modified Approach • Monthly alpha release pushes to Testbed • Point releases as needed to meet major milestones or to address significant bugs • Delayed beta testing until more functionality was implemented • Plan in 20% rework to each development sprint for bug fixes & enhancements per feedback • Continue to request “Top 5”, but start nailing down required bug fixes for major milestones early

  24. BETA RELEASE CHANGED THE DYNAMIC

  25. NCR Metrics Beta release prep-telecon Beta-release Initial Reverb Testbed deployment

  26. REVERB IS THE PLATFORM FOR FUTURE ECHO CLIENT INNOVATION

  27. Moving Reverb Forward • Core Reverb functionality will become part of monthly sustaining activities • Reverb will become Operational on May 16 • Development will continue with monthly Alpha pushes to Testbed • Reverb Beta will continue to exist and serve as a prototype space for T2R4 and potential client technology efforts • Operational Reverb will be updated as part of the normal Operational deployments Specific functionality will be discussed later today

  28. Reverb showcase

  29. Dataset Discovery • First class dataset discovery • Fast keyword filtering of datasets • Ability to add entire datasets to cart to support dataset download and ordering • Ability to applyservices directly todatasets • Temporal and spatialfiltering of datasetswithout doing agranule search

  30. Improved Form Rendering • Fully styled form rendering • Support for native controlsincluding spatial selectionareas on maps withmultiple projects • Inline error and helpinformation • Smooth form expansionand collapsing to showor hide input areas • Same rendering engine usedfor Orders and Service invocations

  31. Bulk Download • Support for bulk download of large number of items • Supports multiple download options • Text file of URLs to support wget and curl • FTP batch download script

  32. Services • Added dynamic servicediscovery and invocation • Services can be associated using tags • Leverages familiar ECHO Forms for service UI presentation to the user • Can be extended to supportwide variety of APIs (e.g.: GLASSubsetter, ESI) • User input can be as simple as selecting an area of interest and desired dataset.

  33. Maintainability • Reverb is based on modern web standards • Ruby on Rails3, HTML(5), CSS, JavaScript • Fully automated valids, ACLs, and keyword generation • Follows well established ECHO Development best practices • Extensive suite of automated tests

  34. Minimized Ops/DAAC Interaction • Minimized Ops/DAAC Coordination • Reverb reduces the number of items on which ECHO data partners need to coordinate with the ECHO Operations team. • Reduced coordination simplifies Reverb maintenance. • Data partners can modify Reverb interactions through modifying metadata.

  35. Reverb-Enabled Metadata Cleanup • Resolved Metadata Inconsistencies & Errors • Missing Collection Temporal Metadata • Missing Collection Spatial Metadata • Incorrectly defined “MODIS Tile” 2D Coordinate System • Invalid NSIDC Descending Crossing Longitude • Archive Center Cleanup • Identified Metadata Issues • Inconsistent Processing Level Usage • Invalid LARC_ASDC Granule Temporal Values • Conflicting “ASTER” Instrument & Sensor • Invalid OnlineResourceURLMimeTypes • OnlineResourcevsOnlineAccessUrl • Non-specific browse file extensions

  36. Reverb AUTOMATION

  37. WIST Valids • WIST • Valids configured to run once/hour for guest users. • Metadata or guest user access changes lags according to valids frequency. • Valids runs as a stand alone process, not integrated with WIST. • Validsprocessing runs on-demand when a user logs in. • There are no lagging issues having to do with metadata modifications. • Information Gathered for Dynamic Presentation • Collections available to guest users • Science Keywords • Additional Attributes • ECHO Operations Configured Information • Data Quality Summaries & Assignments • Radio Button Mappings • 2D Coordinate Systems • Additional Attribute Grouping

  38. Reverb Valids • Reverb • Valids processing configured to run once/hour for guest users. • Metadata or guest user access changes will lag in Reverb interface. • The valids process is integrated with Reverb and runs as a core part of the app. • Valids processing runs on-demand when a user logs in. (No Change) • Valids are refreshed every hour for the registered user. • Information Gathered for Dynamic Presentation • Search Terms • Platforms & Instruments • Campaigns • Science Keywords • Processing Levels • 2D Coordinate Systems • Calendar Events • Provider Holdings (Granule Counts) • Services (Including Dataset Tags)

  39. Calendar Events • WIST • Calendar events managed by providers in WIST Interface. • Username/Password not synchronized with ECHO system. • Alerts/Info displayed in scrolling banner & on separate calendar page. • Reverb • Calendar events managed by providers in PUMP & made available to all ECHO client partners through SOAP & REST API, or end users via RSS. • All provider and system calendar events loaded from the ECHO system. • NCR 11010933 tracking issue re: Reverb displaying non-active events • ALERT events displayed in “Availability” section & INFO events displayed in “Notices” section. • HTML formatting available for improved readability. • Data Partner Responsibilities • Management of ECHO calendar events belongs to Data Partners. • Continue to update WIST & Reverb to reflect accurate alerts & info events. • Remove non-active events when the end-date has passed.

  40. Data Quality Summaries • WIST • Configured by ECHO Ops team in cooperation w/ Data Partners. • Displayed when user adds a collection or granule to their cart. • If a registered user has accepted the DQS and the DQS changes, they will not be presented with the new DQS. • Reverb • Data quality summaries managed by providers in PUMP & made available to all ECHO client partners through SOAP & REST API. • Displayed when a user searches for granules within a dataset. • Modified DQS are displayed to registered and guest users. • HTML formatting available for improved readability. • Data Partner Responsibilities • Management of ECHO data quality summaries belongs to Data Partners. • Update Reverb & coordinate w/ ECHO Ops to accurately configure DQSs.

  41. Reverb technical overview

  42. Implementation Notes • Implemented using an Agile Process • Implemented using Ruby on Rails, HTML, CSS, JavaScript • Mostly built on top of new ECHO REST API • Remainder will be moved to Multiformat REST API late summer • Implementation time and team • ~10 months • ~6 developers

  43. Maintainability

  44. Maintainability • Nearly all testing is fully automated • Unit Tests using Test::Unit • Integration Tests using Cucumber, Capybara, and Celerity @javascript Feature: Spatial search for datasets In order to narrow my dataset results by spatial bounds as a Reverb user I should be able to visually select boundaries for my dataset searches Scenario: Type in a valid polygon Given I am on the home page When I select "Polygon" from "spatial_type" within ".spatial-field" And I wait for datasets to load When I fill in "spatial" with "-9.102, 40.430, 22.594, 22.852, -1.406, 1.055" within ".spatial-field" And I wait for datasets to load Then I should find the dataset with id "Dataset with normal cartesian bounding rectangle" And I should find the dataset with id "Dataset with normal cartesian polygon" And I should not find the dataset with id "Dataset with cartesian point”

  45. Deployment Model Reverb is deployed to multiple servers Users are transparently balanced across instances Reverb Reverb Like ECHO, Reverb can be updated without downtime. REST API REST API Reverb takes advantage of ECHO’s ability to distribute across DB nodes ECHO Kernel(SOAP API) ECHO Kernel(SOAP API) Reverb distributes its search and order load across all ECHO kernels ECHO DB ECHO DB ECHO DB

  46. Supported Browsers • Officially supported • Firefox • IE7 and IE8 • Safari on OS X • Unofficially tested • Firefox 4.x • Chrome 11 • Safari 5.x

  47. Supported Browse Image Types • Reverb will automaticallydisplay browser friendly types (JPG, PNG, etc.) • Reverb can detect HDFbrowse and automatically extract embedded JPGs • Multiple browse images are support and fetched asynchronously for performance

  48. Supported Spatial Projections • Searching • Google Maps Mercator • North/South Polar Stereographic • Subsetting Projections • North/South EASE-Grid (EPSG:3408, EPSG:3409) • Hughes / NSIDC North/South Polar Stereographic (EPSG:3411, EPSG:3412) • Spatial Extent Visualization (results) • OpenLayers • Plate Carree (equirectangular) • North/South Polar Stereographic

  49. Spatial extent drawing • Reverb can render multiple granule spatial extents • Will provide swath and polar rendering before release

  50. WIST 2 REVERB TRansition

More Related