1 / 33

New Tools for a Better Air Quality System Data Flow EN 2012 May 31, 2012

New Tools for a Better Air Quality System Data Flow EN 2012 May 31, 2012. Topics to Cover. Air Quality System (AQS) flow automation enhancements Exchange Network Services Center AQS Plugins for OpenNode2 and EN-Node Q&A / Discussion. Jumping Through Hoops.

emoye
Download Presentation

New Tools for a Better Air Quality System Data Flow EN 2012 May 31, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Tools for a Better Air Quality System Data Flow EN 2012 May 31, 2012

  2. Topics to Cover • Air Quality System (AQS) flow automation enhancements • Exchange Network Services Center • AQS Plugins for OpenNode2 and EN-Node • Q&A / Discussion

  3. Jumping Through Hoops • AQS submissions via a Node or Node Client used to require extra manual steps • More difficult to access AQS submission logs and reports • Most users preferred to use EPA’s non-Exchange Network interface called CDX-Web Can I interest you in an AQS data flow?

  4. Fewer Flaming Hoops…More Puppies • Today, files submitted through the Exchange Network are automatically processed by AQS • Exchange Network Nodes can download the submission log and reports • New tools for submitting AQS data via the Network • Support for AQS submissions through CDX Web is being discontinued Perhaps this is more to your liking?

  5. How Does it Work? • AQS data flow (v2.2a) now uses the Exchange Network Header • The Header file contains information about the submission data file • Security Token • Exchange Network and AQS User IDs • Payload type: 1) XML or 2) FLAT • AQS Load Options • Final Processing Step • Stop on Error

  6. AQS Load Options The submitter defines the desired level of automation: • Stage: Submission file is uploaded to the AQS server and verified to be a valid AQS file type, but with no subsequent processing. • Load: The file is loaded into the AQS database and its contents validated. Sample measurements are left at the status “Pre-production”; all other data types (Site information, Monitor information, and QA information) are set to the status “Production”. • Post: All valid sample measurement data for the file is set to the status “Production”. • Post is the default setting

  7. New Submission Option #1 • Exchange Network Services Center https://enservices.epa.gov/login.aspx • Automatically creates EN Header • Supports manual submissions to AQS • Replacing CDX-Web for AQS submissions

  8. New Submission Option #2 Node Plugins for OpenNode2 and EN-Node • Customized for easy integration with 2 widely used air monitoring systems • OpenNode2 - .NET – AirVision and DR-DAS • OpenNode2 - Java – DR-DAS • EN-Node - .NET – AirVision • EN-Node – Java - AirVision • Automatically create EN Header • Support submissions to AQS and data publishing • In testing. Production-ready versions likely available in June for free download from EN website.

  9. Show and Tell • Exchange Network Services Center • EN-Node Plugin • AirVision • OpenNode2 Plugin • AirVision & DR-DAS

  10. EN-Node AirVision Plugin • AirVision Plugin Features  3 Operations Submit AQS data to CDX Publish AQS data (Solicit) Publish AQS data (Query)

  11. Installation File & Contents • URL: http://code.google.com/p/en-node2/downloads/list • File: AirVisionPlugIn.zip

  12. Configure AirVision Plugin • What is Configuration? • Business logics used by EN-Node to complete the data flow • Config. data saved in an external BEPL file (Business Process Execution Language) • Each Operation has its own BEPL file • Configuration tools • EN-Node Admin Console • BEPL XML file provided with the Plugin installation file • How to Configure: Step 1: Create a Domain (AirVision) Step 2: Register 3 operations under AirVision • Submit AQS data to EPA • Publish AQS (Query) • Publish AQS (Solicit) Step 3: Test Plugin Step 4: Publish AQS data at ENDS or EN Browser

  13. Data Needed to Configure AirVision Plugin Minimum Data Needed Plugin Configuration Options • AirVision End Point • EPA AQS End Point • Organization Name • NAAS ID & PWD • Plugin file location

  14. Before Configuration… Important! • Open 3 DataWizardCinfig.xml files with text editor • Enter Plugin file location

  15. Step 1: Create Domain • Create AirVision domain • Choose web services (Submit, Download, Query, Solicit, Notify) allowed for the domain. • Establish “Administrator” accounts to setup/modify this domain.

  16. Step 2: Register Operation(s) • Register Operation(s) • Submit AQS data to EPA: Task_SubmitAQSXMLData_DataWizardConfig.xml • Publish AQS (Query): Query_GetAQSXMLData_DataWizardConfig.xml • Publish AQS (Solicit): Solicit_GetAQSXMLData_DataWizardConfig.xml • Define • EPA Endpoint URL • Validation rule (if any) • Data viewing stylesheet (if any) • Define • Task schedule

  17. Step 2: Register Operation(s) • Register Operation(s) • Submit AQS data to EPA: Task_SubmitAQSXMLData_DataWizardConfig.xml • Publish AQS (Query): Query_GetAQSXMLData_DataWizardConfig.xml • Publish AQS (Solicit): Solicit_GetAQSXMLData_DataWizardConfig.xml

  18. Step 3: Test Operation(s) • Test Submit AQS to EPA • Test: Enter Start Date and End Date to a date earlier than the current date will execute task after the user clicks save. • Track submission status • CDX will return “Completed” or “Failed” • View • Download • Data Viewer

  19. Step 3: Test Operation(s) 2. Test AQS Data Query • Authenticate • Input Start Date and End Date for AQS monitoring date • Query will return data in XML Authentication Query Request

  20. Step 3: Test Operation(s) 3. Test AQS Solicit • Authenticate • Solicit (Input AQS monitoring date) • Get Status • Download AQS data (in XML) Authentication Solicit Request Get Status Call Download

  21. Step 4: Publish AQS Data at ENDS/ENB • Manual data input to ENDS: • Test: https://ends2.epacdxnode.net • Production: https://ends2.epa.gov • XML Data Upload to ENDS: • Auto Pull from ENDS: GetServices • Manual XML Upload to ENDS • Test: https://ends2.epacdxnode.net/Node2WS.svc • Production: https://ends2.epa.gov/Node2WS.svc • Dataflow: ENDS_v20 • Operation: Service Refresh

  22. Useful Tips for EN-Node AirVision Plugin • Implement Only the Operation(s) you need • Minimum data you need to configure Plugin • AirVision End Point • EPA AQS End Point • Organization Name • NAAS ID & PWD • Plugin file location • Useful resources • Plugin: URL: http://code.google.com/p/en-node2/downloads/list • Data Publication Guide: http://www.enbrowser.net/enbrowser/ClientBin/HowToPublishData.pdf • ENDS & FCD: http://www.exchangenetwork.net/2011/07/15/exchange-network-discovery-service-ends/ • Support • enfoTech, Lijie Xu: lijie_xu@enfotech.com

  23. Show and Tell: OpenNode2 AQS v2.2 Plugin

  24. OpenNode2 AQS v2.2a Plugins • http://code.google.com/p/opennode2 • .NET OpenNode2 • AirVision • DrDAS • Java OpenNode2 • DrDAS

  25. OpenNode2 AQS Submit Workflow AirVision Or DrDAS Web Service OpenNode2 EPA CDXNode 1. Request Data 3. Submit Data 2. Return Data • On a set schedule (or manually triggered), request data from AirVision or DrDAS • Parameters set to filter data request • AQS XML data is returned to OpenNode2 • Relay the data to EPA CDX/AQS system • Optionally, header is added • Optionally, notifications can be sent

  26. Get Status/Download Workflow OpenNode2 EPA CDX 1. Get Status Status Response 2. Download Request 3. Notifications Files • On a set schedule (or manually triggered), check status of outstanding submissions • If CDX returns “completed” or “failed”, download processing reports from CDX • Do nothing if status is “pending” • Optionally, send notification emails to agency staff • Mail sent for all submissions where status changes from “pending” to “completed” or “failed”

  27. Publishing AQS Data from OpenNode2 • “Publishing” makes data available for any authorized partner to come and query from your node • Exposed as Query and/or Solicit services • No additional setup needed to make these services available from OpenNode2 • Setting up the services in Node Admin tool makes them available with a simple checkbox

  28. Setting up the AQS Plugin/Flow • Very similar to setting up any other OpenNode2 data flow • Using the Node Admin interface: • Create the AQS flow • Upload the plugin (contains all workflow logic) • Configure the Services (exposes plugin logic) • Service 1: Get data from AirVision/DrDAS for submission to EPA • Service 2: Check status of submission and download results • Create the Schedules (automates workflow) … and that’s it for the OpenNode2 portion

  29. OpenNode2 AQS Service Setup View in Node Admin of AQS flow with services configured: Flow 1 Service 1 Service 2 Flow 2 Service 1 Service 2

  30. AQS “GetXmlData” Service Detail Note: AirVision settings shown DrDAS settings very similar

  31. AirVision Schedule Detail

  32. DrDAS Schedule Detail

  33. Questions? Kurt Rakouskas301.531.5186kurt@exchangenetwork.net

More Related