Advanced Weather Interactive Processing System II (AWIPS II)

1. Advanced Weather Interactive Processing System II(AWIPS II) AWIPS Development Environment (ADE)and theCommon AWIPS Visualization Environment(CAVE) Module 3: Micro Engine and Scripts

2. Module 3: Micro Engine and Scripts Module 3 Objectives • Learn Why the Micro Engine Exists and the Role It Plays • Understand the Architectural Pattern of the Micro Engine • Learn How to Write and Test Micro Engine Scripts • Learn How to Extend the Micro Engine by Writing Micro Engine Tasks • Learn About Future Evolution of The Micro Engine

3. uEngine Task Execution PatternBreaks Up Execution Into Small Reusable Tasks • uEngine Vision • Create an execution framework for generating custom products on demand • Customer systems can request products by script requests over a network • The script performs small general units of work that get chained together to produce a customer product

4. uEngine Task Execution PatternBreaks Up Execution Into Small Reusable Tasks • uEngine Implementation • Runs out of an SOA services “ProductSrv” and “AutoBldSrv” that are attached for I/O to ESB endpoints • uEngine executes scripts in a XML language similar to “ANT” • Utilizes the open source project “common-digester” for high-performance XML to object parsing

5. uEngine Used Out of Two SOA Services ADE 1.0 Services JMX Remote Service Management Client Delivered T03 Delivered T04 Delivered T05 Mbean Mbean Mbean Mbean Mbean IngestSrv PersistSrv IndexSrv ProductSrv AdapterSrv CAVE Visualization Client ASCIIIngestSrv FileSystem Lucene – Reverse Idx uEngine Scripts Exec Adapter Data Rendering GridIngestSrv RDBMS via JDBC Spatial R-Tree Grid Diagnostics Hydro Adapter Notify Response SBNSatIngestSrv Dissemination Jini Adapter Data Interrogation RadarSrv CAVE Scripting LightingSrv Wx Drawing ProductSrv Wx Warning Hydro Visualization Mbean Mbean Mbean Mbean Mbean NotifySrv SubscribeSrv AutoBldSrv ColabrateSrv GridEditSrv uEngine Scripts SubscriptionNotify AlertNotify Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints Services Independent of End Points

6. uEngine Design Concepts • Idea originated out of ANT files • Think of uEngine tasks as being similar to ANT tasks inside <target> • Most uEngine script tasks are “bodiless” • Simplicity is guiding concept • Tasks get executed from top to bottom in sequence, no controls • Added Apache Commons Digester XML parser • Layered on SAX parsing • Improves the performance of XML parsing • Provides rule-based XML to Java Object mapping

7. uEngine Design Concepts • Each Task defines its own parsing rules • Rules for a task can be changed with minimal impact on the micro engine • The abstract Task class provides convenience methods for many common rules • Class bundle in uEngine identifies tasks • uEngine able to identify tasks included within its jar • Enables new tasks to be added with a system build • New tasks are “pluggable” – they do not require modifications to the existing uEngine

8. <<ESB Service>> ProductSrv +receive() <<Java Class>> Launcher <<Java Class>> Action +parseAndRun() +executeTasks() uEngine Task Execution PatternCommon-Digester Based XML Rule Driven Parsing <action name="ServiceGribImageDecode"> <termQuery> <query name=“grid_type” value=“22” /> </termQuery> <degrib/> <gribMap colorMap=“GribRGB” /> <colorImage colorMap=“GribRGB” /> <imageOut format=“png” /> <fileOut/> <worldFileInfo/> <makeServiceResponse /> </action> {String output = (String) myLaunch.parseAndRun(msg);} Commons Digester {action = DigesterSingleton.parse(reader); action.executeTasks();} {Task task = i.next(); task.init(retObj); retObj = task.execute();}

9. Task makeServiceResponse worldFileInfo termQuery colorImage imageOut gribMap degrib fileOut <<abstract>> Task +addRules(Digester) +init(ChainDataType) +ChainDataType execute() uEngine Task Execution PatternConnecting Units of Work Together - ChainDataType <action name="ServiceGribImageDecode"> <termQuery> <query name=“grid_type” value=“22” /> </termQuery> <degrib/> <gribMap colorMap=“GribRGB” /> <colorImage colorMap=“GribRGB” /> <imageOut format=“png” /> <fileOut/> <worldFileInfo/> <makeServiceResponse /> </action> Input ChainDataType query XML child elements Output ChainDataType colorMap colorMap format Adding a new uEngine Task 1. Extend Task 2. Implement addRules() 3. Implement abstract methods

10. mathScript uEngine Task Execution PatternData Transformation Into Decision Aids - Scripting

11. Writing and Testing uEngine Scripts • General Rules • Remember the KISS principle – scripts generally create a single product • Script generally follow the Input-Process-Output (IPO) pattern • Each script starts with a data query – you need to know the data that may available

12. Writing and Testing uEngine Scripts • A Scripting Example We want a script that will return the latest TAF for Omaha Eppley Airport. The ICAO for Omaha Eppley is KOMA. The data should be returned to the client as a raw TAF record. • AnalysisIn this case, there is no processing to be performed. We simply retrieve the data and return it to the client. Two uEngine tasks are used: • termQuery, which is used to query the data store • makeResponse, which returns the data to the client

13. Writing and Testing uEngine Scripts • The Basic Tag Formats: • action – represents the script document. Encloses other tags. <action name="…"> </action> • termQuery – defines a query to the data store <termQuery> <query name="…" value="…" /> </termQuery> • makeResponse – prepares the data to return to the client <makeResponse returnMethod="…" />

14. Writing and Testing uEngine Scripts • The Completed Script <action name="Get TAF"> <termQuery> <query name="reportType" value="TAF" /> <query name="stationID" value="KOMA" /> </termQuery> <makeResponse returnMethod="ascii" /> </action> • This Returns the Raw Data to the Client • Will Revisit This One on the Lab

15. Writing and Testing uEngine Scripts • A Script Example In this example, we again retrieve the latest TAF for KOMA, but in this case we will return the decoded report in XML format. • Analysis Two additional uEngine tasks are needed here: • decodeAscii, which is used to decode the record • asciiXML, which converts the decoded report to XML format • Note: To be useful, the client needs to “understand” the XML formatting.

16. Writing and Testing uEngine Scripts • The Basic Tag Formats: • decodeAscii – runs the TAF decoder. We will revisit this one later since it utilizes the plug-in pattern in performing the decoding. <decodeAscii /> • asciiXML – converts the decoded TAF to XML. This one also utilizes the plug-in pattern. <asciiXML /> • Note: Neither tag requires that the ascii report type be identified. This is part of the plug-in pattern.

17. Writing and Testing uEngine Scripts • The Completed Script <action name="Get TAF"> <termQuery> <query name="reportType" value="TAF" /> <query name="stationID" value="KOMA" /> </termQuery> <decodeAscii /> <asciiXML /> <makeResponse returnMethod="ascii" /> </action> • Will Revisit This One in the Lab

18. Writing and Testing uEngine Scripts • Summary • Saw two example scripts that work with ASCII data • More examples in the documentation included on the DVD

19. BREAK

20. Hands-On Micro Engine/Scripts • Running the Development Version of EDEX • Script Builder Web Page • Running a Script • Debugging a Script • Script Editor Web Page

21. 2 1 3 Running EDEX • Install EDEX on Development Computer • To Start EDEX: Double Click the Start ActiveMQ, Start Mule, and Start Tomcat Icons on PC Desktop • Linux Users: Start Edex From a Terminal

22. AWIPS ADE 0.1 Includes a Web Site That Can Be Used to Build/Debug uEngine Scripts Does not have an empty editor page; we will use the GRIB Retrieval Page Open the Browser. Navigate to uEngineWeb Test Page. Click GRIB Retrieval Link. Delete Contents of Blue Box. Enter the Script to Test. Script Builder Web Page Steps:

23. Enter the script into the blue box Click the Get Image button Running a Script

24. This is the TAF. The original formatting is retained, but not reflected on page. Running a Script: Results

25. Enter the script into the blue box Click the Get Image button Running a Script: Example 2

26. This is the decoded TAF. XML formatting is hidden by the page rendering. Running a Script: Results

27. uEngine Will Detect Common Scripting Errors Invalid XML Invalid search parameters Missing tags This Error Caused by an Invalid (<degrib /> uEngine Script Tag Debugging a Script

28. Script Editor Web Page • AWIPS Test Driver Interface may be used as a script builder • This interface used create scripts for this demo • Expand the page to see the script

29. BREAK

30. Extending the uEngine by Adding New Tasks • Steps Involved in Writing New uEngine Tasks • Writing a “Hello World” Task • Testing Your New Task

31. General Rules • Remember the KISS principle – each task should perform a single job • Most tasks extend Task. May also extend an intermediate class • Jython tasks extend JythonTask rather than Task, but JythonTask extends Jython. • Override the static addRules(Digester) method • At a minimum, call the static addBasicRules(…) method • Implement the abstract init(ChainData), and execute() methods • The abstract test(ChainData) method is currently deprecated and can be implemented by simply returning the input

32. The “Hello World” Task • Problem Statement We want to create a uEngine task so that a uEngine script can print a “Hello World” message. • Analysis In this case, we will interpret “print” as log a message. This allows for creation of a somewhat more useful task, one that will allow the script writer to log messages. For simplicity, the logging will be at “info” level.

33. The “Hello World” Task • The XML Tag Syntax <echo msg="…" /> The tag name will be echo. The text to be logged will be provided via the message attribute. The message will be static, i.e., it will be determined at the time the script is written. • Log Output Typical log output will be similar to INFO 2007-01-16 14:07:46, 964 [ProductSrv.2] tasks.Echo Message: Hello World!

34. The “Hello World” Task • Basic Class Structure • Basic class generated by Eclipse • Import statements added by Eclipse as needed “msg” and setMsg() follow Java bean naming pattern.

35. The “Hello World” Task • addRules(Digester) method: • “Echo.class” specifies the class and is required since this is a static method • Uses the rules method from Task and previously defined constants to simplify code

36. The “Hello World” Task • The init(ChainData) and Test(ChainData) Methods Init() simply saves its input for later, “dataChain” is provided by Task. test() has nothing to do, so simply return the input.

37. The “Hello World” Task • The Execute() Method • All the action takes place here Use the logger to record the message All Tasks return the dataChain.

38. The “Hello World” Task – Using Eclipse Using the Eclipse Class Creation Wizard to create the basic class.

39. The “Hello World” Task – Using Eclipse • At this point, you compile and deploy EDEX • Once deployed, test the task in a uEngine script

40. The “Hello World” Task – Test Script • Enter the test script as before, and click “Get Image”. • Check the system log to see the message.

41. BREAK

42. Summary • Future Evolution of the uEngine Design

43. Resources • On the ADE 0.1 DVD: • micro_engine_task_readme.pdf • micro_engine_scripting_readme.pdf • generic message format.pdf • AWIPS_ADE_EDEX_Build_Procedure.pdf • JavaDoc documentation for plug-ins is available. • On the Web: • Apache Commons Digester project: • http://jakarta.apache.org/commons/digester / • Apache Jakarta Commons project: • http://jakarta.apache.org/commons/ / • Apache Ant project: • http://ant.apache.org/

44. BREAK