Implementing Critiquing Question #3 using Patterns

1. Implementing Critiquing Question #3 using Patterns UT RKF Group June 26, 2002

2. The process of authoring a pattern is no different from entering an ordinary CMAP. The SME would start off by selecting “Enter Knowledge” from SHAKEN’s main menu followed by selecting “Create a New Concept” in the subsequent menu. The pattern would then be given a name. The concept “Pattern” would then be selected as the superclass.

3. The pattern would be drawn like any other CMAP. The only thing special about entering a pattern is the following: • The SME specifies a critique criterion using property values. • The SME associates a question with a pattern. • Let’s see how this is done.

4. Here’s one way that a question might be associated with a pattern. This will bring up a window in which the SME enters the question. An new option called “Associate Question” is added in a pop up menu. The SME would access this option by right clicking on the screen and selecting it.

5. A critique criterion is added to a pattern via the existing property values mechanism.

6. Another piece of information that can be added to a pattern is a text-generation template. We envision this being used by SHAKEN in generating the final report. The next couple of slides is a proposal for this mechanism. The SME would enter a description of the pattern.

7. Continuing with the example, the SME selects “aircraft” from the text template. The SME can associate a phrase with a node in the pattern by highlighting the phrase and then pressing the select button.

8. Now the SME selects the node labeled “Aviation-Batallion” from the Pattern Selected node. This will bring up the pattern, and the SME can select the node by clicking on it.

9. The label on the selected node (put in angle brackets) replaces the selected text in the template. The SME repeats this process for other words/phrases in the template, then selects “Done”. When SHAKEN generates a final report critiquing a COA, the phrase <aircraft> will be replaced by the called tag of the COA node that matches the Pattern node.

10. Implementation of the SME’s text template. • The output of the SME’s text template can be represented in SHAKEN’s .km file as: The output of the SME’s text template. (_Destroy-Artillery-Early1363 has (instance-of (Destroy-Artillery-Early)) (critique-score (_Critique-Score-Value1377)) (coa-description ('("The" _Aviation-Batallion1365 "destroy" _Artillery-Battery1366 "before" _Attack1369))) (has-pattern (_Destroy1376)) (new-objects (_Destroy1376 _Aviation-Batallion1365 _Artillery-Battery1366 _Time-Interval1367 _Time-Interval1368 _Attack1369 _View1370 _Allegiance-Value1371 _Property-Node1372 _Allegiance-Value1373 _Property-Node1374 _Critique-Score-Value1377 _Property-Node1375))) *With the exception of part circled in red, the above contents can be found in SHAKEN in the file Destroy-Artillery-Early.km under the user porter.

11. To critique a COA, the SME would select “Ask a Question” from SHAKEN’s main menu. They would then enter the name of the COA to be critiqued. This will bring up SHAKEN’s Q/A subsystem.

12. The question associated with the pattern Destroy-Artillery-Early. Assume the SME asks the first question in the list. SHAKEN then applies the pattern associated with that question. At this point, SHAKEN would call UT’s pattern-match function (see http://www.cs.utexas.edu/users/mfkb/RKF/API.txt for the APIs of UT’s functions). The output from the function will then be used to construct the critique report shown later. Let’s first look at the output generated by UT’s pattern-match function.

13. The Output of a Match Pattern’s Name • Below is the actual output of applying the pattern Destroy-Artillery-Early to COA-Ja (Destroy-Artillery-Early and COA-Ja can be found by logging in as porter and barker respectively). A graphical explanation of the output is provided on the next slide. (|Destroy-Artillery-Early| (((|_Allegiance-Value1371| |value| (|*Blue|)) ((|_Allegiance-Value2252| |value| (|*Blue|)))) ((|_Aviation-Batallion1365| |allegiance| |_Allegiance-Value1371|) ((|_Aviation-Batallion2309| |allegiance| |_Allegiance-Value2252|))) ((|_Destroy1376| |agent| |_Aviation-Batallion1365|) (((|_Aviation-Batallion2309| |agent-of| |_Attack2275|) (|_Attack2275| |causes| |_Neutralize2303|)) ((|_Aviation-Batallion2310| |agent-of| |_Attack2275|) (|_Attack2275| |causes| |_Neutralize2303|)))) ((|_Allegiance-Value1373| |value| (|*Red|)) ((|_Allegiance-Value2251| |value| (|*Red|)))) ((|_Artillery-Battery1366| |allegiance| |_Allegiance-Value1373|) ((|_Artillery-Battery2305| |allegiance| |_Allegiance-Value2251|))) ((|_Destroy1376| |object| |_Artillery-Battery1366|) ((|_Neutralize2303| |object| |_Artillery-Battery2305|))) ((|_Attack1369| |time| |_Time-Interval1368|) ((|_Attack2263| |time| |_Time-Interval2280|))) ((|_Time-Interval1367| |before| |_Time-Interval1368|) ((|_Time-Interval2280| |after| |_Time-Interval2304|))) ((|_Destroy1376| |time| |_Time-Interval1367|) (((|_Time-Interval2304| |time-of| |_Attack2275|) (|_Attack2275| |causes| |_Neutralize2303|))))) " The 1st-air and 2nd-air destroy enemy artillery before Main-Attack1.") From the Pattern. From the COA. Match correspondences between the pattern and the COA Instantiation of the text template with the called tags of the matching nodes from the COA.

14. To clarify the output, here’s a graphical view of it. We’re not suggesting that this be generated automatically. Application of the heuristic rewrite: if Time-Interval time-of Event1 and Event1 causes Event2 then Time-Interval time-of Event2 Application of the heuristic rewrite: if Entity agent-of Event1 and Event1 causes Event2 then Entity agent-of Event2 *The other parts of the pattern and COA was able to be matched without the use of any rewrites. These correspondences are not shown here for the sake of brevity

15. SRI computes the overall score by (somehow) tallying the individual scores provided by the critiquers. Options for producing an explanation are shown on the next slide. This critique is generated by instantiating the text template: “The <aircraft> destroy <enemy artillery> before <attack>.” with the called tags of the nodes that match selected nodes of the Pattern. In this example, the underlined phrases come from called tags on nodes in COA-J.

16. Explaining a Critique • low road: show the COA filtered by the pattern that generated the critique. Invite the SME to see for himself how the pattern and the COA got aligned by the matcher. This requires no new functionality (beyond what we’ve already proposed) because SHAKEN can get the information needed to filter the COA from the output our pattern-match function. • high road: show the COA and the pattern and the correspondence between the two. Showing this information graphically would be best. This information is returned by the pattern matcher, but presenting it to the SME graphically would require new functionality.