Authoring and Critiquing COA’s in SHAKEN: A Storyboard showing work in progress

1. Authoring and Critiquing COA’s in SHAKEN: A Storyboard showing work in progress UT May 2002

2. Disclaimers • SHAKEN cannot be used to author some aspects of COA’s, e.g. • More sophisticated spatial knowledge (e.g. “Attack the enemy’s eastern flank”; “Direct the fires deep within the battlespace”; “The 3rd Brigade Overtakes the Cavalry”) • SHAKEN cannot simulate some aspects of COA’s, e.g. • Non-normative outcomes (e.g. “The attack might fail”)  Other components of the E2E system – such as the Umass Simulator and NuSketch – should contribute to these areas.

3. Our Focus at UT • Authoring COA’s:We’re expanding the component library with the Events, Entities, Roles, and relations that are required for describing BS situations and COA’s. We’re anticipating that spatial information will come from NuSketch. • Critiquing COA’s:Several sources can contribute here, including KANAL’s check of pre-req’s and expected affects, as well as Umass’s non-normative simulations. We’re planning to contribute pattern recognition methods which are applied to declarative descriptions of COA’s to identify their strengths and weaknesses. Also, we’re planning to contribute to methods for SME’s to author knowledge to improve these critiques. Let’s discuss each of these in turn …

4. 1. Authoring Battlespace Descriptions • We’ve identified several areas of expansion for the component library • Goals • things an agent intends to accomplish/prevent/etc. • has-goal, objective, Goal, Intentional (cliché), Unauthorized (cliché) • Resources • things used/consumed/output/etc. by Actions • resource, by-product, relationship to participant slots • Views • user-defined collections of participants • viewpoint, View (Aggregate)

5. Library Expansion (cont.) • Time • Event time, Entity “time”, relative time, Temporal Roles • start-time, end-time, time-of-existence, duration, rate, frequency, etc. • Space • Spatial frames (e.g. “geospace”), Spatial Roles, Granularity • Conduit, Corridor, Landmark, Obstacle, Axis, Hide, Cross, Approach, etc. • Communication • Information, Carrier, Medium, Express, Interpret, Transmit, etc. • Encode, Decode, Language, Learn, Confirm, Monitor, Discover, Deceive, etc. • Aggression • Trespass, Invade, Attack, Shoot, Impair, Damage • Protect, Defend, Die/Kill, Live/Survive, etc.

6. Library Expansion (cont.) • “Meta-Events” • Assist, Maintain, Begin, Stimulate, Stop, Control, Motivate, Oppose, Delay • re-, counter-, • Reciprocal, Intentional, Unauthorized http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentations/models.ppt `

7. An Example COA in SHAKEN • We’ve encoded General Otstott’s COA#1 for Objective Jayhawk. A CMAP for it appears on the next slide. • The CMAP is a good start, but it’s incomplete in two ways: • We could not well express some parts of the COA. We’re continuing to expand the component library to add this expressiveness. • We omitted all spatial aspects of the COA, anticipating that this information will come from NuSketch. • Nevertheless, this COA is fully coded in SHAKEN (i.e. it’s not a mock-up) and it gives us a useful laboratory for further work.

8. Put the Cmap for the COA here Enemy Ally

9. A Brief Aside:Enabling SME’s to “speak loosely” while authoring knowledge in SHAKEN • Loose Speak (LS): the phenomenon that human listeners are able to correctly interpret a speaker's imprecise utterance. • Speaking loosely while authoring knowledge requires the correct interpretation of a SME's imprecise utterance. • The next couple of slides show a few examples of loose speak in authoring knowledge in shaken. • For more detail please visit: • http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentations/Pete-slides.ppt

10. Example 1 of LS Given input: Following their attack, attack aviation will refuel and rearm SME encodes the following CMAP: Note: Attack-Aviation is encoded as the object of Refuel and Rearm, but the object of Refuel and Rearm should be Military-Unit

11. Example 1 of LS (Continued) • The Loose-Speak mechanism recognizes the inconsistency, and asks: If the user clicks on the “(See CMAP)” link …

12. The Updated Version of CMAP

13. Example 2 of LS Given input: Fire support starts with the attack SME encodes the following CMAP: Note: The temporally-starts slot is incorrectly used to link the two events directly.

14. Example 2 of LS (Continued) • The LS mechanism links the temporally-starts slot to the time-intervals of Fire-Support and Attack, and asks the user to “Accept” or “Cancel” the new version.

15. Example 3 of LS Given Input: Holding Attack SME enters the new concept:

16. Example 3 of LS (Continued) • The LS mechanism attempts to interpret the new concept: If the user clicks on the “(See CMAP)” link …

17. Example 3 of LS (Continued)

18. 2. UT’s approach to enabling SHAKEN to critique COA’s • Knowledge for critiquing takes the form of patterns to match against COA’s. Each pattern has associated conclusions to draw or actions to take when the match is successful. • For example, General Otstott’s critique of COA#1 included this observation for fire support: • We envision that SHAKEN will generate this observation by matching the patterns on the next two slides with our COA#1 CMAP. COA#1 makes effective use of attack aviation assets by attacking deep to neutralize the enemy artillery early. Attack aviation is available to assist attacking ground elements by the time the attack is joined.

19. An Example of a Pattern An attack by air assets that is able to penetrate and neutralize the enemy artillery before the start of the main attack offers good fire support. time Attack causes Penetrate causes Neutralize agent object object object Time-Interval before Aviation-Battalion Artillery-Battery time Time-Interval Attack plays element-of Enemy “Main Attack”

20. Another Example of a Pattern A military unit that supports an army division when it begins its attack offers good fire support. Time-Interval Time-Interval time time temporally-starts Attack Fire-Support supports agent-of agent Military-Unit Army-Division

21. Pattern Matching • The critiquing patterns are unlikely to match a COA exactly – flexible matching is required. • We’re compiling knowledge to enable flexible matching. Some is sound, some is heuristic. • Matching knowledge is of the form X  Y. X might match with the KB and Y with the user's input or vice versa. • The next couple of slides show the matching knowledge that we have compiled so far. • For more detail please visit • http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentations/Project-talk-final.ppt

22. Matching Knowledge • Some slots are transitive, symmetric, and/or reflexive. For example, has-part is transitive, so • Some slots ascend through parts. For example, object ascends through is-part-of, so • We have compiled extensive (sound) knowledge of the form: For example,

23. More Matching Knowledge • We have also compiled knowledge which encodes complex relationships. For example: • Our matching knowledge also includes heuristic rules, such as:

24. Matching COA#1 through transitivityof causes

25. Matching COA#1 (cont.)

26. An Aside: Using the Pattern Matcher to Improve SHAKEN’s Q/A system • The Q/A system does not look far for an answer to a query. The knowledge we’re compiling for the flexible matcher can broaden its search. • For example, a SME might ask of COA #1 “What attacks did the 23rd Battalion make?” This is expressed in KM as: (the Attack agent-of of 23rd) • The current SHAKEN will return nothing, even though an answer exists. SHAKEN just needs to look a little harder…

27. Improving SHAKEN’s Q/A The flexible matcher can find an answer by applying the matching knowledge we have compiled. “What attacks did the 23rd Battalion make?” “23rd” agent Attack End of Aside. Now back to the main theme of generating a critique…

28. Generating the Report Whenever SHAKEN matches a pattern with a COA, it adds an entry to the final report. Associated with each pattern is knowledge about the dimension of COA critiquing (e.g. Simplicity) that the entry contributes to, and the strength of that contribution (these are somehow tallied to score the COA on the {--, -, +, ++} scale). In SHAKEN, this might be: Pattern pattern time Attack causes Penetrate causes Neutralize contributes-to strength agent object object object Time-Interval Feasibility before Aviation-Battalion Artillery-Battery Strength-Value *high time Time-Interval Attack plays The pattern itself (rooted at “Attack”) might be enclosed in a big node element-of Enemy “Main Attack”

29. Generating the Entry in the Report We could have “canned text” associated with each pattern. A nice alternative would be to have a text template associated with each pattern, then to instantiate the template using the particulars of the COA that got matched. Here’s an example template: time Attack causes Penetrate causes Neutralize Attack by Aviation-Battalion is expected to penetrate and neutralize Artillery-Battery before the start of Attack, thereby offering good fire support. agent object object object Time-Interval before Aviation-Battalion Artillery-Battery time Time-Interval Attack plays Note: our current text generator would be used to generate phrases for the colored nodes. Those phrases would plug-in for the colored text. element-of Enemy “Main Attack”

30. Wouldn’t it be nice to haveA Tool for Editing Patterns • The SME’s main task this summer will be authoring knowledge to critique COA’s. “Basic SHAKEN” (i.e. component library, CMAPs) provides most of the functionality required for SME’s to enter patterns. • A “Pattern Editor” would provide a bit more. It would help a SME to design a pattern that successfully matches a COA, then to generalize it.

31. CMAP Pattern SHAKEN proposes correspondences between a CMAP and a Pattern. The SME is allowed to revise the Pattern, generalizingnodes, editing correspondences, etc.