Exercise Controller and Evaluator Briefing - [Exercise Date]

1. [Insert exercise name] Controller and Evaluator Briefing [Insert exercise date]

2. Welcome & Introductions

3. Meeting Agenda • Exercise Overview • Exercise Location and Area • Exercise Schedule • Exercise Scenario • Exercise Control Concept • Exercise Controllers • Exercise Evaluators • Exercise “Administrivia” • Conclusion

4. Exercise Players [Insert list of exercise players]

5. Purpose [Insert purpose of exercise]

6. Objectives • [Insert list of exercise objectives. Use additional slides as necessary.]

7. Exercise Play • Exercise Play will be on [date], from [time] • Exercise Play will take place at [location] • Play will be restricted to the delineated areas within [location]

8. Exercise Safety • Safety isEVERYONE’Sconcern • Safety concerns override exercise execution • Be aware that operating in this environment is inherently dangerous • Controllers and Evaluators must immediately inform the Safety Controller or Senior Controller of safety concern. • Actual emergencies will be identified by the saying: “Real World Emergency”

9. Weapons Policy • It is standard exercise policy to ensure that every effort is made to provide a safe and secure environment at all exercises for its participants, observers/VIPs, control/evaluation staff, volunteers, and the general public. • Exercise planners and controllers will plan for and promulgate the following control measures regarding weapons, whether introduced as a simulated device during exercise play or used by law enforcement officers in their normal scope of duties: • For the purpose of this policy, a “weapon” means all firearms, knives, explosive devices, less than lethal weapons/tools/devices, and any object capable of causing bodily harm. • Qualified personnel (e.g., law enforcement, security, military) with the legal authority to carry weapons who have an assigned exercise role (e.g., responder and tactical team) and have the potential for interaction with other exercise participants will NOT carry a loaded weapon within the confines of the exercise play area.  They may continue to carry their weapons only after they have been properly cleared and rendered safe (i.e., no ammunition in chamber, cylinder, breach, or magazines) and only after being marked or identified in a conspicuous manner (i.e., bright visible tape around the visible stock or holster).   The use of an area clearly marked as “off limits,” and with assigned armed personnel to secure weapons in a container, vehicle, or other security area is acceptable, and should be consistent with host jurisdiction weapons security policies.

10. Weapons Policy • Qualified personnel (e.g., law enforcement, security, military) with the legal authority to carry weapons who are used to provide “real world” perimeter security for the exercise and have no assigned or direct interaction with exercise participants may continue to carry loaded weapons as part of their normal scope of duty. • ·        All other personnel with no legal authority to carry weapons will not bring, introduce, or have in their possession any weapon of any type in any area associated with the exercise.  Safety briefings will be provided to all exercise participants specifying provisions and policies regarding weapons before the start of the exercise. • ·        Simulated explosive devices, such as “flash bangs,” pyrotechnics, flares, smoke gre­nades, etc., will be handled and/or detonated only by qualified exercise staff or bomb technicians. • ·        Aggressive behavior will not be tolerated during exercise conduct, except in matters of self-defense.  Examples of aggressive behavior may include, but are not limited to, excessive speeding; uncontrolled animals (e.g., K-9s and horses); employment of defense products (e.g., mace, pepper spray, stun guns, tasers, batons); and forceful use of operational response equipment or tools (e.g., pike poles and hose lines used at full stream on victims). • Exceptions to this policy specifying special mitigating circumstances will be directed to the lead exercise planner, in writing, 30 days before the exercise.

11. Exercise Response • Players are going to respond to their respective locations from the Assembly Area by vehicle. • Vehicles should respond with the flow of traffic when the Assembly Area Controller releases. • No red or blue lights should be used to respond to the exericse.

12. Exercise Play Area • [Insert exercise play area map]

13. Exercise Schedule • [Insert exercise schedule]

14. Scenario Overview Date: [Insert] Time: [Insert] Setting: [Insert] Incident: [Insert] Weather: [Insert]

15. Exercise Background Scenario [Insert scenario]

16. Exercise Documentation • Exercise Plan (EXPLAN) • Controller/Evaluator Handbook (C/E Handbook) • Guidelines • Exercise Evaluation Guides • Master Scenario Events List (MSEL) • Timeline • Messages • Activity Log

17. MSEL Timeline See MSEL Handout

18. Controller and Evaluator Overview

19. Control & Observation Concept • Exercise execution is the responsibility of the Exercise Director and the Senior Controller • Controllers/Evaluators organized by player location/function • Experienced controllers in key positions • Observers/Media remain in designated area

20. Control & Observation Guidelines • Coordinate activities with the Lead Exercise Planner, Senior Controller, and other controllers in your area • Notify the Lead Exercise Planner and/or Senior Controller of ANY problems related to safety or scenario play

21. Exercise Control

22. Controller Functions • Inject • Enforce • Report • Attend responder “Hot wash” • Attend final Controller/ Evaluator Debrief at the end of the exercise in one of the classrooms • Complete and turn in exercise documentation

23. Controller Responsibilities Senior Controller • Monitors exercise progress and makes decisions regarding any deviations or changes • Coordinates any required modifications Individual Controller • Introduces, maintains, and coordinates exercise events in accordance with the MSEL • Observes and reports exercise artificialities that interfere with realism Additional information is listed in the C/E Handbook

24. Controller Guidelines • DO: • Deliver injects promptly as directed by the Senior Controller • Notify the Senior Controller of events or need for changes • DO NOT: • Hold personal conversations with Players • Provide extra or advance information to Players • Prompt Players (unless directed by the Senior Controller/Lead Exercise Planner)

25. Exercise Evaluation

26. Evaluation Goals • Validate strengths, and identify improvement opportunities. • Validate plans, procedures, and protocols. • Improve processes and plans. • Determine level of capability in regard to the exercise’s target capabilities.

27. EEG Concept • The Homeland Security Exercise and Evaluation Program (HSEEP) Volume III provides evaluation and improvement planning policy. • Evaluations should be objective-based to remain in line with: • National Preparedness Guidelines • Target Capabilities List (TCL) • Jurisdiction and agency plans • HSEEP EEG templates provide streamlined process for standardized evaluation concept, not necessarily content. 27

28. Consistent across exercises from TCL Consistent across exercises from Universal Task List (UTL) EEG Concept (cont’d) • Capability-based approach relies on similar activities to achieve a capability. • Activities are achieved through successful completion of tactical tasks. • Task detail should come from actual plan being exercised. Content should be based on UTL detail but include actual text from applicable plan 28

29. EEG Function • Allows evaluators to review validation of plan elements • Plans used will depend on participating agencies • Includes direct references to plan sections, tables, and charts • Allows evaluators to better prepare for exercise • Provides detail for what will be expected of players • Remains focused on exercise objectives as they relate to the selected target capability 29

30. Use of EEGs • Evaluation team should use EEGs in conjunction with full plans to prepare. • Use EEGs as guides for observation, but do not focus on completing them during the exercise. • Use EEGs for data analysis after exercise and in preparation for the evaluator debriefing. • Use measured results to make actionable improvement recommendations within the After Action Report (AAR)/ Improvement Plan (IP). 30

31. Use of Evaluation Tools HSEEP Methodology Steps Evaluation Tools Timeline 31

32. Evaluator Notes and Data • Evaluators should capture player discussions, decisions, actions, and activity during exercise play. • The EEGs provided should serve as guides for documentation of exercise activity. 32

33. EEG Analysis Form • Each evaluator will be provided with EEGs that will give specific guidance on what data to collect during the exercise, how to record it, and how to analyze it. • These forms should be completed at the conclusion of the exercise, using data gathered during the exercise. 33

34. EEG AAR Data Input Form • Each EEG will include a second form to be completed following the C/E Debriefing. • This form will serve as an expansion to the notes captured in the EEG analysis form, including an area to provide recommendations. • Evaluators should complete one AAR input form for each identified strength or area for improvement. 34

35. After Action Report (AAR)/Improvement Plan (IP) • Data from the EEGs and debriefing will be compiled into the AAR/IP. • The AAR/IP will include assessments for each capability observed as well as issues and recommendations. • AAR/IP data will be entered into a corrective action tracking system. Agency Corrective Action Program 35

36. Evaluation Process

37. Evaluator Responsibilities • Understand the exercise concept and scenario. • Collect information. • Make observations on player deliberations, actions, and decision processes. • Analyze actions and develop postexercise AAR/IP input. 37

38. Preexercise Responsibilities • Review appropriate plans, procedures, and protocols. • Participate in C/E Briefing. • Review appropriate exercise materials to be familiar with scenario, objectives, and target capabilities. • Review EEGs and other supporting materials for your area of responsibility. 38

39. Exercise Responsibilities • Monitor and document player performance. • Use EEGs to confirm objectives are being met. • Focus on critical tasks, as specified in the EEGs and plans being exercised. 39

40. Evaluator Guidelines At the start of the exercise, evaluators should: • Position themselves within the appropriate group or area • Determine an ideal location to observe exercise play • Be prepared to move within the group to follow movement of play Evaluators should not: • Focus solely on the group or organization leader • Interfere with exercise discussion or action 40

41. Observing the Exercise Evaluators must keep an accurate written record of important issues that take place during the exercise: • Who performed the action or made the decision? • What occurred? • Why did the action take place, or why was the decision made? • How did they perform the action or make the decision? 41

42. Recording Important Events • Evaluators should record events such as: • Initiating scenario events • Actions of players • Key decisions • Deviations from plans and procedures • Times of significant actions • Equipment used • Directives • Movement and activity • Discussion 42

43. Assessment, Review, and Analysis of the Exercise

44. Postexercise Analysis Responsibilities • Participate in the Hot Wash, and take notes on findings identified by players. • Summarize your notes and prepare for the C/E Debriefing. • Have your summary ready for the lead evaluator, including bulleted strengths and areas for improvement from activity analysis. • Use data to determine whether the exercise capabilities and plans were effectively implemented. • Identify observations as strengths or areas for improvement. 44

45. Player Hot Wash • Allows players to participate in a self-assessment of the exercise play. • Allows evaluators to clarify additional points and collect missing information. 45

46. Controller and Evaluator (C/E) Debriefing • Work to complete EEG analysis forms for submission. • Discuss degree to which each objective was completed. • Provide strengths, best practices, lessons learned, and possible IP recommendations. • Begin outlining issues to be included in AAR/IP on AAR data input forms. 46

47. AAR/IP Input • Use the EEG AAR data input form. • Complete one form for each strength and area for improvement. • Observation. Write a one-sentence statement that identifies the strength or area for improvement as noted in the activity analysis tables. • References. Reference relevant plans, policies, procedures, laws, or regulations that could be used to facilitate implementing recommendations for improvement or capturing strengths. • Analysis. Document detailed data on what you observed during the exercise specific to this individual observation. • Recommendations. Suggest steps to allow the participants to further improve the plans or share strengths. • All AAR input is due to [lead evaluator] no later than [date]. 47

48. Evaluation Matrix 48

49. Exercise Implementation

50. Exercise Artificialities Artificialities and constraints, such as the Assembly Area, may detract from realism. Some other artificialities include the following: • Surrogates may be playing in place of some key decision makers • Real distances may be reduced to fit play location constraints • The exercise will be played in real time. However, there may be some time jumps or acceleration to meet certain objectives