AV-8B FRS AIRCREW TRAINING STUDY

1. AV-8B FRS AIRCREW TRAINING STUDY • Study Goals • Is the Fleet Replacement Squadron (FRS) syllabus meeting its stated goals and objectives? • What sort of instructional approaches are most likely to facilitate program goals? • What type of role should technology play in supporting the FRS syllabus? Apogee Solutions The Performance Improvement Company

2. AV-8B FRS AIRCREW TRAINING STUDY • The CIPP Training Evaluation Model • Context Evaluation looks at the setting and goals for the training program. • Input Evaluation Learner entry knowledge and skills. • Process Evaluation Instructional process analysis. • Product Evaluation Learner exit knowledge and skills. • The CIPP Model maps to the operational evaluation process described in MIL-HDBK-29612-2 Apogee Solutions The Performance Improvement Company

3. AV-8B FRS AIRCREW TRAINING STUDY Apogee Solutions The Performance Improvement Company

4. AV-8B FRS AIRCREW TRAINING STUDY • Measures • Review relevant USMC AV-8B publications and course materials. • Conduct training observations. • Conduct individual and group interviews. • Conduct surveys. Apogee Solutions The Performance Improvement Company

5. AV-8B FRS AIRCREW TRAINING STUDY • AV-8B Material Reviews Included: • Training and Readiness Manual (T&R) Feb. 1999. • Tactical Manual (TACMAN), Volume 1. • Tactical Standing Operating Procedures (TACSOP). • Naval Air Training and Operating Procedures (NATOPS). • FRS Flight Syllabus Guide (FSG) Sept. 2000. • RP Course Materials. • Classroom Presentation Materials. • Learning Center Self-Paced Materials. Apogee Solutions The Performance Improvement Company

6. AV-8B FRS AIRCREW TRAINING STUDY • Training Observations • Classroom. • Learning Center (LC). • Simulator Events. • Preflight Briefings & Debriefings. • Head Up Display (HUD) tapes. • Sortie Observation – FBO. Apogee Solutions The Performance Improvement Company

7. AV-8B FRS AIRCREW TRAINING STUDY • Individual Interviews • FRS Phase Heads (6) • Squadron Operations Officers (7) • Squadron Pilot Training Officers (6) • Competent Pilots (7) • Interviews were 60 minutes in duration. • All interviews were audio-taped. • Served as basis for survey design. Apogee Solutions The Performance Improvement Company

8. AV-8B FRS AIRCREW TRAINING STUDY • Group Interviews • RPs (25% complete) (6) • RPs (50% complete) (8) • Interviews were 60 minutes in duration. • All interviews were audio-taped. • Served as basis for survey design. Apogee Solutions The Performance Improvement Company

9. AV-8B FRS AIRCREW TRAINING STUDY • Surveys • RP (18), IP (15) & CP (25) • Paper-based • Demographic information • 39, 37 and 33 statements respectively • Likert scales for responses • Data collected in November - December, 2001 Apogee Solutions The Performance Improvement Company

10. AV-8B FRS AIRCREW TRAINING STUDY • FRS Training Goals • Combat Capable Pilot • Training covers day and night familiarization, instrument and formation flights, and initial exposure to corecompetencies. • 60% Combat Ready Pilot (CRP) • Pilot can perform any low/medium-threat, day/night OAS mission, as a wingman, from the main base or appropriate sized expeditionary airfield. Apogee Solutions The Performance Improvement Company

11. AV-8B FRS AIRCREW TRAINING STUDY • FRS Training Context • During FY01 there were 12 course convenings, each approximately 27 weeks in length. • 36 simulator events and 57 training sorties. • Student throughput for FY01 was 51 REs (replacement equivalents). • RP pooling eliminated. • FRS Grads not combat capable or 60% CRP. Apogee Solutions The Performance Improvement Company

12. AV-8B FRS AIRCREW TRAINING STUDY • FRS Training Context • New FSG to be fully implemented by Q4 FY02. • New FSG calls for 12 course convenings, each approximately 30 weeks in length. • 51 simulator events and 78 training sorties. • Student throughput will be 45 REs (replacement equivalents). • No RP pooling anticipated. • FRS Grads approaching combat capable and/or 60% CRP. Apogee Solutions The Performance Improvement Company

13. AV-8B FRS AIRCREW TRAINING STUDY • Input Evaluation Results • RPs are fully qualified Marine aviators. • “Cream of the crop.” • RPs have an excellent understanding of the training environment. • Pace of training greater at the FRS than at previous training commands. • Curriculum not designed to winnow RPs. • Current RPs tend to be more technically proficient. Apogee Solutions The Performance Improvement Company

14. AV-8B FRS AIRCREW TRAINING STUDY • Process/Product Evaluation Results • Classroom Instruction. • IP technical knowledge is excellent. • Lectures content is presented in a logical manner. • IPs use questions to check for understanding and provide opportunities for discussion. • Quality of visuals is adequate. • Some materials are unclear or outdated. • Materials are being updated. Apogee Solutions The Performance Improvement Company

15. AV-8B FRS AIRCREW TRAINING STUDY • Process/Product Evaluation Results • The Learning Center. • Self-paced lecture materials do a fair job of showing component location and function. • Self-paced lecture materials do a poor job of teaching system operation. • Self-paced lecture materials do help relieve IP instructional burden. Apogee Solutions The Performance Improvement Company

16. AV-8B FRS AIRCREW TRAINING STUDY • Process/Product Evaluation Results • The Simulators. • Pre-event briefings are a good advance organizer. • Simulator events do a fair job of supporting basic aircrew skills development. • Simulator events do a fair job of supporting the development of procedural & tactical skills taught in FRS instructional phases. • OFT & WST simulator upgrades are planned in the near term. • Post-event debriefs are a useful instructional tool. Apogee Solutions The Performance Improvement Company

17. AV-8B FRS AIRCREW TRAINING STUDY • Process/Product Evaluation Results • Training Sorties. • Pre-event briefings are a good advance organizer. • IPs use a variety of coaching styles or approaches during sorties. • Reviewing HUD Videotapes during debriefs are a useful instructional tool. Apogee Solutions The Performance Improvement Company

18. AV-8B FRS AIRCREW TRAINING STUDY • Process Evaluation Results • Training Transfer. • Most RPs and IPs were neutral when queried on whether the current FSG adequately prepares them to be combat capable pilots. • CPs did not believe that the current FSG adequately prepared them to be combat capable pilots. Apogee Solutions The Performance Improvement Company

19. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions • RPs are well prepared and highly motivated. • IPs are knowledgeable and highly motivated. • VMAT-203 ISD has completed a thorough task analysis and has an excellent understanding of the training requirements. • The newly implemented self-paced lectures have the potential to reduce IP teaching load. • There is a planned upgrade scheduled for the OFT & WST simulators. Apogee Solutions The Performance Improvement Company

20. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions • Components of a robust mental model. • Conceptual knowledge about the system. • Procedural knowledge about system operation. • Decision making skills related to monitoring system condition. • Strategic knowledge about how/why the system operates in a variety of contexts. Apogee Solutions The Performance Improvement Company

21. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions • Classroom/Learning Center Materials • Conceptual knowledge about the system. • Some procedural knowledge about system operation. • Little decision making skills related to monitoring system condition. • Little strategic knowledge about how/why the system operates in a variety of contexts. Apogee Solutions The Performance Improvement Company

22. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions • Simulator events support • Conceptual knowledge about the system. • Procedural knowledge about system operation. • Decision making skills related to monitoring system condition. • Strategic knowledge about how/why the system operates in a variety of contexts. • However, the simulator is a limited resource. Apogee Solutions The Performance Improvement Company

23. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions • Current training training technologies are too few and not appropriately designed to support learning, particularly early on in the training cycle. • Lack of appropriate scaffolding leads to instructional situations where RPs have not mastered prerequisite skills. • The instructional event should remain within the RPs’ zone of proximal development. • Each event should challenge the RP to expend effort. • The appropriate level of effort should result in the RP reaching the instructional goal. Apogee Solutions The Performance Improvement Company

24. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions – Self-Paced Instruction. • Declarative knowledge (facts, concepts & principles) needs to be proceduralized and automatized in order to facilitate mastery of prerequisite skills, thus freeing up cognitive resources for more complex learning activities. • High fidelity training devices may actually inhibit learning among novices. • Learning and instructional theory appear to support this notion. Apogee Solutions The Performance Improvement Company

25. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation. • Develop 16 self-paced CBT lessons covering all AV-8B systems. • Level III guided interaction. • Each lesson would include self-paced instruction on component location, function, system operation and any user-initiated system tests. • Each lesson would also feature a test bank. Results would be forwarded to the relevant IP. Apogee Solutions The Performance Improvement Company

26. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation-Develop 16 self-paced CBT lessons covering all AV-8B systems. • We estimate the development cost for each CBT lesson at $40,000 per one hour lesson. • Assuming 16 lessons, the total development cost would be $640,000. • Each lesson would replace at least one hour of simulator time. • 16.5 hours reduction in simulator time per RP per class. • 45 RPs x 16.5 hours = 752.5 hours per year. • 742.5 hrs per year x $247.22 per hr = $183,560 ROI yr. Apogee Solutions The Performance Improvement Company

27. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation-Develop 16 self-paced CBT lessons covering all AV-8B systems. • RP can repeat CBT as often as necessary. • CBT can be made available 24/7. • Lesson/module templates, interaction models and visuals can be leveraged for use in maintenance training. Apogee Solutions The Performance Improvement Company

28. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions – Part Task Training. • Strengthen AV-8B systems procedural knowledge through both guided and unguided practice in order to facilitate mastery of prerequisite skills, particularly those related to decision making related to system monitoring. • Unguided practice would be used to increase RP understanding of system “switchology.” • Guided practice would be scenario-based. Apogee Solutions The Performance Improvement Company

29. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions – Simulator Events. • Strengthen AV-8B aircrew procedural and tactical knowledge through guided practice in order to facilitate greater strategic knowledge of system operation in archetypical contexts. • Lower fidelity training devices may actually improve learning among novices. • The CNET MISSLE (Micro-Simulator Systems for Immersive Learning Environments) Project is an example of an inexpensive, non-immersive, single participant flight training simulator that features low physical fidelity while retaining some psychological fidelity. Apogee Solutions The Performance Improvement Company

30. AV-8B FRS AIRCREW TRAINING STUDY • Conclusions – Simulator Events. • Psychological fidelity is often referred to as Presence when discussing Virtual Learning Environments (VLEs). • Presence is defined as the degree to which the participant actually feels he is in the place represented by the VLE. • Presence (psychological fidelity) can be significantly increased by developing VLEs that are fully immersive and multi-participant. • Using increasingly complex VLEs to to strengthen RP procedural and tactical knowledge through multiple guided practice opportunities should result in more robust mental models earlier in the training cycle. Apogee Solutions The Performance Improvement Company

31. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation. • Implement Airbook in order to enhance procedural and tactical training. • Install nine units (one IP station and eight RP stations) in an electronic classroom. • RP stations would be used as a single learner part task trainer or a multi-participant, multi-threat tactical trainer. • IPs can demonstrate procedures and tactics. • RPs can fly missions singly, in sections or division while being monitored by the IP or a virtual instructor. • All missions can be saved and replayed. Apogee Solutions The Performance Improvement Company

32. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation. Implement Airbook in order to enhance procedural and tactical training. • Install four laptop units in the squadron briefing rooms for mission planning and remediation purposes. • RP can rehearse mission prior to the actual training sortie. • IPs can demonstrate procedures and tactics. • Debrief could include an opportunity for the RP to virtually re-fly the sortie if deemed necessary by the IP. Apogee Solutions The Performance Improvement Company

33. AV-8B FRS AIRCREW TRAINING STUDY • Recommendation. • VTSG estimates the Airbook development and implementations costs as described to be $805,000. • 69 hours reduction in simulator time per RP per class. • 45 RPs x 69 hours = 3145.5 hours per year. • 3145.5 hrs per year x $200 per hr = $629,100 ROI per yr. Apogee Solutions The Performance Improvement Company

34. AV-8B FRS AIRCREW TRAINING STUDY • Conclusion. • Additional ROI could be generated by: • Making some training events multi-participant. • Monitoring some of the training events with the virtual instructor. • Allowing RPs multiple opportunities for mission rehearsal. • Allowing RPs multiple opportunities for mission remediation. Apogee Solutions The Performance Improvement Company

35. AV-8B FRS AIRCREW TRAINING STUDY QUESTIONS? Apogee Solutions The Performance Improvement Company