Human Factors Survey of Aviation Maintenance Technical Manuals

1. Human Factors Survey of Aviation Maintenance Technical Manuals Alex Chaparro PhD, Leonidas Deligiannidis PhD, Chris Hamblin, Bonnie Rogers National Institute for Aviation Research

2. Increased Attention Given to Maintenance by Regulatory Agencies • Precipitated by • Seminal Incidents: Aircraft accidents • Increases in aircraft traffic • Percentage of accidents attributable to maintenance error appears to have change little over that last 50 years.

3. Maintenance Error and Aircraft Safety • Aloha flight 243 (1988) Structural failure of the upper cabin • Two experienced inspectors failed to detect cracks in the aircraft skin. • BAC1-11 (1990) Left windscreen blew out • 84 of 90 securing bolts were smaller than the specified diameter http://www.volpe.dot.gov/infosrc/journal/30th/images/safe_aloha.jpg

4. Fatal Accidents of Large Commercial Jets (1959-2001) www.flightsafety.org/ priorities.html

5. Predicted Increases in the number of commercial flights and aircraft accidents**pre 9-11.

6. Maintenance Error • Analysis of maintenance errors indicates that the majority of incidents involve omitted or incorrect execution of tasks, incorrect installations, and the use of incorrect parts (Boeing, 1995). • Johnson and Watson (2001) identified information as being the highest ranked contributing cause, being implicated in approximately 38% of all maintenance errors. • An analysis of NASA Aviation Safety Reporting System data regarding maintenance incidents found document procedures to be related to 60% of incident reports from 1986 to 1992 and 45% of incidents from 1996 to 1997.

7. Review of User-reported problems with maintenance documentation • Publication Change Request (PCR’s) represent errors that the current maintenance manual development and review process have failed to detect. • Two companies provided copies of PCR’s to the maintenance manual submitted by users. • Reviewed a total of 300 PCR’s • Categorized requested changes into four groups: technical, procedural, graphic, and language

8. Types of errors for each category • Technical • Incorrect parts, values or tools are specified • Procedural Poor sequencing, missing/unnecessary procedures, failure to specify how to check/test equipment • Language • Typos, grammar, clarity • Graphics • In correct dimensions, incorrect graphic, poor quality graphic

9. Analysis of the PCR’s • 39% of PCR’s report errors in procedures • 31% of the PCR’s report errors in technical information • 24% of the PCR’s report problems with language

10. Examples of PCRs • Aircraft Manufacturer #1: • Torque values were called out in percent rather than foot pounds • Changed to foot pounds • The figure indicated the Left and Right oxygen bottles connecting incorrectly into a tee fitting • Replaced with an accurate illustration of the components • Also included a drawing of the oxygen system layout

11. Cont’d: Examples of PCRs • Aircraft Manufacturer #1: • After maintenance operation, electrical connection to the propeller synchronizer and automatic feathering solenoid had been switched • Added note on possibility of misconnecting components • Added check/test for source of system malfunction • The check valve can be installed backwards • Added check arrow on component in a graphic • Added a note after step suggesting that the technician confirmed the direction in which the arrow should point

12. Norman’s Action Cycle Mechanic’s Goal(s) Execution Evaluation Gulfs of Execution Gulfs of Evaluation Intention to act Evaluation of interpretations Sequence of action(s) Interpreting the perception Execution of the action sequence Perceiving the state of the world The World (Norman, 1988)

13. Norman’s Action Cycle Mechanic’s Goal(s) Execution Evaluation Gulfs of Execution Gulfs of Evaluation Intention to act Evaluation of interpretations KIH ≠ KIW KIW ≠ KIH LTM STM Chunking Graphics Affordances Mapping Constraints Feedback Check/test Inspections Sequence of action(s) Interpreting the perception Execution of the action sequence Perceiving the state of the world The World (Norman, 1988)

14. Bridging the Gulfs • Facilitating task execution by using • Mapping in the form of color coding • Physical constraints • organizing tasks into smaller subtasks reducing working memory demands • Identifying task critical information that the user needs • Including the user in the document development process

15. Bridging the Gulfs • Facilitating task evaluation by using • Mapping in the form of color coding • Physical constraints • Feedback in the form of check/tests • Improving visibility to allow system monitoring by the maintenance personnel

16. User-Centered Design • Analysis of the PCR’s suggests that many of the problems stem from a mismatch between mental model of the writer and the mechanic regarding how maintenance is performed and what is maintenance relevant information. • Technical writers may not have experience performing aircraft maintenance • User (mechanic) is rarely consulted in the development of the maintenance procedures • Maintenance procedures are typically not validated

17. Current practices • Validation of the maintenance procedures is often limited because • access to aircraft is limited • Aircraft is not in the configuration that the customer will receive • assumed cost of validation • Lack of familiarity with evaluation techniques • Time constraints • Low priority given to maintenance issues by manufacturers

18. Using Virtual Reality Technology to Validate Maintenance Procedures • Allows evaluation of maintenance procedures to be performed earlier in aircraft development • Can use latest engineering drawings for an aircraft or its components • Physical access to the aircraft is not required • Maintenance procedures can be evaluate at any time • Assembling a component incorrectly does not impact a safety • Many aircraft manufacturers operate VR labs

19. Limitations of Virtual Reality • Limited or distorted cues/feedback • Ability to simulate tactile cues is limited • Ability to use both hands to interact with a simulate object is not supported

20. Application of VR • Problems that are being addressed at WSU • Multi-view environments for observation and for training • Interaction techniques for single or two-handed environments • Navigation techniques

21. Multi-View Environments • Intended for the evaluation of maintenance procedures (observer views subject performing the maintenance task) • Adds educational value

22. Navigation issues within VR • Need navigation because of: • limited tracker range • physical room dimensions • View larger components from a distance • Need to move in 3 dimensions to view or access components positioned out of reach.

23. Gestures For Navigation • Gestures needed to move around and to interact with the virtual environment • Navigation required two flavors • FLY user can point at the direction of travel • DRIVE similar to FLY but user does not move vertically

24. Interaction Techniques • Need to grab/select and manipulate components to assemble/de-assemble a larger part • Need to be able to work with two hands • May need to manipulate remote objects (fly back and remove unwanted components)

25. Interaction Techniques (Cont.)

26. Implementation • JWSU a java based VR toolkit that supports: • multiple view • different interaction techniques • different navigation techniques • easy to add on new interaction/navigation techniques • utilizes the Xj3D VRML loader to load complex models (www.xj3d.org)

27. Remaining technical Issues • Need for better haptic simulation • Need for better bimanual interaction with VR models • Need to evaluate the effectiveness of VR for validating aircraft maintenance procedures.

28. Summary • The development of aircraft maintenance documentation has proceeded without consideration of the needs of the user population. • Further improvements in aircraft safety may be realized by the application of cognitive principles as part of a “User Centered” approach to aircraft maintenance. • Virtual Reality holds promise as a means to validate maintenance procedures earlier in the development of an aircraft.