1 / 82

Course Schedule (con’t) two Principles of Testing Principles of Assessment

Course Schedule (con’t) two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances. Course Schedule two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances. Principles of Testing. Outline General Principles

mhair
Download Presentation

Course Schedule (con’t) two Principles of Testing Principles of Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Course Schedule (con’t) two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances

  2. Course Schedule two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances

  3. Principles of Testing Outline General Principles Common Rating Errors Self-study handout

  4. Principles of Testing Outline General Principles Common Rating Errors Self-study handout

  5. Principles of Testing General Principles A good test should be: 1) Valid 2) Reliable 3) Comprehensive 4) Discriminating 5) Objective

  6. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Validity

  7. Validity Does test measure what it is supposed to measure? 1) make the test realistic 2) make the test complete 3) apply standards carefully 4) no test of items outside std

  8. Validity Example Which is the most valid? 1) describe action in briefing 2) simulate action in aircraft 3) do action in aircraft 4) do action in simulator

  9. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Reliability

  10. Reliability How accurately does it measure what it measures? Must be consistent: 1) Avoid extremes - distractions - aircraft - weather 2) Plan compensations (eg. V1) - beforehand - carefully

  11. Reliability Example Which test is most reliable? 1) PPC in marginal VFR weather 2) PPC in good weather, light winds 3) PPC in good weather, strong winds 4) PPC in IFR weather near minima

  12. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Comprehensive

  13. Comprehensiveness The test must include all the relevant sequences - important "need to know" info - do not focus on details - vary the program from test to test - change approaches, sequences, emergencies (where discretion)

  14. Comprehensiveness Example Which test is most comprehensive? 1) departure, steep turns, hold, NDB, ILS 2) dep, hold, ILS, ILS, VOR 3) dep, hold, ILS, VOR 4) dep, steep turns, LOC (BC), PAR

  15. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Discriminating

  16. Discrimination Detects different levels of achievement - separate qualified from unqualified - this is main goal of test - rank those who are qualified - use overall performance for borderline cases

  17. Discrimination Example Which test is most discriminating? 1) no oral test before flight 2) oral test before flight with general briefing 3) oral test and specific test sequences 4) oral test before and during flight

  18. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Objective

  19. Objectivity Test outcome must not be affected by personal examiner 1) do not impose your own standards 2) adhere to standards in FM, SOP, AOM and DCP Manual 3) do not let rating errors affect your assessment 4) carefully record events as they occur

  20. Objectivity Example Which written assessment is most objective? 1) poor judgment during emergency 2) excessive swing during engine fail 3) altitude 100ft low inbd to beacon 4) airspeed unstable on final

  21. Principles of Testing Outline General Principles Common Rating Errors Self-study handout

  22. Common Rating Errors 1) Error of Standards 2) Error of Central Tendency 3) Error of Halo 4) Error of Logic 5) Error of Familiarity 6) Error of Narrow Criterion

  23. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of central tendancy

  24. Error of Central Tendency Rate of performance at centre of scale Hesitant to give excellent or poor ratings Affects all assessments by this observer Most common with inexperienced observers

  25. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of Standards

  26. Error of Standards Use personal standards to rate others If good at it - rate harder If poor at it - rate easier Affects all assessments by this observer

  27. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of Halo

  28. Error of halo Allow general impression to affect rating Can be favourable or unfavourable Affects only certain persons rated Can include stereotypes

  29. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of Logic

  30. Error of Logic When several similar traits Traits are related in mind of observer Could relate engine fail at V1 to SE landing Observer generally not aware of error

  31. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of Familiarity

  32. Error of Familiarity • Grading objectivity weakened • Accustomed to weaknesses and eccentricities • Occurs over long period of time • Stop and regain perspective

  33. Principles of Testing Outline General Principles Common Rating Errors Self-study handout Error of Narrow Criterion

  34. Error of Narrow Criterion • Related to assigning rating • Often occurs in assessing two-crew or multi-crew assessments • One who does the best of perhaps a marginal performance obtains a high rating

  35. Principles of Testing Outline General Principles Common Rating Errors Self-study handout

  36. Course Schedule day 2 Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances

  37. Principles of Assessment Outline PPC general guidelines Assessment format Overall assessment

  38. Principles of Assessment Outline PPC general guidelines Assessment format Overall assessment

  39. PPC general guidelines 1) Take note soon after event 2) Terminate test if failure 3) Allowance for turbulence 4) No advice, criticisms, training or demonstrations 5) Create realistic scenarios 6) Do not simulate multiple, unrelated failures

  40. PPC general guidelines (con’t) 7) PPC - Use Designated Check Pilot Manual 8) Simulated weather at or below minimums 9) Need a "U" to fail 10) Take weather info into account in scenarios 11) PNF responsible if gross deviations, violations

  41. Performance Assessment According to the following publications: • DCP Manual PPC - AFM - AOM, Pilot Operating Handbook - Operations Manual - SOP Manual IFT - Instrument Procedures Manual - Approach Procedures (Jepp) - Company Procedures

  42. Principles of Assessment Outline PPC general guidelines Assessment format Overall assessment

  43. PPC Assessment Format Sexercise flown within acceptable tolerances and procedures correct and effective SB a brief excursion from the criteria or a deviation from standard practices. Corrected by acknowledging the deviation during de-briefing; or regaining the criteria during flight U gross excursions from acceptable tolerances or a violation of airspace or clearance or a fault in procedure that could create an unsafe or ineffective situation

  44. PPC Assessment Format S - Minor errors only - Airspeed and altitude control acceptable - Handling and knowledge are acceptable and safe considering the experience of the candidate

  45. PPC Assessment Format U - Endanger aircraft, passenger, crew; - It results in a crash; - Multiple errors made in one exercise; - Violates ATC clearance or an altitude; - Major deviation from standard procedures or practices or safety was jeopardized; - Candidate required continued prompting or help from other candidate; - It exceeds aircraft limitations; or - Candidates demonstrates unsatisfactory knowledge of systems, equipment, or procedures

  46. PPC Assessment Format SB - Aircraft handling and knowledge are safe but of a lower standard that can be corrected by briefing; - Brief excursion from tolerances but corrected promptly; - Deviated from SOP but did not create a more hazardous situation and is clarified by candidate - Candidate experienced some difficulty or required slight prompting to accomplish the task

  47. Principles of Assessment Outline PPC general guidelines Assessment format Overall assessment

  48. Overall Performance • is it safe and effective? • are criteria broken? • are any fail points done? • is crew concept used? • are crew in control of situation? • are procedues known and executed? • are limitations adhered to? • does an error create an unsafe situation? • are errors recognized and corrected? • would you send your family with this pilot?

  49. Course Schedule day 2 Principles of Testing Principles of Assessment Conducting a PPC PPC/IFT Tolerances

  50. Conducting a PPC Outline Flight test requirements - PPC Pre-flight preparation/briefing Flight test scenarios Safe checking practices Post-flight briefing Writtencomments

More Related