ABA Roundtable May 2014

1. ABA RoundtableMay 2014

2. IN THE BEGINNING, . . . . • There was nothing.

3. OVERALL EXAM STRATEGY • A MOVE TO CONSCIOUSNESS

4. SOME EXAMPLES • Format • Specifications • Purpose • Drafting guidelines

5. STRATEGY FOR EXAM DATABASE • Make maximum use of available exam information each year

6. TACTICAL EXAMPLES • We can now generate an exam in final format from the database • This was humongous for us – we had previously used Word documents in preparing exam content for 21 specialty areas • It reduced the work for our volunteers

7. MORE TACTICAL EXAMPLES • Questions can be cataloged by subject and a unique ID number • Questions can be tied to a specific provision in the exam specifications • We can track who drafted, graded or reviewed • We have authority for the correct answer • We can determine if the question needs to be revised • We can revise a model answer if needed

8. CONTINUED • We can track performance on multiple choice and essay questions • We can determine if a question is invalid and adjust scores accordingly in the database • We generate a detailed report on multiple choice performance • We can easily use different combinations of questions to keep an exam from being too predictable

9. CONTINUED • We can better quantify the coverage of the exam so that topics were covered in the proper proportion to their importance in the specialty area • We can determine how examinees performed by topic • We can much more easily track changes to questions and retire items which are no longer valid