100 likes | 222 Views
Designing Good Tests: Item Analysis is Part of the Equation!. Molly Herman Baker, Ph.D. Black Hawk College. Planning Tests. Determine amt of time, location, level of difficulty desired, resources available during test, and % weight of test compared to total # of assessments
E N D
Designing Good Tests: Item Analysis is Part of the Equation! Molly Herman Baker, Ph.D. Black Hawk College
Planning Tests • Determine amt of time, location, level of difficulty desired, resources available during test, and % weight of test compared to total # of assessments • Based on objectives, determine what is important to know, not “nice to know” • Based on amt of time, estimate number of questions possible (see Test Writing Basics handout for estimates) • Prepare specifications chart
Specification Chart Basics • Specifications chart: • Type and number of items for each content area or objective. • Type and number of items for each type of HOT skill (HOT=discipline-specific; Bloom often selected, but others may be more relevant for your discipline)
Writing/Choosing Good Items • Write good objectives, using verbs that indicate desired performance; share with students • Write/select test items based on these verbs; • See “Comparison of Bloom and Ebel’s Taxonomies” handout for samples • See “Test Writing Basics” handout • Discuss strategies for selecting test bank items
What about HOTS? • HOTS (Higher order thinking skills)=application, analysis, prediction, synthesis, evaluation or alternative taxonomy relevant to your discipline • Important: practice HOT in class (critical thinking, creative thinking, problem-based learning, case-based learning, scenario-based lng) • Important: provide guidance, practice & feedback on preparing for and answering HOT exam questions
Item Analysis to Improve Tests(Scantron QuickScore II: QC1-106) • # correct and # missed per item • What does this tell you? • # who answered each MC distractor • What does this tell you? • Correlation between answers and overall test performance (Discrimination Index/Pt. Biserial) • What could this tell you? • Questions with poor item analysis score • Discuss with students to determine item problems
Testing Services at BHC • Administer individual tests to students in Study Unlimited or online classes • Administer individual tests to students taking make-up exams • Score and return Scantron tests if the key is provided by the instructor (campus mail or file drawer). More Scantron scoring machines in Bldg 2 and 3. • Email ilctesting@bhc.edu or submit ILC Testing Form to make arrangements for online testing (course, instructor, exam#, student(s), deadline for completing, time limit, allowable notes or books, test password, what to do with hard copies)
QuickScore II Tips • Item analysis software in QC1-106 (ILC) • Use score sheet #101864-ERI-L only • Options for output: • Save electronic analysis reports in PDF format to your H: drive ( or floppy, Flash drive) • Print reports (bring paper from your dept) • Laminated instructions in 1-106 binder (or download from http://facweb.bhc.edu/tlc/tech/stepbystep/quickscore.html )