1 / 37

Classroom-based learning diagnostic model for teacher-made tests: Program and applications

萬用卡. Classroom-based learning diagnostic model for teacher-made tests: Program and applications. 1 Tsai-Wei Huang, 2 Pei-Chen Wu 1 National Chiayi University, Taiwan, 2 National Ping Tung University of Education, Taiwan 2013/01/15. Why Teacher-made test is necessary? .

mikel
Download Presentation

Classroom-based learning diagnostic model for teacher-made tests: Program and applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 萬用卡 Classroom-based learning diagnostic model for teacher-made tests: Program and applications 1Tsai-Wei Huang, 2Pei-Chen Wu 1National Chiayi University, Taiwan, 2National Ping Tung University of Education, Taiwan 2013/01/15

  2. Why Teacher-made test is necessary? • Confirmation of instructional objectives • Formative assessment for learning outcomes • Integration between instruction and learning

  3. What students need from tests? • Feedbacks of progressions • Feedbacks of concept mastered or misconceptions revealed • Feedbacks of their cognition styles detected

  4. What teachers need from tests? • Feedbacks of item/test quality • Feedbacks from students’ outcomes • Promotion of Item construction skills • Promotion of teaching effectiveness • Diagnostic information on students’ learning

  5. Aberrance index Detecting students’ (or items’) aberrant response patterns based on a theoretical model, e.g., Guttman ideal response model Person-fit index Item-fit index

  6. What aberrance index can do? • Able to reflect each student’s (item’s) aberrant responses • provide rich diagnostic information about misconceptions and psychological response patterns during learning processes.

  7. Illustration of PERSON/ITEM Aberrant Response Patterns (N=10 persons, K=15 Items).

  8. RATIONALE--BW family indices • Person-facet • W (carelessness) • B (guessing) • C (capability) • M (misconception)

  9. RATIONALE--BW family indices • Item-facet • W (hint ) • B (disturbance ) • C (difficulty ) • M (misfit )

  10. BW learning diagnostic Model

  11. Wbstar program

  12. Design ideas

  13. Functions--Person aspect Able to estimate the probabilities of • items correctly answered • concepts mastered • teaching objectives reached Able to classify students to adequate classes of psychological cognition status for diagnosis under a certain criterion (e.g., alpha = .05)

  14. Functions--Item aspect • Able to provide item information about • Difficulty • Discrimination • Option distraction • Hint • Disturbance • Misfit • Able to examine item’s functions by classification under a certain criterion (e.g., alpha = .05)

  15. Program’s looks

  16. Program’s looks

  17. Program’s looks

  18. S-P chart

  19. Example 1: Fraction and Decimal • Data • 32 students in the 4th grade elementary mathematics class in Taiwan. • Teacher-made test (22 items) with five concepts of fraction and decimal: • transforming decimal into fraction (TDF) • equivalent fraction (EF) • comparing fraction with decimal (CFD) • transforming fraction into decimal (TFD) • unit transformation (UT) • Q matrix of items on concepts judged by 2 teachers

  20. Classification design

  21. Item analysis--high w value (11.36) Irrelevant Hint: 160 4/5 >1.605! How about 160 2/5?

  22. Item analysis--high b value (4.83) ID 8 student gave an correct answer of 68 for q2. Sequentially, he gave q3 an answer of 68/1000.

  23. Concepts mastery

  24. Aberrance patterns (H1,H3)

  25. Aberrance patterns (A1, A1’, A2, A4 )

  26. Aberrance patterns (L1, L2, L3, L4 )

  27. Example 2:Two-tier acid-base test • Examining the effects of classifying response patterns by BWCM indexes 67 sixth grade students (41 boys, 61.19%; 26 girls, 38.81%) in Taiwan.

  28. Four cognitive styles of the two tier items

  29. Questions • Q1 : How do the classified clusters based on the BW indices be verified? • Q2 : What characteristics of the classified clusters will reveal? • Q3 : What characteristics will change for individuals between the know-what test and know-how test?

  30. RESULTS Classification

  31. Properties of 2 clusters Group of low misconception with aberrant responses Group of high competence with normal responses

  32. 3 clusters Low competence Group with guessing aberrant responses Middle competence Group with complex aberrant responses High competence Group with normal responses

  33. Change between know-what (ANSWER) and know-how (REASON) H1H1 H1A3 L1’H1

  34. Recommendations • The BW model can provide sufficient and matching information of interpretations to an original response data matrix in both person-and item facets, especially in a small sample size situation like that of students takes a teacher-made test in class. • Teachers can use the useful information to improve their test development skills and to have immediate diagnostic feedbacks from their students’ response patterns. • The BW model can provide the extent to which students mastered in concepts and achieved to objections so that teachers can go further for designing remedial instructions.

  35. Future Outlook • The BW model in this study is now only suitable for the dichotomously scoring situations; • Polytomously scoring system is under developing...

  36. Related articles and resources • Huang, T. W. (2008). A study of cutoffs for aberrant indices under different data structures. The Journal of Guidance and Counseling, 30, 1-16. (in Chinese) • Huang, T. W. (2011). Establishing and examining the diagnostic space of two new developed person-fit indices: The W* and the B* indices. Psychological Testing, 58(1), 1-27. (in Chinese) • Huang, T. W. (2011). Robustness of BW aberrance indices against test length. Knowledge Management & E-Learning: An International Journal, 3, 310-318. • Huang, T. W. (2012). Aberrance Detection Powers of the BW and Person-Fit Indices. Journal of Educational Technology & Society, 15(1), 28-37. • Huang, T. W. & Wu, P. C. (in press). A classroom-based cognitive diagnostic model for teacher-made tests: An example with fractions and decimals. Journal of Educational Technology & Society. • Huang, T. W. (2007, July). Establishing Cutoffs for Two New Aberrance Indices: The Within- Ability-Concern Index and the Beyond-Ability-Surprise Index. Paper presented at the annual meeting of the International Conference on the Teaching of Psychology, Vancouver, Canada. • Huang, T. W. (2012, July).Verifying the effects of item-fit indices on gender-related differential item functioning. Paper presented at the XXX International Congress of Psychology, Cape town, South Africa. • Huang, T. W., & Tsai, S. W. (2012, August). Examining the effect of classifying response patterns on an acid-base test by a Guttman-based person-fit index set. Paper presented at the 1st International Conference on Education Measurement and Evaluation 2012 (ICEME2012), Manila, Philippines. • The WBStar program now is available in CHINESE version at http://140.130.46.9. English version is not completely done, but coming soon…

  37. THANKS FOR YOUR ATTENTIONS!

More Related