Constructing Learner-Centered Assessment: Designs, Systems, and Outcomes

1. Constructing Learner-Centered Assessment: Designs, Systems, and Outcomes Steven J. Ross SPS-KGU Kobe-Sanda, Japan

2. Preliminaries • Proficiency: absolute (criterion-referenced) or relative (norm-referenced) abilities in language domain irrespective of context. • Achievement: absolute (criterion-referenced) or dynamic (growth-referenced) learning based on a syllabus • Self-and-Peer Assessments here refer to school-based achievement assessment.

3. Rationale for SBA • Not all learners are extrinsically motivated • Instrumental motivation is the norm • Motivational void is often large in foreign/second language learning • Locus of control is mostly external • Learner empowerment for self-and-peer assessment may shift motivation

4. Passivity • Most language learners are socialized into taking a passive role • Assessment is done to them, rather than what they can do for themselves. • The goal is to gradually shift from passive to active participation in assessment

5. Achievement Components • Teachers’ on-going record of individual student performance on achievement tasks. • Learner self-assessment of absolute mastery and dynamic change • Peer-assessment of cooperative learning contribution and performances on achievement tasks

6. Adjusting Self-Assessment • Achievement is more validly assessed than proficiency • Achievement is experiential with episodic memory for reference • Proficiency events may be too abstract and not directly experienced • Over-estimation (low) and under-estimation (high) prevalent in neophytes

7. Factors in Peer-Assessment • Benchmarking is advisable in early dry run simulations • Quality standards need to be established early on • PA can be misinterpreted as popularity • Learners often need assessment training

8. Triangulation • Teacher, Self, and Peer Assessments should correlate with each other • Early and frequent data collection can facilitate calibration of the assessment system • Learners need to be aware of importance of assessment validity • Self and Peer assessment has to count towards record of achievement.

9. The Research Fair as Goal • Semester or Annual Event • Language curriculum can be designed around the RF: e.g., seminars and presentation courses. • One year (presentation course) or two year (seminar) preparation time frame • Groups nominate research theme • Cooperative learning (research, analysis, rehearsal, in-class and RF presentation)

10. Assessment Content • Cooperative learning projects are ideal • In and out of class group preparation for presentations • Group presentation can be globally and locally peer- assessed • Individual presentations can be globally and locally peer-assessed

11. Assessment Contexts

12. Research Fair Project Assessment

13. In-Class Project Assessment

14. Individual Global Peer-Assessment

15. Global Peer-Assessment Criteria • Clarity 1 2 3 4 5 6 7 • Voice 1 2 3 4 5 6 7 • Eye Contact 1 2 3 4 5 6 7 • Organization 1 2 3 4 5 6 7 • Balance of Roles 1 2 3 4 5 6 7 • Originality 1 2 3 4 5 6 7

16. In-Group Peer Assessment Criteria • Cooperation 1 2 3 4 5 6 7 • Contribution 1 2 3 4 5 6 7 • Initiative 1 2 3 4 5 6 7 • Effort 1 2 3 4 5 6 7 • Attitude 1 2 3 4 5 6 7 • Expertise 1 2 3 4 5 6 7

17. Reflective Self-Assessment • Contribution 0 1 2 3 4 5 6 7 • Effort 0 1 2 3 4 5 6 7 • Revision 0 1 2 3 4 5 6 7 • Growth 0 1 2 3 4 5 6 7 • Cooperation 0 1 2 3 4 5 6 7 • Motivation 0 1 2 3 4 5 6 7

18. Potential Long-Term Benefits • Most learners join the work force after graduation • Presentation skills are developed • Group cooperation skills are developed • Technical skills (computers, Power Point, etc) are developed • Confidence may be enhanced • Language proficiency increases

19. Organisational Troubleshooting • Organizational workload issues • Delegation of task management • Assessment monitors in Research Fair • Using technology for frequent peer-assessment • Archiving continuous assessment

20. Toward Formative Assessment • Self-and-Peer Assessments can be either summative or formative: usually summative before formative. • Early-mid-final assessments can capture dynamic changes. • Achievement assessment can credit both absolute performance and growth over time.

21. A Systems Analysis Approach to Assessment Design • Engineered Cooperative Learning Events • Self-and-Peer Assessment training • Inclusion of dynamic improvement criteria • Efficient data collection • Efficient data storage and retrieval • Efficient data analysis • Triangulation and validation

22. Empirical Research Evidence • Longitudinal Sequential Cohort Design • S1 S2 S3 S4 F1 F2 F3 F4 • Each cohort contains about 250 students • S1 to S4 employed mostly summative assessments • F1 to F4 employed increasingly more formative assessments

23. Assessment Practice Shift

24. Preliminary Outcomes: Listening Proficiency (TOEFL)

25. Preliminary Outcomes Reading Proficiency (TOEFL)

26. Research Agenda • Longitudinal comparative impact analyses of formative vs summative assessment • Peer-assessment training impacts • Analysis of self-assessment residuals (differences between predicted and observed self-assessments) • Motivational change over time as a consequence of formative assessment