The challenges of assessment in a new learning culture

1. The challenges of assessment in a new learning culture Olga Dysthe University of Bergen Norway

2. A new learning culture • based on insights of constructivist and sociocultural perspectives of learning • deals with new demands on education due to changes in society

3. OVERVIEW: 5 CHALLENGES I.The ’wash-back effect of assessment” II. A changing world – new expectations to education III. Aligning theories of learning and assessment IV. Practicing new modes of assessment (institutions, teachers and students) V. Countering the strong international trend of measurement and accountability

4. What lessons can we learn? • 1. Learning and assessment cultures can change • 2. Assessment as engine in the change process • 3. Researchers + teachers = strong force • 4. The need for theoretical underpinning of assessment practice: major conceptual shift • 5. Quality assurance of new assessment forms • ”what you assess is what you get” WHY CHANGE? To prepare students for future demands of broader writing competence

5. II. The challenge from changes in society – new expectations • COMPETENCES MOST NEEDED IN FUTURE • Example 1: • OECD program for higher education ”Tuning education strategies in Europe”: • Large scale investigation in 16 countries • university graduates • employers • academics

6. OECD-countries’ competency list: 1. Capacity for analysis and synthesis 1. Capacity to learn 1. Problem solving 2. Capacity for applying knowledge in practice 3. Capacity to adapt to new situations 3. Concern for quality 4. Information management skills 4. Ability to work autonomously 5. Teamwork 6. Capacity for organisation and planning “Tuning education strategies in Europe” 2003

7. Key question: How to foster and assess broad competences? • Australia 2003: National, standardized ”Graduate Skills Assessment Test” • US: plans for large scale ”objective” tests Shavelson: ”We need to assess the whole range of what we value, not just the cognitive outcome of education”

8. A national poll of education goals in the United States • Sense of maturity and ability to manage on one’s own (71%) • Ability to get along with people (68%) • Problem solving and thinking abilities (63 %) • High-technology skills (61%) • Specific expertise and knowledge (60%) • Writing and speaking ability (57%) • Responsibility of citizenship (44%) • Immervahl 2000

9. My conclusion New modes of assessment are needed to foster and to assess the broad range of educational goals for the 21st century

10. Distinguishing factors of new modes of assessment • Contextualized • Broader aspects of learning – multiple sources • Purpose: primarily learning, not selection • Integration of assessment in learning process • Shift from low levels of competence to higher • Social and affective aspects included • The teacher plays an important role in assessment • More student involvement in assessment processes – not just as objects • peer and self assessment

11. III. The challenge of aligning theories of learning and assessment Lorrie Shepard: 1900-1980ies: traditional testing and measurement aligned with behaviorist theories of learning, hereditary views of intelligence and social efficiency curriculum • “Because it is difficult to articulate or confront formal theories once they have become a part of popular culture, their influence may be potent but invisible long after they are abandoned by theorists”(Shephard 2000, p 1068)

12. Cognitive, Construct. , SituativeLearning Theories Classroom Assess-ment 20th cent Dominant Paradigm (ca 1900s -2000+) Dissolution of Old Paradigm: New Views of Instruction/ Old Views of Testing (ca 1980-2000+) Emergent Paradigm (ca 1990-2000+) Shepard (2001), Reformed Vision of Curric. Social Efficiency Curriculum Herditarian Theory of IQ Assoc. & Behaviorist Learning Theories Construct. Theories of Curriculum & Instruction Scientific Measure- ment Traditional Testing

13. Views of knowledge and learning- consequences for assessment? • Behaviorist perspective • quantitative: how much? - atomistic items • Cognitive • general principles, strategies, problem solving • The situative /sociocultural • Participation • Integration • Identity formation • Multiple intelligences

14. IV. Challenges to teachers and students in practicing portfolios • Two portfolio definitions: • A portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress, or achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for judging merit, and evidence of the student’s self reflection (Paulson, Paulson & Meyer (1991p. 60). • Portfolio assessment is a purposeful, multidimensional process of collecting evidence that illustrates a student’s accomplishments, efforts, and progress (utilising a variety of authentic evidence) over time” (Gillespie et al 1996 p. 487).

15. portfolios • Two origins • The competency movement • Externally defined standards and norms • Example: The European Language Portfolio • Humanism & constructivism • Personal development, understanding • Wide scope for students’ own documentation of their learning • Reflection, metacognition, peer response

16. A Norwegian assessment project: electronic portfolios in teacher education • Department of Teacher Education University of Oslo • University College of Vestfold • University College of Stord/Haugesund • Subject areas where portfolios were used • Norwegian literature and language • Pedagogy • Mathematics • Natural science • Religion • Subject didactics • Overall finding: high student satisfaction • Dysthe & Engelsen: Mapper som pedagogisk redskap 2003 • Wittek: Mapper som lærings- og vurderingsform 2003

17. Some challenges of portfolio assessment 1 • Balancing summative and formative assessment • Balancing teacher and student control • Balancing individual and collaborate work • Digital portfolios: ” filing cabinet or learning arena”?

21. Challenges of portfolio assessment 2 • ”Giving students the tools they need to succeed” • Digital literacy and CSCL competence • How to deal ethically with sources (plagiarism, referencing) • Peer assessment • Reflection Self assessment • Giving & utilizing feedback • Negotiating and using quality criteria • How to assess and mark portfolios

22. V. The challenge from global trends towards testing and accountability • California revisited • Scandinavia ? • Research needed • to clarify theoretical basis of new modes of assess • to document learning effects and critical factors • to understand how to improve feedback and use • to investigate constructive use of criteria • to develop self and peer assessment • to expand traditional concepts of validity & reliability • to improve the quality of new modes of assessment

23. My line of argument • If student learning is our concern • Changes in society warrants a new learning culture • Alternative assessment needed to foster and assess the broad range of competence needed in the 21 century • New assessment need to be aligned with theories of knowledge and learning • Students and teachers need • New forms of assessment need quality assurance in order to gain legitimacy and in order to withstand the strong international measurement trend