250 likes | 263 Views
Knowing What Students Know. Ganesh Padmanabhan 2/19/2004. Overview. Nature of Assessment Need for Reform Advances in Cognition Advances in Measurement Role of Technology. Assessment.
E N D
Knowing What Students Know Ganesh Padmanabhan 2/19/2004
Overview • Nature of Assessment • Need for Reform • Advances in Cognition • Advances in Measurement • Role of Technology
Assessment • “Assessment is the means used to measure the outcomes of education and achievement of students with regard to important competencies.” • Purpose determines priorities. • Context of use imposes constraints on design.
Purposes of Assessment • Achievement vs Aptitude • Formative vs Summative • Evaluation of Educational Programs • More and more high-stakes decisions • One type does not fit all • Need for better alignment
Reasoning From Evidence • “Assessment is a tool designed to observe students’ behavior and produce data that can be used to draw reasonable inferences about what the students know.” • Also inferences about how, when, and whether they use what they know. • Imprecise by-nature
Assessment Triangle Observation Interpretation Cognition
Cognition • “…refers to a theory or set of beliefs about how students represent knowledge and develop competence in a subject domain.” • Cognitive Theories • Educational Theories • Experience of Expert Teachers • Need not be exhaustively complex
Observation • “Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills.” • Not arbitrary, needs careful design.
Interpretation • “…expresses how the observations derived from a set of assessment tasks constitute evidence about the knowledge and skills being assessed.” • Methods for drawing inferences • Classroom – usually qualitative • Large-Scale – usually formal, statistical
Making Sound Inferences • Depends on making explicit connections in assessment triangle between • Cognition and Observation • Cognition and Interpretation • Observation and Interpretation • Interdependent and Iterative Development
Advances in the Learning Sciences • “How People Learn” by NRC (1999) • Systematic investigation dates back to the late 19th Century • Recently, very diverse fields converging • “Cognitive Revolution” (1960s onward) • Four significant perspectives: Differential, Behaviorist, Cognitive, and Situative
Differential Perspective • “The differential perspective focuses mainly on the nature of individual differences in what people know and in their potential for learning.” • Early 1900s • Bell Curve Assumptions • Focus on aptitude not achievement
Behaviorist Perspective • Skills are thought to be composed of stimulus-response associations. • Considers learning as the process by which one acquires those associations and assembles them into skills. • Stimulus, reinforcement, conditioning theories etc.
Cognitive Perspective • “In cognitive theory, knowing means more than the accumulation of facts and knowledge; it means being able to integrate knowledge, skills, and procedures in ways that are useful for interpreting situations and solving problems.” • Aids assessment in figuring how, when, and whether students use what they know.
Situative Perspective • Sociocultural View • Places individual thinking within a practical context • Authentic Assessment • Needs more here.
Interpreting Evidence • Psychometrics • Probabilistic Approach to Reasoning • Cognition Construct • Observation Observational Model • Interpretation Measurement Model
Student Construct Observations Generation of Data ө x Interpretation of Data “…statistical models can be developed to predict the probability that people will behave in certain ways in assessment situations, and that evidence derived from observing these behaviors can be used to draw inferences about students’ knowledge, skills, and strategies…”
Classical Test Theory Construct ө Sum of Responses Observations
Generalizability Theory ө Construct Sum of Responses Observations Raters Type of Task
Item Response Modeling ө Construct Response 1 Response 2 Response 3 Response 4 Response 5 Observations δ1 δ2 δ3 δ4 δ5
Latent Class Models • Fig 4-8
Multi-Attribute Models • Fig 4-9
Role of Technology • Theory-Based Item Generation • Concept Organization • Complex Problem Solving (e.g. IMMEX) • Text Analysis and Scoring • Portal computer-aided approach for design of assessment
References All quotes and figures were taken from the NRC Report “Knowing What Students Know” by Pellegrino et al. (2001)