1 / 24

The Importance of being Valid

The Importance of being Valid. Alastair Pollitt & Ayesha Ahmed Cambridge Exam Research alastair@camexam.co.uk ayesha@camexam.co.uk. Defining ‘Validity’. There is no true meaning of’ Validity’ Definitions of ‘Validity’ are opinions about what the word should mean

felton
Download Presentation

The Importance of being Valid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Importance of being Valid Alastair Pollitt & Ayesha Ahmed Cambridge Exam Research alastair@camexam.co.uk ayesha@camexam.co.uk

  2. Defining ‘Validity’ • There is no true meaning of’ Validity’ • Definitions of ‘Validity’ are opinions about what the word should mean • How do we decide what ‘Validity’ should mean? • We need an inclusive concept of ‘Validity’ that is useful to everyone involved in the process of assessment, and not too far from what they normally mean by ‘Valid’. • A useful definition of ‘Validity’

  3. The real world meanings of “valid” Relative Rank Concordance frequency • 1 offer 102 • 2 ticket 32 • 3 licence 25 • 4 certificate 25 • 5 passport 19 • 6 law 15 • 7 voucher 14 • . . . . . . . . . . . . . . . • 18 reason 6 Source: Collins Cobuild Corpora (British + American)

  4. What we mean by Validity • We have worked for many years with professionals who: • Write questions; Review questions; Write mark schemes; Mark tests; Design tests; etc • The key issue for them, and us, is what the students’ minds are/were doing during the test – • in other words, • the Evidence that gives us about achievement.

  5. What we mean by Validity • This is the essence of Validity – the interaction between a test question and the student’s mind. • If this goes wrong then Validity is impossible. • But how do we get it right? • By considering further the concept of Evidence.

  6. Evidence • It is not enough to be Logically Valid. • A “widely used definition of assessment”: The process of seeking and interpreting evidence for use by … (Assessment Reform Group, 2002: quoted by Stobart, 2009)

  7. Importance Statement • A negotiated consensual statement of what is really important in the study of that subject during that phase of education. • Available to teachers, students, parents, examiners, employers, selectors – to everyone with an interest in the subject.

  8. The Importance of Design & Technology • Design and technology prepares pupils to participate in tomorrow's rapidly changing technologies. They learn to think and intervene creatively to improve quality of life. The subject calls for pupils to become autonomous and creative problem solvers, as individuals and members of a team. They must look for needs, wants and opportunities and respond to them by developing a range of ideas and making products and systems. • They combine practical skills with an understanding of aesthetics, social and environmental issues, function and industrial practices. As they do so, they reflect on and evaluate present and past design and technology, its uses and effects. Through design and technology, all pupils can become discriminating and informed users of products, and become innovators.

  9. The Importance of the Importance Statement • To ensure that teachers emphasise what is important • To ensure that test designers, question and mark scheme writers test what is important • To ensure that markers give credit for what is important • To ensure that test results are interpreted in terms of what is important • The source of Validity

  10. A model of Validity • Validity is about measuring and interpreting what is important in education. • It is not merely about measuring and interpreting what is measured. • Our model describes where Validity comes from, and how it is maximised.

  11. The Bucket Brigade model Suppose: Y is a house on fire. Suppose: X is a well full of water. How do you put out the fire?

  12. A typical assessment professional

  13. The Entropy law of validity • Version 1 • During the assessment process, validity always decreases Version 2 Validity can only be lost, it can never be recovered.

  14. How to maintain validity - 1 Writing the Importance Statement

  15. How to maintain validity - 2 Specifying the test

  16. How to maintain validity - 3 Writing the questions and mark schemes

  17. How to maintain validity - 4 The students

  18. How to maintain validity - 5 Marking

  19. How to maintain validity - 6 Grading, etc

  20. How to maintain validity - 7 Interpreting results

  21. How to maintain validity • 1 Writing the Importance Statement • 2 Specifying the test • 3 Writing the questions and mark schemes • 4 Communicating with students • 5 Marking • 6 Grading • 7 Interpreting

  22. Summary • The meaning of test results depends crucially on a proper development of the test, so that it elicits evidence of what is important. • Without meaning, Validity is an empty concept. • The source of Validity must be a clear and shared concept of Importance.

  23. The importance of being Valid Importance Statement IS Teaching A T Idea of a Question i E Q Elicitation M Evaluation 8 Assessment Evidence

More Related