VALIDITY

1. VALIDITY

2. Validity is the most important concept in assessment Think of VALIDITY as accuracy

3. VALIDITY refers to the degree to which a test measures what it purports to measure

4. A test is valid to the extent that it measures what it is supposed to measure.

5. Teachers use assessments to make decisions about students and those decisions can only be accurate if the test results themselves are accurate.

6. The test results can only be accurate if the questions are representative of the entire subject area.

7. The principle premise of assessment is that you can tell how much a person knows about an area by asking a person to show a representative example of his knowledge of the field.

8. Thus�

9. Based on the student�s good score on the exam

10. Validity centers on the accuracy of the inferences that teachers make about their students.

11. A representative example would be some of every different type of content within the field. �

12. In theory, you would need only one example of each different bit of information in the field in order to say you have sampled the field adequately.

13. Basic Math Knowledge All math operations

14. Poor sampling results in anon-representative sample which doesn�t look like the original that it came from

15. A good test has some questions about every area that was studied although areas that are more important should have more questions.

16. The larger the field of study, the more test questions it takes to get a true idea of all the student knows.

17. Therefore�The more questions that you have on a test, the more accurate or valid the test becomes�

18. Because you are increasing the likelihood that you are getting a representative sample when you increase the number of questions.

19. There are three types of validity

20. The 3 types of Validity are� Content Criterion Construct

22. Content Validity describes the knowledge or skill that we want the students to master. It was formerly called INSTRUCTIONAL OBJECTIVE

23. Content Validity How well the content of the test represents the content area of the subject being studied. This is more than just factual knowledge although facts are important. It includes reasoning and attitudes. This is the most important form of validity.

24. 2 Ways to determine Content Validity The teacher reviews the subject material and then makes sure that each area or concept is represented on the test A knowledgeable third party reviews the test and offers an opinion as to how the test can be improved to make it more representative of what the students should know.

25. This third party can be� Another teacher in the school who teaches the same subject The state or national standard for the grade level and subject Someone who is especially knowledgeable about the subject or a panel of specialists who work in the field

26. Purpose The purpose of Content Validity is to make sure that an assessment instrument gives a comprehensive and accurate measurement of the content of the original field under study. It shows mastery of the field which was studied. It allows for comparison between different students, schools or methods of instruction.

27. For Example� Those students who perform well on tests are passed to the next grade while those students who do not do well are held back. College admissions take students� grades into account even though the students come from different schools and backgrounds and have been exposed to different levels of instruction.

29. Purpose The purpose of criterion-related evidence of validity is to predict how well students will perform on a subsequent criterion.

30. For Example� A scholastic aptitude test is used to predict future grades and success in school. The test is taken when the students are in high school and are supposed to predict the students� college grades. Scholarships and honors are frequently awarded based on the results of these tests (i.e. the National Merit Scholarship).

31. Criterion Related Validity is not just related to tests. It can be used in any circumstance where there is a need to predict future events such as success in basketball based on the student�s height at a certain age or the likelihood that a student will want to go into medicine based on having certain personality traits.

32. Methodology The standard method for computing criterion related validity is to give an assessment based on the first criterion and then measure the second criterion and calculate the correlation between the two. If the correlation is high between the two variables, we infer that the first variable is a good predictor of the second variable.

33. Example A test is constructed which will supposedly predict future income in the 5 years after graduating from college. The test is given to 1000 college seniors. Five years later, the former seniors are contacted and their income is recorded and correlated with their result on the initial test.

34. Those questions which correlated highly with the student�s income are retained in the test. The students� scores on the modified test (minus the poor questions which were eliminated from the test) should then again be correlated with their incomes to confirm a high correlation. The test is now ready to be given to a new group of college seniors in order to predict their earnings 5 years into the future.

35. This method of making a test is obviously time consuming (i.e. waiting 5 years for the 2nd criterion to occur. There is a faster way to make a test but it is not necessarily as accurate.

36. The researcher can start with a group of subjects who have been out of college for 5 years, administer the predictive test, and correlate the two scores.The correlation between the two measures is called concurrent criterion validity

37. Take care that you do not place too much trust in tests which predict future success. Other factors such as study habits, motivation, interest, attitude, etc. frequently play such a large part in performance that many times test scores designed to predict success are rendered meaningless.

39. Construct Validity measures intangible concepts such as anxiety, motivation, attitudes, etc.

40. The more elusive the construct being assessed, the more numerous and varied validation studies you will need to make sure your test is valid.(It is easier to prove that a test measures addition ability and more difficult to prove a test measures leadership ability)

41. There are 3 major ways of measuring Validity Intervention Studies Differential-Population Studies Related-Measures Studies

42. Intervention Studies One way to measure Construct Validity is to give a pretest which purports to measure the concept in question. Present a program or other intervention. Give a post-test to see if there are changes in the students� scores. Attribute the difference in scores to your program.

43. Differential-Population Studies Based on his knowledge of the construct, the researcher hypothesizes that different populations will score differently of a test measuring this construct. For example, it could be hypothesized that a group of nurses would have higher scores than a group of juvenile offenders on a test to measure nurturing. The test could be given to both groups and the scores compared to see if there are significant differences.

44. Related-Measures Studies Another name for this is convergent validity There are 3 variables: The Test which is being developed The concept which is being tested An existing test which has already been shown to measure the concept (e.g. a standardized test)

45. Each student is given the test which is being developed and the existing (standardized) test which is already accepted as measuring the concept that is being studied. The results of the two scores are correlated. If there is a high correlation between the two tests, it can be inferred that because the second test is accepted as measuring the concept, the new test also measures the concept.

46. Regarding validity of a test � the more evidence you have to support the validity, the better

47. Validity is the keystone of educational measurement

49. The teacher must make sure that the test is being given to the group for which it was intended!

50. If an otherwise valid test is given to a group of students other than those for which it was intended

51. The inferences drawn from the test results will not be valid.

52. What is Face Validity? Face Validity means that the test appears to be valid.

53. What is Consequential Validity? Consequential validity refers to whether the uses of test results are valid. � Is the test being used for the purpose for which it was intended?

54. Standards for Educational and Psychological Testing This joint publication of the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education (published in 1999) outlines the latest thinking and practices in testing and education today.

55. If a test is found to bevalid it is also reliable. The converse is not true: (i.e. A test can be reliable but not valid.)

56. Review There are 3 types of Validity� Content-Test samples the entire subject content* Criterion-Test predicts future achievement Construct-Measures vaguely defined concepts *This is the primary validity for classrooms