Measurement in research

1. Measurement in research 報告者：許之馨 授課老師：任維廉 教授

2. 自我介紹 • 許之馨 • 台北人 • 運管系大四 • 興趣：看電影、吃美食

3. Outline • Measurement • Validity • Content validity • Criterion-related validity • Construct validity • Reliability • Standard error measurement • Internal consistency • Equivalence • Stability • Response set

4. Measurement • All descriptive and experimental research studies involve some kind of measurement. • Two related terms • Test • Evaluation • Formative evaluation • Summative evaluation

5. Validity • The tools used in descriptive research • Test • Questionnaires • Interview guide • The extent to which a research tool measures what it intends to measure.

6. Validity • Three major type • Content validity • Criterion-related validity • Construct validity

7. Content validity • Item in research tool • Related to the subject matter tested / stated objectives of a course of study or program? • Reflecting the emphasis placed in a course or a program? • Representative of the universe of items? • Non-statistical

8. Content validity • Test blueprint • Two-way chart ( related specific objective to specific content areas ) http://www.utexas.edu/provost/sacs/pdf/Test%20Blueprint%20handout.pdf

9. Criterion-related validity • Empirical (statistically) • Comparison the scores on the to-be-validated test and the scores on a criterion measure • Correlation coefficient

10. Criterion-related validity • Concurrent validity • The to-be-validated test and the criterion test were made at same time or after a short interval • Predicative validity • A much longer time interval exists between the two test situations or the two assessment situations

11. Construct validity • Construct • Psychological traits or characteristic • Not directly observable but be inferred on the basis of overt behavior • To the extent of measuring a theoretical construct or trait • Controversial

12. Reliability • The consistency of getting the same or similar responses • The accuracy of the score of one person • The person’s score obtained on a test is not the person’s true score • Standard error measurement(SEM) • The consistency of score of a group of people • Internal consistency • Equivalence • Stability

13. Standard error measurement • Test someone with all our tremendous number of equivalent forms. Then we will find that scores are not the same. The distribution of scores familiar are resemble the familiar “ normal ” curve. • The average of scores is his true score.

14. Standard error measurement • The standard deviation as a measure od the variation of observed scores around the true score SD : standard deviation of obtained scores of a group : the reliability coefficient computed for the same group

15. Internal consistency • Reliability derived from the administration of a single test (questionnaire, interview) • Methods of determining reliability correlation coefficient • Split half • Divided into two equal halves. • Correlation coefficient the of scores of two half-tests. • The Spearman-Brown formula to get the reliability of the entire test r : reliability of the full test r1/2 : reliability of the two half tests

16. Internal consistency • Kuder-Richardson : estimate of reliability based on Kuder-Richardson n : number of items in the test

17. Equivalence • Parallel test form are administration to a group of people at the same time or with very little time lapse. • The correlation coefficient is computed between the scores of the two tests.

18. Stability • The same test after substantial time lapse • One test version here or now and equivalent test version after substantial time interval

19. Response set • A consistent tendency to follow a certain pattern in responding to items in a test (questionnaire, interview) • Interference with getting usable data • Avoiding extreme response option • Socially expected response • Faking

20. Response set • Weaken validity • Halo Effect • Generosity Error • Error of Central Tendency

21. Thanks for your attention Q&A