1 / 52

Psychometrics

Psychometrics. Marcin Zajenkowski, PhD. Marcin Zajenkowski zajenkowski@psych.uw.edu.pl COURSE PAGE http://icelab.psych.uw.edu.pl/students-area/ Password: intelligence. Aim of the course. B asic issues concerning construction and utilizing psychological tests.

jivy
Download Presentation

Psychometrics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Psychometrics Marcin Zajenkowski, PhD

  2. Marcin Zajenkowski zajenkowski@psych.uw.edu.pl COURSE PAGE http://icelab.psych.uw.edu.pl/students-area/ Password: intelligence

  3. Aim of the course • Basic issues concerning construction and utilizing psychological tests. • Fundamental vocabulary and logic of psychological measurement. • Logic of design and psychometric properties behind any psychological test.

  4. Expectations • You can have 4 absences. • The exam questions will be based on assigned basic literature (60%) and lectures (40%)

  5. Grading • There will be two summative exams: midterm and final worth in total 50 points. • Midterm exam (25 points) will cover material from lectures 1 – 7 • Final exam (25 points) will cover material from lectures 8 - 15. • Retake will cover all material. • Exams will consist several multiple choice and open ended questions covering material from the literature and lectures.

  6. Extra credit • You can obtain additional points for 3 tasks. • Important: it should be brought by yourself to the class.

  7. Extra Credit 1 Individual work worth 1 point. • Go to Lab of Psychological Tests (1st floor, room 52) or find elsewhere (e.g. Internet), ask for any test. Write a short report including: • name of the test • what does it measure • how its reliability has been assessed, how how it is • Does the test have norms? If so, what kind of norms • Bring to 7th class, November 15th

  8. Extra credit 2 Individual work worth 1 point. • Find any published study that used a psychological test or a questionnaire, write a short report: • Describe briefly the study • Name tests used • If there are more than 1 test, choose one and say what kind of validity is measured in this study. • Bring report to class, January 17

  9. Extra Credit 3 Group assignment worth additional 3 points. • Select one of five proposed constructs and create a draft of the tool to measure it. • Presentations (5 minutes) – last class, January 24

  10. Extra credit 3 • Choose one psychological construct (e.g. altruism, verbal intelligence). Prepare presnetation and provide: • 10 items assessing the construct • Which response type would be best for your test? • Which method of determining reliability would be most adequate for your test? (justify your claim) • Describe three possible studies that might examine the validity of your test– base on the psychological literature, scientific articles, studies reports. -more details can be found on the course page-

  11. Psychological constructs to choose 1. Sensation Seeking is a trait defined by the search for experiences and feelings, that are varied, new, complex and intense, and by the readiness of experiencing those physical, social, legal, and financial experiences.

  12. 2. Altruism is a concern for the welfare of others. Altruism is the opposite of selfishness. It can be distinguished from feelings of duty and loyalty. Altruism is a motivation to provide something of value to a party who must be anyone but the self, while duty focuses on a moral obligation towards a specific individual (for example, a god, a king), or collective (for example, a government). Pure altruism consists of sacrificing something for someone other than the self (e.g. sacrificing time, energy or possessions) with no expectation of any compensation or benefits, either direct, or indirect (for instance from recognition of the giving).

  13. 3. Math anxiety is an emotional reaction to mathematics based on a past unpleasant experience which harms future learning. A good experience learning mathematics can overcome these past feelings and success and future achievement in math can be attained.

  14. 4. Optimism is a mental attitude that interprets situations and events as being best (optimized), meaning that in some way for factors that may not be fully comprehended, the present moment is in an optimum state. The concept is typically extended to include the attitude of hope for future conditions unfolding as optimal as well. This understanding leads to a state of mind that believes everything is as it should be, and that the future will be as well.

  15. 5. Verbal intelligence is the ability to analyze information and solve problems using language-based reasoning. It is associated with the ability to listen to and recall spoken information; understanding the meaning of written or spoken information; solving language based problems of a literary, logical, or social type; the ability to perform complex language-based analysis.

  16. Extra credits during classes • There will be additional tasks during some of the classes • E.g. calculate, use a formula etc. • You can obtain extra points for this (up to 3).

  17. Grading

  18. Book Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications, (6ed). Upper Saddle River, NJ.: Prentice-Hall, Inc.

  19. Course schedule 1. Introduction (4.10.2018) Course description, Definition of psychological test, Item format 2. Psychometrics basic concepts and statisticsrefresher (11.10.2018) Basic aspects of psychological testing - what is it, why we are using it and how we are using it?Types of test, Standardization. • Chapter 1: Tests and Measurements - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications, (6ed). Upper Saddle River, NJ.: Prentice-Hall, Inc. Basic statistic - variable types, distributions, probabilities, sampling • Chapter 4. Basic Concepts in Measurement and Statistics - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

  20. Course schedule 3. Reliability (18.10.2018) -True score theory, methods of determining reliability: test-retest, alternate form • Chapter 6. Murphy, K. R. & Davidshofer, C. O. (2005).Psychological testing. Principles and applications. 4. Reliability (25.10.2018) - Methods of determining reliability: internal consistency • Chapter 7. Murphy, K. R. & Davidshofer, C. O. (2005).Psychological testing. Principles and applications. 5. Reliability: new approaches, Item Response Theory (8.11.2018)

  21. Course schedule 6. Norms (15.11.2018) Normalization, transformation of raw scores, types of normalized scales, criterion reference. • Chapter 5. Scales, Transformations, and Norms - Murphy, K. R. & Davidshofer, C. O. (2005).Psychological testing. Principles and applications. • Bring: Extra credit 1, November 15th 7. Midterm Exam (22.11.2018) November 22nd 2018!!!

  22. Course schedule 8. Validity (29.11.2018) Definition of validity, content validity, examples • Chapter 8. Murphy, K. R. & Davidshofer, C. O. (2005).Psychological testing. Principles and applications. 9. Validity (4.12.2018) Criterion-related validity, examples, construcs validity, examples • Chapter 9. Murphy, K. R. & Davidshofer, C. O. (2005).Psychological testing. Principles and applications.

  23. Course schedule 10. Validity of specific tests (11.12.2018) • Tests from various areas of psychology and their specific features, examples of personality, intelligence, neuropsychological, affect tests 11. Test construction 1(18.12.2018) Strategies of test development. • Chapter 11. The Process of Test Development - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

  24. Course schedule 12. Test construction 2 (10.01.2019) Item sampling, Structure exploration and confirmation, Item analysis: difficulty and discrimination • Chapter 10. Item Analysis - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications. (p.202-225) 13. Response biases (17.01.2019) • Tests in different cultures, tests adaptation process • Bring Extra Credit 2, January 17.

  25. Course schedule 14. Presentations of extra credit’s work (24.01.2018)

  26. Short test…

  27. Tests items examples

  28. 1. Describe briefly what a psychological test is?

  29. Open-ended / essay • What could be measured: knowledge, opinion. • Easy to construct • Encourages more appropriate study habits • Measures higher-order outcomes (i.e., analysis, synthesis, or evaluation goals), creative thinking, writing ability

  30. Open-ended / essay - Cons • Hard to score • Can yield great variety of responses • Not efficient to test large bodies of content • If you give the choice of three or four essay options, you can find out what people know, but not what they don't know

  31. 2. Which type of coffee do you like best? a. black b. white c. latte d. cappuccino e. don’t like coffee

  32. Multiple Choice • What could be measured: abilities, knowledge personality, prefernces. • Which scale of measure is used? • More answer options (4-5) reduce the chance of guessing that an item is correct • Difficult to write four or five reasonable choices

  33. Multiple Choice • With correct and incorrect anwers. • What can be measured: knowledge, cognitive performance, abilities. • Which scale of measure is used?

  34. 3. Knowledge about psychological tests is important in psychological practice. • Strongly disagree • Disagree • Neither agree nor disagree • Agree • Strongly agree

  35. Likert scale • What can be measured: personality, all constructs assessed with self-report methods. • Which scale of measure is used? • Involved in research that employs questionnaires. • Named after its inventor, Rensis Likert. • The range captures the intensity of respondents feelings for a given item.

  36. 4. Mark anywhere along the line to show how much do you like to wake up at 7 am? It’s horribleIt’s terrific

  37. Graphic rating scale • What can be measured: prefernces. • Which scale of measure is used? • Respones are converted to numerical form. • Also: semantic differential, e.g. hot-cold.

  38. Supplied Response • What can be measured: knowledge and fact outcomes, terminology, formulas. • Which scale of measure is used? • Chances of guessing reduced.

  39. 5. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? 6. If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?

  40. 7. What does this person feel?

  41. 8. Mark whether you like (L), dislike (D), or are unsure (?) about the following activity: Partying with friends. D L ?

  42. Muliple Choice • Sometimes an even-point scale is used, where the middle option is not available. • This is sometimes called a "forced choice" method

  43. Are there any other items?

  44. Psychological test - definition

  45. What is a psychological test • „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999 • „Test – objective and standardized measure of a sample of behaviour” Anastasi (1988) • „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

  46. What is a psychological test • „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999 • „Test – objective and standardizedmeasure of a sample of behaviour” Anastasi (1988) • „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

  47. What is a psychological test • „An evaluative device or procedurein which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999 • „Test – objective and standardized measure of a sample of behaviour” Anastasi (1988) • „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

  48. What is a psychological test • „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999 • „Test – objective and standardizedmeasure of a sample of behaviour” Anastasi (1988) • „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

  49. What is a psychological test • systematic procedure • a sample of behavior • standardization • scoring and evaluating

More Related