Measurements and Validity in Psychology

1. Measurements and Validity Week 1, session 2

2. Today’s questions • How do you measure a psychology attribute? • What makes a good measurement?

3. How do you measure a psychology attribute?

4. Latent constructs in science • Latent = “present but invisible” • Psychological constructs are abstract • They must be inferred. • But this is not unique to psychology.

5. Latent constructs in science • Dark matter (physics) must also be inferred • https://www.youtube.com/watch?feature=player_embedded&v=HneiEA1B8ks

6. Problem • We can’t see it, so people disagree how best to measure it. • Potential for miscommunication (“Blind men & elephant”) • Potential for miscommunication fraudulent claims

8. https://www.youtube.com/watch?v=GCxAlxJBc4c

9. What makes a good measurement?

10. Recall • Last session, we discussed how to measure greed, rudeness, etc. • That is: Given theory about X, how do you measure X? • This session, how do we evaluate X?

11. Evaluating a measurement • Theoretical • Statistical • Methodological • Logical

12. Theoretical constructions • Defining the construct and the domain of observables • Example: • What is shyness? • What is self-esteem?

13. Statistical methods • Does the measure correlate well with other measures? (Jargon: “predictive validity”) More if you take SRM I

14. Statistical methods • Does the measure correlate poorly with measures that it should correlate poorly with? It measures… Levels of radiation in water Levels of oxygen Levels of liquid More if you take SRM I

15. Statistical methods • Does the measure correlate poorly with measures that it should correlate poorly with? (Jargon: “discriminant validity”) Love Anger Disgust More if you take SRM I

16. Methodological reasoning • Construct has been defined. • A measurement has been made. • Measurement does predict outcome. No problems, right?

17. Example: Oxygen therapy Watch the “What is an Oxygen Bar” Video http://www.oxygenbars.com/

18. Example: Oxygen therapy • A researcher wants to find out if oxygen therapy works in improving happiness • He gives oxygen therapy to all customers and found that they became happier over time

19. Methodological reasoning Study A Study B More in Week 13 & SRM II

20. If oxygen therapy worked… • 1. You’d be superior to a fish • 2. You’d be dead (oxygen is toxic at high doses)

21. Logical reasoning • The theory behind the measurement should be sound. • This is difficult if you are not an expert. Hence we are susceptible to heuristic processing: “Dr Oz has an MD  he must be correct” • If it sounds too good to be true, it probably is.

22. How much would you pay?

23. Measuring personality: BraincapTM

24. The brain

25. Group exercise

26. Let’s examine the logic <True, False, Not sure> • We have a brain. • Brains consists of neurons. • An active neuron produces electrical charges • These electrical charges can be detected • Brain have different parts • Different parts are perform different functions (functional specialization) • When a part is active, electrical charges can be detected.

27. Let’s examine the logic <True, False, Not sure> • People are different • People think differently • Because people think differently, they have different personalities • Thinking produces electrical charge • EEG measures electrical charge • Therefore, EEG measures personality

28. References • Korjus K, Uusberg A, Uusberg H, Kuldkepp N, Kreegipuu K, Allik J, Vicente R and Aru J (2015) Personality cannot be predicted from the power of resting state EEG. Front. Hum. Neurosci.9:63. 

29. Take home messages • Measurement problems are not unique to psychology • Four ways to evaluate a measurement • Theoretical • Statistical • Methodological • Logical • Use these methods to examine the issues in this course

30. Beware • “Science has proven…” • Pictures of brains sell • Profit-driven • Advocates who have not published scientific papers in reputable journals • Confirmation bias • Shoddy science • Premature science

31. The broader picture • Critical thinking in psychology as an inoculation against pseudoscience • https://www.ted.com/talks/james_randi?language=en