Finishing off reliability

1. Finishing off reliability

2. Today • Check in • Finish off reliability • In class exercise

3. Check in • Thursday • In class review exercises due • Article assignment number 2 is due • Next tuesday: • Midterm exam • Odds and ends • Previous article assignment • APA style reminders

4. Previous article assignment • What type of design was it? • What was the non-overt exclusion group for? • What did the findings mean? • Future studies? • What did you think about overall article • Writing style?

5. APA style • References are double spaced throughout • They also use “hanging indent” • A “citation” is what is “in the text.” • Article with 6+ authors gets only first author citation • Article reference • Include journal volume but NOT issue # • Journal and volume are both italicized • Include full page numbers of article

6. APA style • Tables go on a separate page after the references • One table per page • Tables are also double spaced throughout

7. Heading levels? • Blech. • How headings are arranged and formatted depends on how many there are • You are probably likely to be using two or three headings for your proposals

8. Your proposals? • What is the difference between a proposal and a study? • Proposal is the lit review and proposed methods • It is essentially the parts of an uncompleted study • Title, abstract, introduction, proposed method

9. More reliability • Interobserver/Interrater reliability • Do two trained raters who see the same phenomenon rate it the same way? • Why would this matter? • Goal: To make sure that a phenomenon can be reliably measured by different observers • And that the observed phenomenon has a shared definition across observers

10. Interobserver reliability • Depends on the “questions” • An observable characteristic should be…observable • Easiest when we break a quality into small behaviors • E.g. “safe driving” vs. “stops completely at stop signs, does not exceed speed limit, etc.” • “happy baby” vs. “frequency baby smiles, makes eye contact, baby laughs…etc.”

11. Interobserver reliability • An observational measure must be “reliable” across observers • In practice • Two observers independently observe the same thing • Make ratings • We calculate how similar ratings are • The statistic is Kappa or Cohen’s Kappa

12. Internal consistency reliability • Also sometimes called “alpha reliability.” • Do similar questions (or questions intended to be similar) get similar responses? • If my survey is on delinquency, do people on average respond to similar questions in a similar way? • If not my “construct” may be faulty, or my questions may be poorly worded or both

13. Internal consistency reliability • Measured using a calculation of how scores on individual questions are correlated • Statistic referred to as “Cronbach’s alpha” • Easy to calculate in stats packages

14. Split-half reliability • Is what it sounds like • Split your survey in two (where questions are all similar) • See if the responses for the two halves are correlated • If they are, then responses are consistent

15. Reliability and validity • A measure is only as valid as it is reliable • Though you can have a very reliable measure that is not valid • Reliability is necessary but not sufficient for validity