Economics 105: Statistics

1. Economics 105: Statistics GH 19 not due Tuesday ! RAP assignment

2. The Structure of Research(aka, “the scientific method”) An "hourglass" notion of research Begin with broad questions to a problem. Narrow down, focus in. Operationalize. OBSERVE Analyze data. Reach conclusions. Generalize back to questions.

3. The Research Process • Identify a problem/issue/topic of interest (or be intellectually curious) e.g. obesity, low literacy, healthcare, income inequality, poverty, crime, investing, child mortality • Read the existing literature and write a thorough literature review

4. The Research Process 3. Narrow the problem/issue down to a focused research question e.g., what is the effect of education on income? what are determinants of child mortality? 4. Outline a theory or conceptual framework 5. State a testable hypothesis 6. Design a study to answer the research question ... how does one infer causality?

5. The Research Process • 7. Collect data • measure outcomes and the factors causing those outcomes • 8. Analyze and interpret the data • 9. Revise the theory as needed • 10. Replicate the study/repeat for different populations (no single study is definitive)

6. Where Do Research Topics Come From? • Practical problems in each field/discipline • Literature in the field/discipline • Your own thinking, experience, knowledge

7. The Structure of Research What kinds of questions does science address? What is the general opinion about democracy in a country? Do women and men view democracy differently? **Note gender does not “cause” different opinions about democracy Does an increase in per capita income cause a country to become more democratic? Descriptive: Relational: Causal: http://www.vanderbilt.edu/AEA/students/Econliterature.htm

8. Elements of a Research Question • What? Obesity (BMI) • Who? Children, Adolescents, Adults, “Vulnerable” population • Where? Hospital, Worksite Neighborhood, State, Country • When? Today, last week, last month, last year • Why? Diet, exercise, socioeconomic status, availability of sidewalks, aggressive fast food advertising

9. + + Dependent variable Body Mass Index Dependent variable Body Mass Index - - - + - + Independent variable Exercise Independent variable # of Fast Food Burgers Types of Association Between Variables

10. + Cardio Exercise + Dependent variable Weight (lbs) Strength Training Dependent variable Body Mass Index - - - + - + Independent variable Time in Weight Management Program Independent variable Hair Color Types of Association Between Variables • Remember that a sample correlation coefficient can quantify the magnitude of an association, but it does not imply causality!

11. How do I determine if higher per capita income causes a country to become more democratic? • 3 criteria must exist in order to infer causality • Association • Direction of Influence • The cause must precede its effect. (none of this ) • Hypothesized relationships should always specify direction of influence whenever possible. • Non-spuriousness • Elimination of the rival hypothesis

12. Number of storks Number of births Number of storks ? Number of births Wallis & Roberts 1956 Nonspuriousness

13. Number of firefighters college G.P.A. Amount of damage Income 20 years after college Number of firefighters Income 20 years after college ? ? Amount of damage college G.P.A. Nonspuriousness

14. Nonspuriousness • Consider the following fake headlines: • “Bottled Water Linked to Healthier Children” • It invites a causal inference, but is one warranted? • “Longer Hair & Higher GPA go hand-in-hand at college X” • “Companies that use SAP software earn 32% higher profits.”

15. Nonspuriousness • Famous orchestra conductors were found to have a sample mean life expectancy of 73.4 years (Atlas, 1978). • Is this relatively long? What’s the relevant comparison group? • Orchestra musicians? Nonfamous conductors? General public? • Researcher chose the U.S. population, and mean life expectancy at the time was 68.5 years. • Famous conductors get 5 extra years! Causal?

16. Effect of income on democracy? Source: Acemoglu, Johnson, Robinson, and Yared (AER 2008)

17. Effect of beer taxes on fatality rates? Source: Ruhm, Christopher (J Health Economics, 1996)

18. Determinants of Long-Run Economic Growth • Recommended reading • NY Review of Books book review by Diamond is here • Acemoglu & Robinson blog post: What does Geography Explain? and What really happened in Neolithic Revolution?

19. Effect of Institutions on Long-Run Growth? Source: Acemoglu, Johnson, & Robinson (2005) “Institutions as the Fundamental Cause of Long Run Growth” (Handbook of Economic Growth, Aghion & Durlauf, editors). Figure from NBER WP 10481version.

20. Effect of Latitude on Long-Run Growth? Source: Acemoglu, Johnson, & Robinson (2005) “Institutions as the Fundamental Cause of Long Run Growth” (Handbook of Economic Growth, Aghion & Durlauf, editors). Figure from NBER WP 10481version.

21. Brief Introduction to Research Design Design Notation Internal Validity Experimental Design

22. Design Notation • Observations or measures are indicated with an “O” • Treatments or programs with an “X” • Groups are shown by the number of rows • Assignment to group is by “R,N,C” • Random assignment to groups • Nonequivalent assignment to groups • Cutoff assignment to groups • Time

23. There are two lines, one for each group. Vertical alignment of Os shows that pretest and posttest are measured at same time. X is the treatment. Subscripts indicate subsets of measures. Os indicate different waves of measurement. Design Notation Example R O1,2 X O1,2 R O1,2 O1,2 R indicates the groups are randomly assigned.

24. Yes Randomized (true experiment) No Nonexperiment Types of Designs Random assignment? No Control group or multiple measures? Yes Quasi-experiment

25. Non-Experimental Designs X O Post-test only (case study) O X O Single-group, pre-test, post-test X O O Two-group, post-test only (static group comparison)

26. Experimental Designs • Pretest-Posttest Randomized Experiment Design • If continuous measures, use t-test • If categorical outcome, use chi-squared test • Posttest only Randomized Experiment Design • Less common due to lack of pretest • Probabilistic equivalence between groups

27. Experimental Designs Solomon Four-Group Design • Advantages • Information is available on the effect of treatment (independent variable), the effect of pretesting alone, possible interaction of pretesting & treatment, and the effectiveness of randomization • Disadvantages • Costly and more complex to implement

28. Establishing Cause and Effect Single-Group Threats Multiple-Group Threats “Social” Interaction Threats Internal Validity • Internal validity is the approximate truth about inferences regarding cause-effect relationships.

29. Threats to Internal Validity History Maturation Testing Instrumentation Mortality Regression to the mean Selection Selection-history Selection- maturation Selection- testing Selection- instrumentation Selection- mortality* Selection- regression Diffusion or imitation* Compensatory equalization* Compensatory rivalry* Resentful demoralization* R X O R O Single-Group Multiple-Group Social Interaction

30. Single-Group Threatsto Internal Validity

31. Administer program Measure outcomes X O Administer program Measure outcomes X O What is a “single-group” threat? Two designs: Post-test only a single group Measure baseline O

32. Example • Diabetes educational program for newly diagnosed adolescents in a clinic • Pre-post, single group design • Measures (O) are paper-pencil, standardized tests of diabetes knowledge (e.g. disease characteristics, management strategies)

33. Pretest Program Posttest O X O History Threat • Any other event that occurs between pretest and posttest • For example, adolescents learn about diabetes by watching The Health Channel

34. Pretest Program Posttest O X O Maturation Threat • Normal growth between pretest and posttest. • They would have learned these concepts anyway, even without program.

35. Pretest Program Posttest O X O Testing Threat • The effect on the posttest of taking the pretest • May have “primed” the kids or they may have learned from the test, not the program • Can only occur in a pre-post design

36. Pretest Program Posttest O X O Instrumentation Threat • Any change in the test from pretest and posttest • So outcome changes could be due to different forms of the test, not due to program • May do this to control for “testing” threat, but may introduce “instrumentation” threat

37. Pretest Program Posttest O X O Mortality Threat • Nonrandom dropout between pretest and posttest • For example, kids “challenged” out of program by parents or clinicians • Attrition

38. Pretest Program Posttest O X O Regression Threat • Group is a nonrandom subgroup of population. • For example, mostly low literacy kids will appear to improve because of regression to the mean. • Example: height

39. Regression to the Mean pre-test scores ~ N When you select a sample from the low end of a distribution ... Selected group’s mean Overall mean the group will do better on a subsequent measure. post-test scores ~ N & assuming no effect of treatment pgm The group mean on the first measure appears to “regress toward the mean” of the population. Overall mean Regression to the mean

40. Regression to the Mean

41. Sir Francis Galton (1822 – 1911) 903 adult children & their 250 parents Regression to the Mean

43. Regression to the Mean • How to Reduce the effects of RTM (Barnett, et al., International Journal of Epidemiology, 2005) • When designing the study, randomly assign subjects to treatment and control (placebo) groups. Then effects of RTM on responses should be same across groups. • Select subjects based on multiple measurements • RTM increases with larger variance (see graphs) so subjects can be selected using the average of 2 or more baseline measurements.

44. Multiple-Group Threats to Internal Validity

45. The Central Issue • When you move from single to multiple group research the big concern is whether the groups are comparable. • Usually this has to do with how you assign units (e.g., persons) to the groups (or select them into groups). • We call this issue selection or selection bias.

46. O X O O O The Multiple Group Case Alternative explanations Measure baseline Administer program Measure outcomes Do not administer program Measure baseline Measure outcomes Alternative explanations

47. Example • Diabetes education for adolescents • Pre-post comparison group design • Measures (O) are standardized tests of diabetes knowledge

48. O X O O O Selection-History Threat • Any other event that occurs between pretest and posttest that the groups experience differently. • For example, kids in one group pick up more diabetes concepts because they watch a special show on Oprah related to diabetes.

49. O X O O O Selection-Maturation Threat • Differential rates of normal growth between pretest and posttest for the groups. • They are learning at different rates, even without program.

50. O X O O O Selection-Testing Threat • Differential effect on the posttest of taking the pretest. • The test may have “primed” the kids differently in each group or they may have learned differentially from the test, not the program.