The Data Consultant as Archaeologist

1. The Data Consultant as Archaeologist Digging for Meaning in World War II Era U.S. Public Opinion Surveys

2. The Project A historian of the twentieth century United States wants to incorporate public opinion survey data into research on the growth of the national state in the eras of the Great Depression and Second World War

3. The Sources • Largely unmined U.S. public opinion polls from the 1930s and ’40s • The pioneering age of scientific survey research • Survey takers included Gallup (AIPO), NORC, OPOR (Hadley Cantril), Roper

4. The Documentation • Quality varied, but at its best far short of modern metadata standards • Typically scans of questionnaires with handwritten notes • Some typed or handwritten code lists for response options not directly listed on the questionnaire; missing for some variables • Other random bits of documentation, such as frequencies, interviewer instructions

5. Gallup example

6. Code List for Question with Many Possible Responses

7. Case 1: Economic Class “N”? Roper/Fortune Survey, January 1939: (Unknown variable) Roper/Fortune Survey, January 1940:

8. Confirming N=“Negro” • Professor’s “informed guess” was that N represented “Negro”, i.e., black, respondents • Strategy for confirming was to compare code Ns with remainder of sample (codes A-D, presumably white), on documented variables known to differ sharply by race in 1939

9. Test 1: Occupation

10. Test 2: Feelings Toward Roosevelt

11. Test 3: Voter Turnout

12. Case 2: What Church? • 1941 OPOR survey • Sample: Residents of Pittsburgh, Pennsylvania • One question of interest: What church do you belong to, i.e, What is your religion? • Answers had been coded alphanumerically, but what church corresponded to which code was on documentation that had been lost

13. The documentation

14. The data

15. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

17. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

19. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

21. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

23. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

25. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

27. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

29. -> tab q13b (If "yes" on a, ask: Which church?) | Freq. Percent Cum. ------------+----------------------------------- | 155 12.88 12.88 & | 8 0.67 13.55 0 | 141 11.72 25.27 1 | 669 55.61 80.88 2 | 20 1.66 82.54 3 | 29 2.41 84.95 4 | 87 7.23 92.19 5 | 27 2.24 94.43 6 | 49 4.07 98.50 7 | 9 0.75 99.25 8 | 1 0.08 99.33 9 | 8 0.67 100.00 ------------+----------------------------------- Total | 1,203 100.00

30. Final Check

31. Confirmation! • Codebook for 1943 OPOR poll

32. Couple of other examples • Both from earliest Gallup surveys • Case 3: Used presence or absence of a city identification code on undocumented “city” variable to divide “Urban” category on “Rural-Urban” variable into “Big city” and “Small city” • Case 4: Used farm residence, gender, and age to split useless “Other and none” category on occupation variable into farmer, housewife, student, and retired

33. Avoiding Unintelligent Appearance Bias (Roper, 1946)