New and enduring Topics and trends in economic education research

1. New and enduring Topics and trends in economic education research Michael Watts Purdue University 2012 Korean Economic Education Association International Conference, Seoul 2012 Council on Economic Education and National Association of Economic Educators Conference, Kansas City

2. Simple theses • The basic (“bread and butter”) set of questions asked in economic education researchover the past five decades has not changed and is not likely to change. • Theoretical models and insights featured in the literature are not dramatically different from what they were several decades ago, either. • Instead, most new insights and findings have been based on empirical research featuring various kinds of datasets, and improvements in statistical and econometric methods for analyzing data.

3. Future progress is also most likely to come from new and better data sets and improved statistical methods, • A key challenge for economic education researchers is to find and report results in ways that have meaningful practical significance, not just statistical significance. Doing that will attract more attention from other economists, educators, policy makers, and the general public. For example….. • In this presentation, I will touch only briefly on points 1- 4, and focus on 5.

4. I. Five basic and enduring research questions in economic education research Question 1: • How can we teach economics more effectively, especially in terms of student learning and interest in the subject, but also in terms of constraints that are inescapable across all fields of education – most notably tight budgets, limited classroom time for instruction, and offering course and instructional formats that provide more flexibility to students and school administrators?

5. Question2: • At the precollege level, how can we get more students to take economics and (not or) personal/financial education courses, and (not or) see more good economics instruction in other courses (especially history, civics/government, math, science/environmental studies, and languages/language arts)?

6. Question 3: • How can we get more students to major or minor in economics at the undergraduate level?

7. Question 4: • How can we better prepare Ph.D. students in economics to be better teachers, and provide appropriate incentives and opportunities for current professors to improve their teaching?

8. Question 5: • Can we teach economics in ways that make it more memorable and useful to people making decisions as consumers, workers, and citizens after they finish their schooling, and can we show the general public and policy makers in particular that economic and financial literacy really do affect “real world” decisions people make all • of the time?

9. This list of core questions strikes me as self-evidently enduring. But the answers to these questions can change over time and vary across countries, based on changes or differences in social, demographic, and technological factors, as well as public policies.

10. II. Trends and Enduring Patterns in Economic Education research • A. Data is Half (Maybe More) of the Battle • Key “Regular” Data Sources Have Been: • Single-school studies. • Nationally normed standardized tests in economics/economic education (TUCE, TEU, TEL, TEK, BET)

11. Important “Special/Irregular” Data Sources Have Been: • “The American Economy” television series and surveys (early 1960s) • Other educational films, audio-visual, and technology programs (late 1970s and 1980s) • The Pew Foundation program (late 1980s) • Advanced Placement (AP) exams in economics (1990s and continuing) • Major Field Tests (MFT) Also published by ETS. One study in 1999 by Walstad and Allgood. • National Assessment of Educational Progress (NAEP) in economics 2006 and 2012 • Other federal datasets U.S. National Center for Educational Statistics (NCES), including periodic High School Transcript Studies. Baccalaureate and Beyond. National Survey of Postsecondary Faculty (NSOPF). Release schedules vary, some regular others irregular.

12. Important “Special/Irregular” Data Sources -- Continued: • Statewide assessments Some in early 1980s, but most after “No Child Left Behind” reforms, and legislation and testing mandates of the early 1990s. • Surveys A few national surveys conducted by leading polling organizations such as NORC and Gallup, 1990s. Others commissioned or conducted directly by economists or economic educators on a state or occasionally national basis. Examples from all decades WWII. • Studies of long term effects 2003 survey of college graduates from four public universities (Florida Atlantic, Nebraska, North Carolina, Purdue) who took class in 1976, 1986 or 1996. • Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD) Funded by NSF. First economic education studies in 2011 and 2012.

13. II. Trends and Enduring Patterns in Economic Education Research • To help keep research in economic education on track (at least with shorter lags) research conferences and training programs and materials have been sponsored by such groups as the Joint (later National) Council on Economic Education, the Journal of Economic Education, and the American Economic Association Committee on Economic Education. • Early conferences were tied to providing publication opportunities for studies in the field, and to launching the JEE in 1969. Later programs have been directed more at providing training on new econometric methods. B. Advances in Econometric and Statistical Methods

14. Most extensive and expensive training programs at Princeton University, 1987 and 1988. Emphasis on discrete choice modeling and estimation of educational outcomes, using William Greene’s LIMDEP software and the large national data set from U.S. secondary students and teachers, described earlier.

15. In 2010-11, the AEA Committee on Economic Education and the Council on Economic Education sponsored the development of four online modules providing training and practice in the use of advanced econometric methods. Each module includes a “real” data set and sample programs using the data in three different statistical software packages: Stata, Limdep, and SASS. The modules are available on line on the Web page of the AEA Committee on Economic Education at • http://www.aeaweb.org/home/committees/AEACEE/Econometrics_ • Handbook/index.php.

16. Module 1: Data Management and Heteroskedasticity Issues • Module 2: Endogenous Regressors with Natural Experiments, Instrumental Variables, and Two-stage Estimators • Module 3: Panel Data • Module 4: Sample Selection Issues

17. In a more extensive review of research on precollege economic education (Watts 2005), the key findings I stressed were that elementary and secondary students can and do learn economics. They learn more when teachers have more training in economics/economic education, and when they use good instructional materials. • College students learn economics too, of course, although we have less to say about faculty characteristics and instructional materials and method that lead to more learning at this level – largely because there is much less variance in instructors’ training in economics and in the teaching methods they use. III. The (Selected past as Prologue: Say Something empirical Where Size (Not just Statistical significance) matters

18. I do not intend to minimize the importance of those findings, but… • These general results are neither surprising nor compelling enough to confirm current practice or to change the status quo. • Because they don’t really change people’s views they are more easily ignored, unless you believed economic education was important before you knew about the research. • To some extent, then, the research becomes a form of preaching to the choir.

19. Examples of an alternative way of framing and reporting findings from past and (I hope) future research follow. Not as many as I would like to have, nor are they as high a share of all economic education research as I would like to see. Findings are drawn from earlier research – some from very early work – I suspect many economic educators will not be familiar with at least some of the results, at least framed this way. Frankly, I fear I may well be missing other good examples of these kinds of results myself, simply because of the framing issues and not being able to break through past views and habits. Therefore, many examples come from my earlier work, because I am familiar with the results.

20. Bach and Saunders (1965) -- secondary teachers with more college economics coursework scored higher on a standardized test of basic economics, but effects were only significant if teachers had taken at least three courses, and not meaningfully sizeable unless they had taken five or more courses (which raised mean scores by about seven percent, after controlling for watching “The American Economy” TV series, self reported class standing, and other personal and professional characteristics). • A threshold effect at about four undergraduate courses on test scores collected years after graduation has resurfaced in some later studies, sometimes in a different context – for example, Lynch (1990) found that secondary teachers who had taken four or more course increased student scores on the TEL, while teachers with less economics coursework did not. • The finding that taking only one or two college courses in economics makes little long-term difference in understanding is also consistent with the still provocative Stigler hypothesis.

21. Later research on the lasting effects of college principles courses using the TUCE or other measures of knowledge pitched at a higher level than the TEU or TEL showed positive gains from taking college principles courses five years after graduation, in the range of about half the measured gains found when students were tested as they completed the principles courses (Saunders 1980). That represents a partial refutation of the Stigler hypothesis, but Saunders also found evidence that the lasting gains were concentrated more on application questions than on definitional, “recall and understanding” questions, which generally supports Stigler’s criticism of “encyclopedic” principles courses and textbooks that stress breadth and rote learning rather than depth and critical thinking based on economic analysis. A final point on the observed threshold effect for economic understanding at four or five courses is that this level of coursework basically translates to the group of college students who decide to major or minor in economics. Once again that raises questions about how much self-selection before graduation and self-interest after graduation drives these outcomes, rather than the course and discipline content per se.

22. In four national surveys of college and university economics instructors collected every five years, teaching methods in four different types of undergraduate courses (principles, intermediate theory, statistics and econometrics, and other upper-level field courses) were found to be remarkably consistent and persistent, even across different Carnegie classifications of schools. Using a non-linear Likert scale with five responses, median responses showed over 80% of instructors lecturing and writing on chalkboards (“chalk and talk”) in every class with little or no use of other teaching methods more popular in other disciplines, including cooperative and student-centered active learning methods. • Textbooks were required by over 80% of instructors in all types of courses. Calculus was not seen as important by a majority of respondents in any of the four types of courses, although the perceived importance seems to be rising slowly over time, particularly in intermediate theory and statistics and econometrics courses. Graphs are viewed as extremely important in introductory and intermediate theory courses and as somewhat important in the other two types of courses – despite some evidence that graphs do not help students learn more economics.

23. This summary picture of economics classes matches closely with what students who took economics courses in the mid-1970s, 1980s, and 1990s say they remember about the courses.I.e, the economics courses featured more straight lecture and graphs, with less discussion or use of other teaching methods than other courses. • The survey results on basic and narrow teaching methods is one of the exceptional itemsthat has been widely cited by researchers in economic education, perhaps illustrating the attraction of reporting direct empirical “size” results.

24. Harter, Becker, and Watts (2011) report that national samples of U.S. economists surveyed in 1995, 2000, and 2005 reported spending a little more than half their time teaching, a little over 20% of their time doing research, about 9% doing service, and presumably the rest of their time on administration, consulting, and other activities. This is very much in line with the NSOPF surveys of faculty members from all disciplines. • The economists also reported that reward structures in their departments put more weight on research than their time allocations reflected. Male faculty were more likely to devote slightly more time to research than female faculty – again consistent with NSOPF data for all disciplines – and to be employed at schools that put higher premiums on research.

25. In the general education literature we see that different kinds of teaching assessments – e.g., current student assessments, assessments by former students, faculty self-assessments, peer assessments, and assessments by external reviewers from outside a faculty member’s own discipline – are only weakly correlated. Differences in one measure explain half or often considerably less of the variation in other kinds of assessments. • For a large national sample of U.S. economists, Bosshardt and Watts (2001) confirmed that correlations between student and faculty self assessments of teaching are low, and on most individual assessment items very low (below 0.35). • Despite that, in surveys of economics department chairs at U.S. colleges and universities, Becker and Watts (1999) and Becker, Bosshardt, and Watts (2012) find that student evaluations of teaching are universally and almost exclusively used as measures of teaching effectiveness. Virtually no change in these practices across the 13 years between the two surveys of department Chairs, except for greater use of online collections of student evaluations.

26. The key question this all raises is whether schools and departments rely exclusively on SET measures because they believe these are the best, most important, or most reliable measures, or because they are inexpensive to collect, or because they provide the appearance of caring about students and teaching while allowing departments and schools enough leeway to interpret results to reinforce assessments based mainly on research or other performance measures.

27. In three papers dealing with economics learning and teaching at the elementary, middle school, secondary, and undergraduate levels (Bosshardt and Watts 1990 and 1994, and Watts and Bosshardt 1991),at the precollege level student characteristics (including general measures of ability or academic performance, gender, etc.) explained about three to five times as much of the variance in student scores on standardized tests as differences in instructor and school characteristics. Across a sample of micro, macro, and “hybrid” micro/macro principles classes at Purdue University, taught by doctoral students (except in the hybrid class, where the doctoral students led recitation sections once a week after students attended large lectures taught by a faculty member twice a week), variance in measures of instructor effects were about 46% as large as measures of student variance in the macro classes, about 11% as large in the micro class, and only 5% as large in the hybrid class. Consistently, across all of these grade levels and course formats, student differences appear to be more important than instructor or school differences; but the instructor effects are large enough to have important effects on student performance.

28. Schaur, Watts, and Becker (2012) investigated school, department, and faculty variables associated with instructors’ choices about what kinds of assessment methods to use in different kinds of undergraduate economics courses. Higher faculty teaching loads and larger classes drive out the use of essay questions, term papers (except in principles classes, where term papers are not seen as appropriate by most instructors) and other kinds of writing assignments.

29. For example, as class size increases from 40 to 100 students in principles classes, the probability of an instructor using multiple choice questions rises from 76 to 88 percent and the probability of using essay questions falls from 54 to 42 percent. • In upper-level field courses, as class size increases from 25 to 40 to 75 students, the probability of using term papers fell from 67 to 61 and then 47 percent; in econometrics and statistics classes the probabilities fell from 39 to 31 and 16 percent. Decreases in the use of essay questions for the same increases in class size were: in intermediate theory courses, from 77 to 71 to 56 percent; in other upper-division field courses, 84, 80, and 71 percent; and in econometrics and statistics courses, 47, 39, and 24 percent. The rise in the use of multiple choice questions for the same class size increases were: in intermediate theory courses, 34, 41, and 54 percent; in other upper-division field courses, 31, 47, and 80 percent; and in econometrics and statistics classes, 12, 21, and 47 percent.

30. In the long-term effects transcript and survey study described in the earlier section on databases, Allgood et al. (2004, 2010, and 2011) found that, compared to a random sample of non-business majors, economics majors had a 17% higher chance of earning more than $40,000 a year and a 14% greater chance of earning more than $100,000 a year. (The survey data using these values was collected in 2003.) They also had an 8% greater chance of having been self-employed at some point in their careers, and were 20% more likely to be paid at least partly on a commission/bonus basis. Economics majors were 4% more likely to own a home – probably because of higher income. But even after controlling for income those who took some economics coursework as undergraduates were less likely to have home equity of less than $50,000. They also saved 57% more, were more likely to invest in stocks, held about a third fewer credit cards, and were more likely to pay the full balance on credit cards each month. The economics majors were 8% less likely than the general majors sample to have completed an advanced degree. Differences between economics majors and business majors were generally not significant, except that business majors were about 7% more likely than economics majors to have been laid off at least once and were even less likely to have completed a graduate degree.

31. In a survey of a representative sample of U.S. adults, Blinder and Krueger (2004) found that over 60 percent of respondents reported they “regularly or often” got information about economic policies from television, with 49% checking newspapers, 35% checking both “friends and relatives” and “political leaders,” and only 17% checking “economists.” They also found that “ideology seems to play a stronger role in shaping opinion on economic policy issues than either self-interest or knowledge,” and noted that was “not terribly different” (p. 386) from conclusions offered by Fuchs, Krueger, and Poterba (1998) in a survey of professional economists.

32. Using survey data from secondary social studies and economics teachers, economists, and journalists, and comparing responses across the groups using rank-order correlations, Becker, Walstad, and Watts (1994) found that high school economics teachers held ideas closer to responses by economists than other social studies teachers, but even the economics teachers’ responses were closer to those of journalists than those from economists. The relationship between more economics coursework and reporting similar responses to economists on questions about public policies was supported for broader groups of college graduates in Allgood et al. (2010). Still, the key role and influence of information sources about policy issues raises serious questions about limits on the effects of economic education and coursework in this area.

33. IV. Conclusion • Despite the space setting out the importance of basic questions, data sets, and advances in econometric and statistical techniques, the call for more stress on developing and reporting results that have practically important quantitative results is the main argument here in what we do and how we do it. • Perhaps other economic educators will think it is not a major change, or see it as a mistake to move in that direction. If so, still might be a useful conversation to have. • The key point is that we should do something, or at least do more of what we do well, to get more economists, educators, and policy makers interested in and using research findings from economic education.

34. Many of the factors affecting answers to the bread and butter research questions in economic education are quite general. Others are more limited, in some cases affecting only the supply or demand for economics coursework and majors. • Any factors that can change the supply or demand for economic education represent a source of demand for new and updated studies in the field. • To paraphrase Keynes, then, there are no settled policy conclusions in economics or in economic education research because they are both inherently contextual – tied to current problems and institutional arrangements as well as to changes in production methods and consumer preferences. As these “background” features change so too may some of the answers to the enduring questions addressed by researchers in the fields.

35. Every new generation will want some education and training in economics given their roles as consumers, producers, and citizens, while exactly what that education and training entails, and when and where it is best delivered, will be changing rather than static. To anticipate and recognize those changes, and to assess the effectiveness of responses to them, will require new generations of research (and researchers) in economic education.

36. Economic education may remain at relatively small and arguable underrepresented field in both economics and education departments, but I believe if we can attract some of the best and brightest economists and educators to join in that research.

37. appendix • Examples of factors that have changed answers to some of the basic questions include: • 1) the rapid rise of acceptance of experimental economics as an alternative empirical methodology, and of behavioral economics as a systematically different way to frame and structure a wide range of research questions – especially those involving survey responses from the general public.

38. 2) greater access and reliance by students, businesses, and almost everyone else on the Internet and other new information, communications, and entertainment technologies; or

39. A wide range of recent public policies initiatives, many directed at education reforms. For example: • Over the past decade U.S. political and education institutions have devoted uncounted staff time and other resources writing out performance measurement standards to promote (or at least provide the appearance of promoting) accountability across all levels of education. • Other programs and budget rules or guidelines have been established to enhance affordability and “on time” graduation rates for college students.