Assessing Sustainability Literacy

1. Adam Zwickle Tomas Koontz Kristina Slagle Assessing Sustainability Literacy An Instrument to Measure Knowledge of Environmental, Economic, and Social Sustainability

2. Overview • Need for a sustainability literacy measure • Developing an assessment • Results • Where we go from here

3. Need for a sustainability literacy measure • STARS • ER Credit 13: Sustainability Literacy Assessment “This credit recognizes institutions that are assessing the sustainability literacy of their students. Such an assessment helps institutions evaluate the success of their sustainability education initiatives and develop insight into how these initiatives could be improved.” • Criteria “Institution conducts an assessment of the sustainability literacy of its students. The sustainability literacy assessment focuses on knowledge of sustainability topics, not values or beliefs.”

4. Need for a sustainability literacy measure • Lacking in the scientific community • “Sustainability” has been defined and measured a myriad of ways • Often focused merely on the environmental aspect • Immensely applicable • Can test for correlations with various values, attitudes, and behaviors • Can serve as an explanatory variable

5. Developing an assessment • Built upon the “triple bottom line”, the “three legged stool”, the “3 p’s” • Environmental (planet) • Economic (prosperity) • Social (people) • Basic knowledge from each knowledge domain

6. Developing an assessment • Replicated questions used in the past • Coyle, 2005. “Environmental Literacy in America.” • Solicited topics and questions from experts • Held expert focus groups • Pilot tested among professors, graduate, and undergraduate students • Narrowed down to 30 questions

7. Developing an assessment • Administered 30 question assessment • Through an online survey • Sent via email • Sent to over 10,000 currently enrolled undergraduates • 19.3% response rate • Non-response follow up

8. Developing an assessment • Demographics • OSU – average demographics • This study - Slight trend towards more liberal and environmentalist, but differences were non-significant • Non-respondants • More likely to get a subset of the sustainability knowledge questions wrong…(small difference, but significant) • No differences in pro-environmental behaviors or GPA

9. Developing an assessment • Therefore, those choosing to take the survey were more knowledgeable about sustainability issues to begin with • These findings represent an ‘upper bound’ of the student population’s knowledge

10. Developing an assessment • Needed to reduce the length • Utilized Item Response Theory (IRT) • To reduce down to 16 questions

11. And aside on IRT… • Developed in the educational testing field • Used for standardized tests (GRE) • Focuses on the likelihood of answering something correctly… • At a given level of the construct of interest

12. And aside on IRT… Correct Theta - The latent construct that is being measured Information 50/50 threshold Probability of answering question correctly Standard deviations Incorrect

13. And aside on IRT… Correct “Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?” Information 50/50 threshold Incorrect

14. And aside on IRT… Incorrect A question that was removed… 50/50 threshold Probability of answering question correctly Standard deviations Information Correct

15. And aside on IRT… Correct “Approximately how much of the oil used in the U.S. in 2010 was imported from other countries?” 50/50 threshold Very low information Incorrect

16. Questions - Environmental • What is the most common cause of pollution of streams and rivers? • Ozone forms a protective layer in the earth's upper atmosphere. What does ozone protect us from? • What is the name of the primary federal agency that oversees environmental regulation? • What is the primary benefit of wetlands? • Which of the following is an example of sustainable forest management? • In the U.S., what do we currently do with the nuclear waste generated by nuclear power plants?

17. Questions - Social • Which of the following is the most commonly used definition of sustainable development? • The wealthiest 20% of people in the U.S. own approximately what percent of the nation’s privately held wealth? • Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans? • Higher levels of education generally lead to... • Which of the following populations has the highest rate of growth?

18. Questions - Economic • Many economists argue that electricity prices in the U.S. are too low because… • Which of the following countries has now passed the U.S. as the biggest emitter of the greenhouse gas carbon? • Which of the following is a leading cause of depletion of fish stocks in the Atlantic Ocean? • Which of the following is the most commonly used definition of economic sustainability? • Which of the following is the primary reason that gasoline prices have risen over the last several decades in the U.S.?

19. Results • Environmental 6 questions Mean: 4.39/6 SD: 1.48 73% • Social 5 questions Mean: 3.55/5 SD: 1.23 71% • Economic 5 questions Mean: 3.03/5 SD: 1.27 61% • Total 16 questions Mean: 11.08/16 SD: 3.21 69%

20. Example Analysis ANOVA F(3,1330)=8.09, p<.001

21. Example Analysis • Compare sub score by major • Economic score • Economics: 3.82 • Aerospace engineering: 4.00 • Animal science: 3.15 • Finance: 3.02 • English: 2.80 • Accounting: 2.63

22. Moving Forward • Refine • Weaknesses • Some answers change over time • Give us your suggestions! • Publish • Apply (put path model here?)

23. Acknowledgements • Funded by • The Ohio State Office of Sustainability • http://sustainability.osu.edu/ • OSU’s School of Environment & Natural Resources • http://senr.osu.edu/ • The many faculty and graduate students who contributed their time and expertise

24. Questions? • Environment and Social Sustainability Lab • www.ess.osu.edu • Contains: • This presentation • The knowledge questions with multiple choice answers • Lab email • essl@osu.edu