Impact Evaluation of Brazil’s School Based Financial Education Program

1. Impact Evaluation of Brazil’s School Based Financial Education Program Preliminary results  June 6, 2011 Arianna legovini Miriam Bruhn Bilal Zia World Bank

2. Impact Evaluation Objectives (Students) • Measure the impact of a three semester long program that teaches high school students financial concepts • Does the program improve students’ financial knowledge? • The program also includes exercises and case studies related to the students’ daily life (e.g. making a budget) • Does the program change attitudes about financial decisions, as well as actual financial behavior and decision making?

3. Impact Evaluation Objectives (Parents) • Some exercises are meant to be done at home, jointly with the parents • Is this enough to also improve parents’ financial knowledge and to change their attitudes and behavior? • Can we achieve a greater impact by also providing a financial literacy workshop directly to parents?

4. Methodology • Pilot test the financial education program in close to 900 schools in various states in Brazil • Schools that were interested in participating in the pilot were randomly divided into two groups • Treatment group receives financial education text books for free AND receives teacher training for how to implement the material • Control group will receive the same two school years later (possibly with modified material and training, based on evaluation results) • Half of the parents in treatment schools will also randomly be selected for participation in a parent financial education workshop in schools

5. Pilot Sample Size Total of 891 schools in six states (439 treatment and 452 control schools) Approximately 26,000 students and parents (one class per school)

6. Timeline

7. Results Presented Today Use data from August 2010 baseline survey and November 2010 follow-up survey Students had received about 1 semester of financial education before the November 2010 follow-up survey Full program takes 3 semesters – the pilot ends in November 2011 Results presented today are preliminary since they only cover the first out of three semesters of financial education

8. Baseline Characteristics: Balance across Treatment and Control Group

9. Follow-up Data: Impact Analysis • At baseline, average student and parent characteristics, as well as test scores, were the same in the control and treatment group • This is because the treatment was randomly assigned • Therefore the control group is a valid comparison group for the treatment group • Thus, any differences in averages between treatment and control group at follow-up are CAUSED BY the financial education program

10. Students’ FinancialProficiency To measure students’ financial knowledge, the survey firm (CAEd) developed a test tailored to the program’s material Based on this test, CAEd calculated the level of financial proficiency for each student, ranging from 0 to 100

11. Impact on Financial Proficiency In the follow-up survey, the average level of financial proficiency was significantly higher in the treatment group (60.4) than in the control group (56.1)

12. Statistical Analysis of Impact on Financial Proficiency The statistical test of these differences shows that the financial education program increased the level of proficiency on the financial knowledge test by 3.6 points (compared to 56 points in the control group)

13. How large is the effect on financial proficiency? • The 3.6 point increase corresponds to a 6.4% increase compared to the average in the control group (in one semester) • According to CAEd, other school programs they have worked with in Brazil tend to improve test scores by 2.5% per school year • In Mexico, providing cash transfers to parents conditional on children’s school attendance improved test scores by 8-9% for 6-14 year olds (measured 6 years after the start of the transfers) • The effect size (0.25 of a standard deviation) is similar to the effect of • A program in Colombia that provided low-income children with vouchers for half the cost of private secondary school (0.2 std. dev. after 6 years) • A program in India that provided remedial education to students lagging behind in school (0.28 std. dev. after two years, the effect of the same program after one year was 0.14 std. dev)

14. Effect Size (est.)

15. Students’ Self-Reported Knowledge Ratings • The surveys also asked students to rate their knowledge about different financial concepts • On a scale from 0 to 10, how would you rate your knowledge about • Interest rates • Loans/financing • Insurance • Income tax • Minimum payment for credit cards

16. Impact on Students’ Self-Reported Knowledge Ratings (Follow-Up Data) At follow-up, average self-reported knowledge ratings for all these concepts were higher in the treatment group than in the control group

17. Two Questions Often Used in the Economics Literature to Measure Financial Literacy • 1. Suppose you are interested in taking a loan of R$50,000. Bank “A” offers you a loan of R$50,000 payable in one year with an annual interest rate of 15%. Bank “B” offers you a loan of the same amount, R$50,000, but you will have to pay $60,000 within one year. Which loan would you prefer? • Bank A • Bank B • Don’t know • 2. Suppose the interest rate on your savings account is 1% per year and inflation is 2% per year. After a year, if you had not made any deposits or withdrawals in this account, would you be able to buy more than, exactly the same as, or less than today with the money in this account? • More thantoday • Thesame as today • Lessthantoday • Don‘t know

18. Impact on Students’ Correct Answers to Financial Literacy Questions Used in Economics Literature (Follow-Up Data) At follow-up, a higher percentage of students in the treatment group than in the control group gave the correct answer to the financial literacy questions often used in the economics literature

19. Student Financial Autonomy • Autonomy index, designed by CAEd, based on 3 dimensions of Autonomy • Reflexive Autonomy e.g. “ I like to think carefully before deciding to buy something” • Emotional Autonomy e.g. “I am prepared to talk about money with my parents” • Functional Autonomy e.g. “I always try to save some money to do things I really like” • These three components were combined into one overall index of financial autonomy, where higher values indicate more financial autonomy

20. Impact on Student Financial Autonomy (Follow-up Data) At follow-up, students in the treatment group obtained higher scores on the autonomy index than students in the control group

21. Students’ Intention to Save • Intention to save index, designed by CAEd, based on 3 components • Attitudes towards behavior e.g. “In my opinion, saving money every month extremely beneficial • Subjective norms and expectations e.g. “My family has the habit of saving money every month” • Perception of capacity of controlling one’s behavior e.g. “I believe I can save a little money every month” • These three components were combined into one overall index of intention to save, where higher values indicate higher intention to save

22. Impact on Students’ Intention to Save (Follow-Up Data) At follow-up, students in the treatment group had a higher measured intention to save than students in the control group

23. Students’ Saving Behavior • In order to assess whether intention to save is also reflected in savings behavior, the survey asked students • Which fraction of their income they save • About 61% of students have non-zero income (from work or from parents) • About 37% of students work • Whether they save to be able to finance future projects

24. Impact on Students’ Saving Behavior (Follow-Up Data) • At follow-up, a higher percentage of students in the treatment group than in the control group • Saved at least some of their income • Saved to able to finance projects in the future

25. Students’ Spending Behavior • The survey asked a number of questions to assess students’ spending behavior • Do students make a list of expenses every month? • How do students pay for purchases? • Do they use a credit card? • Do they use a booklet/installment plan in the store?

26. Impact on Students’ Spending Behavior (Follow-Up Data) • At follow-up, a higher percentage of students in the treatment group than in the control group made a list of expenses every month • A lower percentage of students in the treatment group than in the control group • Often shopped with a credit card • Often shopped with a booklet/installment plan in the store

27. Other Measures of Financial Behavior • Apart from saving and spending behavior, the survey asked about several other features of students financial behavior • Did the student learn to manage money in school? • Does the student negotiate how to pay when making a purchase? • Does the student search for other brands and model when deciding which product to purchase (i.e. do the compare options?) • These are often thought to be savvy/smart financial behaviors

28. Impact on Students’ Financial Savvy (Follow-Up Data) • At follow-up, a higher percentage of students in the treatment group than in the control group • Say they learned to manage money in school • Negotiate how to pay • Search similar model/brands when deciding what to buy

29. Parent Outcomes • Parents have not received financial education directly • But the students’ financial education material includes exercises to be completed with parents (e.g. make a household budget) • This can have positive “spillovers” on parents’ financial knowledge • To measure parent’s financial knowledge, the parent survey included the same two financial literacy questions often used in the economics literature that were also included in the student survey

30. Impact on Parent Outcomes (Follow-Up Data) • At follow-up, a higher percentage of parents in the treatment group than in the control group • Say they talk to the student about finances • Say the student helps organize the household budget • But, there are only small differences in the percentage of parents who answer the financial literacy questions correctly

31. Statistical Analysis of Impact on Parent Outcomes • The program significantly increased the % of parents who • Say they talk to the student about finances (by 3.6% compared to 67% in the control group) • Say the student helps organize the household budget (by 4.9% compared to 48% in the control group) • So far, we see no statistically significant impact of the program on parents’ financial knowledge

32. Conclusions • The first semester of the high school financial education program increased students’ financial knowledge • The program also led to improvements in students’ financial attitudes • And, the program changed financial behavior – due to the program, students are more likely to • Save and manage their expenses • Engage in “smart” financial behaviors • Students are also more likely to talk to their parents about financial matters and to help with organizing the household budget • “Spillover” effects on parents’ financial knowledge are small so far  need for the parent workshop

33. NextSteps • Implement the parent workshop • Currently working on this • Have received updated student lists from close to 150 schools and have randomly allocated parents in these schools into two groups • One will watch financial literacy video • Other will watch health video • Still need to receive student lists from the remaining treatment schools • Start planning the November 2011 survey

34. World Bank Team & Contact Info • TTL • Rogelio Marchetti (LCSPF) rmarchetti@worldbank.org • Researchers • Miriam Bruhn (DECFP) mbruhn@worldbank.org • Bilal Zia (DECFP) bzia@worldbank.org • Field coordinator • Luciana Leão(LCSPF) lululeao@terra.com.br • Advisor • Arianna Legovini (DECOS) alegovini@worldbank.org