CE’s Data Driven Approach to Customer Satisfaction & Engagement

1. CE’s Data Driven Approach to Customer Satisfaction& Engagement Using Data to Drive Content

2. GOAL: Provide plain language content that anticipates customer needs and encourages them to self-serve. CE’s Data Driven Approach to Customer Satisfaction & Engagement

3. Sandra Vuong Digital Engagement Strategist Digital Engagement GroupAwareness & Outreach Sandra.Vuong@ed.gov

4. WebsiteData CE’s Data Driven Approach to Customer Satisfaction & Engagement

5. Referral Search Terms StudentAid.gov October 2014 Top Referral Search Terms • Student loan forgiveness • FAFSA • Public service loan forgiveness • Scholarships • Teacher loan forgiveness • Fafsa.ed.gov • Loan forgiveness • Teach grant • Fasfa • Studentaid.ed.gov The term people “Googled” to find your site. We use referral search terms to identify hot topics for the month. What are people wanting to learn about? CE’s Data Driven Approach to Customer Satisfaction & Engagement Source: Google Analytics

6. On-Site Search Terms StudentAid.gov October 2014 On-Site Search Terms • loan forgiveness • income based repayment application • ibr • parent plus loan application • public service loan forgiveness • parent plus loan • consolidation • pslf • exit counseling • ibr application form Once people get to our site, what terms are they searching? On-site search terms are another way to identify hot topics, but they can also be a good indicator of what users can’t find. CE’s Data Driven Approach to Customer Satisfaction & Engagement Source: DigitalGov Search

7. Top Pages Which pages on the site are the most visited? We use referral search terms to identify hot topics for the month. What are people wanting to learn about? CE’s Data Driven Approach to Customer Satisfaction & Engagement Source: Google Analytics

8. Customer ListeningData CE’s Data Driven Approach to Customer Satisfaction & Engagement

9. Customer Listening Social Media CE’s Data Driven Approach to Customer Satisfaction & Engagement

10. Customer Listening Studentaid.gov/repay StudentAid.gov Survey “To hard to figure out how to make a payment” “Information on how to actually make a payment is not easy to find.”

11. Customer Listening FSAIC Tweet and video clip (Q series)

12. Customer Listening Ombudsman CE’s Data Driven Approach to Customer Satisfaction & Engagement

13. MessageTesting CE’s Data Driven Approach to Customer Satisfaction & Engagement

14. Message Testing How does Facebook message testing work? Posts are “dark.” This means they do not appear in on our public page. They are only targeted to the newsfeeds of the audience we indicate. Facebook determines who gets which variation. In most cases, a person will only be served one variation, although it is possible that they are served multiple variations. Testing structure: Day one serves as our baseline. On day one, all variations are given equal budgets, targeted to the same audience. For the remaining days in the flight, we allow Facebook to auto-optimize and serve the variation(s) that are performing best, based on our KPIs. We do this to ensure an efficient campaign. CE’s Data Driven Approach to Customer Satisfaction & Engagement

15. Test 1: How to Reference IDR Paid Social Media – Message Tests Question: Do people understand/comprehend the term “Income-Driven Repayment”? Constant (message): Need to lower your student loan payment? Test Apply for an Income-DrivenRepayment Plan Apply for an Income-BasedRepayment Plan Switch your Repayment Plan CE’s Data Driven Approach to Customer Satisfaction & Engagement

16. Paid Social Media – Message Tests Test 1: How to Reference IDR A C B CE’s Data Driven Approach to Customer Satisfaction & Engagement *Top posts by cost per website visit

17. CE’s Data Driven Approach to Customer Satisfaction & Engagement

18. Paid Social Media – Message Tests Test 2: How to Promote IDR Question: What’s the best way to entice borrowers to considerincome-driven repayment? Constant (message): Compare repayment plans Test More than 3 million federal student loan borrowers are enrolled in repayment plans that base their monthly payments on their income. You could reduce your monthly student loan payment to as little as $0, depending on your income. You could reduce your student loan payment to as little as 10% of your discretionary income. CE’s Data Driven Approach to Customer Satisfaction & Engagement

19. Paid Social Media – Message Tests Test 2: How to Promote IDR C A B CE’s Data Driven Approach to Customer Satisfaction & Engagement *Top posts by cost per website visit

20. CE’s Data Driven Approach to Customer Satisfaction & Engagement

21. Paid Social Media – Message Tests Head-to-Head: Positive vs. Negative Framing vs. Question: When trying to promote IDR, is it better to promote the benefits of the program (positive), or the situation the borrower may be facing (negative)? Analysis 0.40% Link Click Rate $1.48 Cost per visit $0.55 Cost per visit 0.87% Link Click Rate CE’s Data Driven Approach to Customer Satisfaction & Engagement

22. Positive Negative 0.87% Link Click Rate 0.40% Link Click Rate CE’s Data Driven Approach to Customer Satisfaction & Engagement

23. Native Advertising – Headline Tests

24. Headline (changeable) Headline variations we test 24 CE’s Data Driven Approach to Customer Satisfaction & Engagement

25. Paid Media – Message Tests Results: Tips for the Future Consider using “income-based repayment” over “income-driven repayment” when messaging about the plans. When promoting the benefits of the income-driven repayment plans “$0 monthly payment” message point drives the strongest click-through rate, even over forgiveness. In looking at reasons for needing the programs, “Not making much money”, drove the strongest click-through rate. Promote the benefits of the income-driven repayment plans (i.e. low payments, forgiveness), instead of targeting borrowers by the life situation that may result in them needing the programs (i.e. not making much money). Use wording our customers use if you want them to understand you (i.e. “Obama Loan Forgiveness”) CE’s Data Driven Approach to Customer Satisfaction & Engagement

26. Jessica Barrett Simpson Acting Senior Advisor for the Borrower Experience & ISEJB.Simpson@ed.gov CE’s Data Driven Approach to Customer Satisfaction & Engagement

27. Email Campaigns: Effective Communications 3 Keys to Improving Customer Email Communication Create a control group Do subject line and A/B message testing Utilize behavioral insights CE’s Data Driven Approach to Customer Satisfaction & Engagement

28. Email Campaigns: Create a Control Group The email made a difference CE’s Data Driven Approach to Customer Satisfaction & Engagement

29. Email Campaigns: Test Subject Lines CE’s Data Driven Approach to Customer Satisfaction & Engagement

30. Email Campaigns: Do A/B Message Testing Email A Email B CE’s Data Driven Approach to Customer Satisfaction & Engagement

31. Email Campaigns: Utilize Behavioral Insights • FSA has partnered with the White House Social and Behavioral Sciences Team (SBST) to utilize academic research from the social and behavioral sciences about how people receive information and make decisions • URL: whitehouse.gov/sbst CE’s Data Driven Approach to Customer Satisfaction & Engagement

32. Email Campaigns: Utilize Behavioral Insights CE’s Data Driven Approach to Customer Satisfaction & Engagement

33. MarketResearch CE’s Data Driven Approach to Customer Satisfaction & Engagement

34. Recommended Resources • FSA Market Research (available upon request) • Customer Segmentation • Income-Driven Repayment Survey • Borrowers In School, Grace , Delinquency, and Default Surveys • Borrower Messaging Survey • New America’s College Decisions Survey: • Deciding to go to college, The Application Process, Familiarity with Financial Aid, Understanding Student Loan Debt, Searching for the Right College • Communicating in plain language: • PlainLanguage.gov • 18F Plain Language Content Guide • DigitalGov’s Plain Language Web Writing Tips • Google Trends: Identify hot topics, side-by-side wording comparisons • Customer Listening Reports • CE employees • FSA Employees (access upon request) CE’s Data Driven Approach to Customer Satisfaction & Engagement

35. Recommended Resources • Other Enterprise Services– Customer Experience provides support for: • Translation and Interpretation • Publication Layout and Design • Printing • Publication Distribution (Point of Contact : Tina Pemberton) • FSA Style Guides: • Brand Style Guide • Web Developer Guide • Writing Style Guide • Templates • Branding and design support are available for FSA business units (Point of Contact: Adam Essex, Tina Pemberton) • Customer Testing Support: • Customer Experience is establishing a contractual vehicle that business units can user for customer testing (e.g., usability, focus groups, surveys, research)

36. QUESTIONS? CE’s Data Driven Approach to Customer Satisfaction & Engagement