Managing Participant Satisfaction

1. Managing Participant Satisfaction Elisabeth Lang Intangible Asset Manager, Senior Consultant, CFI Group +46-8-562 800 00, Box 70373,107 24 Stockholm, Sweden Thursday October 9th, 2008

2. Content The point of satisfaction The CFI Group method Describing IFL from the participants’ perspective Analyzing results Identifying focus areas How to leverage the information to achieve IFL’s goals The IFL perspective • Feel free to interrupt with questions whenever you like • NB! Results have been altered, data is not real

3. Is there a point to focusing on quality? Example • Increased satisfaction, loyalty and willingness to recommend • Increased participation in alumni activities, strengthening the alumni network • Stronger brand • Higher revenue

4. Source: The Satisfied Customer,Claes Fornell What’s the effect of satisfied customers? • The cost of retaining customers is much smaller than the cost of acquiring new ones • Satisfied customers are much more likely to consume more and again, and are also much more likely to recommend the company, generating new business at a lower cost

5. S&P 500 ACSI Fund Satisfaction drives profitability and shareholder value! ACSI Fund vs. S&P 500: April 2000 (start) – May 26th, 2008 200% 150% 100% Cumulative Performance 50% 0% -50% -100% Source: S&P 500 from msn.com

6. The reasons why IFL focuses on quality and satisfaction • Identify relevant areas from the customers’, i.e. participants’, perspective • Identify the present situation, i.e. the participants’ assessment of IFL as a supplier. This is achieved by: • Asking questions in relevant areas • Calculating the scores for the individual questions that comprise the areas, as well as for the areas as a whole • Identify key areas that affect participant satisfaction and desired behaviors. This is achieved by: • Statistical analyses, e.g. factor analyses and structural equations modeling (PLS regression) • Combining scores and impacts derived from the statistical analysis • Initiate actions for improvement • Maintain current position on local and international market • Improve position on local and international market

7. Content The point of satisfaction The CFI Group method Describing IFL from the participants’ perspective Analyzing results Identifying focus areas How to leverage the information to achieve IFL’s goals The IFL perspective

8. Cause and effect The method captures cause and effect relationships, and identifies the driving factors Example: Perception Gut feeling Action Faculty? Satisfied? Recommend?

9. Information Applicationprocedure Facilities &accommodation Program director & program admin. Investment Faculty's inspiration Activities Faculty’s pedagogy PSI Personal development Faculty's competence Loyalty Literature & materials Recommend Interaction Participant group Program structure Effects The model is a conceptual version of the participants’ relationship with IFL • The model includes relevant questions and performance areas (e.g. Image) that describe the participants’ opinion on IFL • Each performance area consists of one or more questions • The questions are weighted together to form the performance area scores, thus the results are not percentages • PSI consists of three separate questions, also weighted together • If a respondent answers less than 2/3 of the questionnaire, the answer is not included in the analysis • If a respondent leaves a question unanswered, it is left blank Example

10. A value is calculated for each performance area on both the left hand side… Example

11. …and the right hand side of the model Example

12. Each performance area consists of one or more questions, weighted together to minimize the error term PSI Literature &Materials Example: • Litterature & materials consists of four questions, for example: • The format of the material that was distributed (compendia, hand-outs etc.) was user friendly • The literature and material used was up-to-date • The literature and material used complimented the teaching process • The literature and material used will be helpful to me in my work • PSI (Participant Satisfaction Index) is not an average score of all other questions, PSI is an index based on three separate questions: • How satisfied are you with the program? • To what extent has the program met your expectations? • Imagine a program which is perfect in every sense. How satisfied are you with the program compared to this ideal program? • It is necessary to have a separate PSI to be able to do the cause and effect analysis Literature &Materials PSI

13. The questions are answered on a scale from 1-10, in the analysis, the answers are converted to a score on a scale from 0-100 • In the questionnaire, the questions are answered on a scale from 1-10 • “Don’t know” is an option • To make it easier to interpret results and detect differences between segments and/or changes between years, the answers are converted to a score on a scale from 0-100 • If the respondent answers “7” on a question, this becomes a score of “67” • Please note that this is a score, not a percentage • If a respondent answers less than 2/3 of the questionnaire, the answer is not included in the analysis • 1 0 • 11 • 22 • 33 • 44 • 56 • 67 • 78 • 89 • 100

14. Multiple item scale, optimal weights Multiple item scale, equal weights Single item, ten point scale POWER: Ability to detect change Single item, five point scale Top Box approach: % 4 and 5 Prediction Error interval PRECISION: Width of Score Confidence Interval Scores can be more or less precise, depending on the method used • Using optimal weightsall the available information is used, including the relationship to all other questions and performance areas • Calculating a performance area value based on several questions increases the reliability of the value, decreasing the width of the confidence interval • A scale with ten points allows for more response alternatives and increases the possibility of detecting actual changes • In a “top box” approach only a small part of the information is put to use (typically responses 4 and 5)

15. Content The point of satisfaction The CFI Group method Describing IFL from the participants’ perspective Analyzing results Identifying focus areas How to leverage the information to achieve IFL’s goals The IFL perspective

16. External benchmark indicates performance compared to other industries, but internal benchmark will prove most useful in day-to-day management Example * Source: SKI 2007 * Full time MBA

17. PSI There is no significant difference in PSI scores between age groups Example PSI score

18. PSI There is only a slight difference in PSI scores between genders Example

19. PSI The private sector is significantly more satisfied Example

20. Identify programs with abnormally low or irregular scores, rather than just focusing on low scores Example • Ask yourself: • Why is this program so much better/worse in these areas? • Is there some key learning that can be applied when comparing to other programs? (* Difference compared to IFL Overall indicated in parentheses)

21. 1 10 Natural score variations occur between types of questions • When analyzing score levels it is unwise to compare scores across performance areas • Performance scores should be compared with relevant benchmark instead

22. Compare PSI scores across programs and participant groups

23. Focus on important performance areas and compare to benchmark

24. Identify programs which deviate from the overall pattern • Performance areas will always have varying levels • Identify “normal” pattern • Identify programs that deviate from this pattern

25. New programs could have uneven performance – it is important to identify weaknesses quickly and take action!

26. Established programs that run smoothly should have relatively even scores – if not, what is the quality issue?

27. Find patterns and identify underlying reasons Example • Participants from the private sector give notably higher scores • What is the reason (background, prior knowledge, levels of expectation, quality issues on targeted courses)? • If found, can something be done to affect participants from the public sector in the same way?

28. Find changed patterns and identify underlying reasons Example • Why is the pattern broken for two performance areas? • Which steps need to be taken to prevent quality issues in the future? • What is the cost of implementing these steps – and what is the cost of refraining from implementation?

29. Content The point of satisfaction The CFI Group method Describing IFL from the participants’ perspective Analyzing results Identifying focus areas How to leverage the information to achieve IFL’s goals The IFL perspective

30. Information Applicationprocedure Facilities &accommodation Program director & program admin. Investment Faculty's inspiration Activities Faculty’s pedagogy PSI Personal development Faculty's competence Loyalty Literature & materials Recommend Interaction Participant group Program structure Effects Where should improvement efforts be focused? • Resources (e.g. time, money) are always limited • Improvements should be focused on areas that will have a higher impact on PSI and the desired behaviors • These areas are identified in the cause and effect analysis

31. How to identify the drivers of Satisfaction Illustration Low impact High impact Scores for PSI Scores for PSI +1,5 +0,2 +5 +5 Scores for Information Scores for Faculty’s competence • The graphs show the impact of Information and Faculty’s competence on Satisfaction • The slope of the line represents the impact • The steeper the line, the higher the impact on Satisfaction • In this example: • If the score for Information increases with five (5) units, the score for PSI will increase with 0,2 • If the score for Faculty’s competence increases with five (5) units, the score for PSI will increase with 1,5 • The impacts are calculated using PLS-analysis (an advanced type of regression analysis) using PSI as the dependent variable

32. n=2 n=20 n=100 n=250 To be able to rely on the results we need a significant amount of answers Examples • The analysis can in theory be carried out with a few answers, but… • The results would be very unreliable as an individual respondent would influence the results heavily • The results might be due to chance, rather than certainty • As the number of responses increases so does the stability of the results

33. Example comparing high impact vs. low impact performance areas

34. Working with the PSI results – prioritizing actions • To prioritize actions, focus on those areas that will have the greatest impact on PSI and recommendation • Performance areas with high impacts will affect PSI and recommendation more than performance areas with low impacts • It is easier to improve performance areas with lower scores • The Priority Matrix combines these two aspects: • Focus primarily on performance areas within the Improve-quadrant – these have higher impact and lower scores High 2 Score 1 Low Impact on PSI

35. Example of a priority matrix Example 4. Exploit/Save 2. Improve/Maintain High Application procedure Faculty's competence Score 3. Monitor 1. Improve Information Room and food Low Impact on PSI

36. Content The point of satisfaction The CFI Group method Describing IFL from the participants’ perspective Analyzing results Identifying focus areas How to leverage the information to achieve IFL’s goals The IFL perspective

37. Working with PSI results to achieve the goals PSI results Revision Planning: Internal coordination group Follow-up Get all co-workers involved Execution Action plans

38. How do other companies do it?Example from a real estate firm A strong vision dictates all actions Actions are value-based and firmly established on all levels Strong endorsement from top management Humor Welcome to a better world (landlord)! Example

39. Consistency is key The vision to become Sweden's premier real estate company – and the conviction that this is achieved through customers – guides all actions The company wants to be number one… Employer Landlord Investment

40. Symbolic and systematic The customer is always in focus All processes have been… Analyzed Documented Implemented as a part of daily lives

41. An example taken from a telecom firm Example • Symbolism – King Customer! • Clear, attainable goals • Champions are appointed to each area • Everybody is involved in quality assurance and customer delivery (e.g. billing, complaint handling, technicians, sales personnel, PR, HR…) • Endorsed by management • Management actively ensures that the whole company sees King Customer as important

42. The ultimate goal is clear – and customer satisfaction is just a stepping stone More satisfied customers More paying customers The goal with ”Kung Kund” is a larger customer base and increased revenue!

43. Company culture focuses on the customers • Use of humor to focus on customers, e.g. using known sayings and proverbs, “adapted” to the situation • For Customer and country • The best with this company is the customer • Don’t wait until tomorrow what you can give to the customer today • A good customer makes your life longer • Priorities are set straight by management

44. Everybody influences customer satisfaction • HR: potential employees are also customers • Billing: how can we best meet customer needs? • PR: marketing influences customer expectations • Product development: pricing solutions have a great impact on customers

45. Specific actions and deliberate work leads to positive results • Structure is everything • Without proper organization results will not be achieved • SMART actions and goals • (Specific, Measurable, Attainable, Realistic, Timely) • Use technology! • E.g. a database to keep track of actions and that sends out automatic reminders

46. Low returns on investments in CSI High returns on investments in CSI Loyalty 100% 80% 60% CSI 40% 50 60 70 80 90 100 Working with the PSI results – setting goals and increasing scores Example • When results are low it is easier to increase them and they will have a higher effect on both CSI and Loyalty

47. Positive factors CSI level Score Hygiene factors Working with the PSI results – Positive and hygiene factors • The priority matrix consists of both positive factors of satisfaction as well as hygiene factors. Normally, the positive factors have a high impact on the participants’ satisfaction with IFL. The hygiene factors don’t. • However, hygiene factors have a greater impact on PSI when the scores are low – once the scores have reached the hygiene level the impact on PSI decreases. If IFL would underperform in an area that is considered hygiene it may start to become relatively more important for the overall satisfaction. • Positive factors are qualifiers – you need to maintain a certain level to even be considered. Once this level is achieved increases will lead to higher CSI scores.

48. Living, breathing and thinking PSI is a key factor • Set goals!E.g. today PSI is 81 – next year we aim for 84 • Include the co-workers in the creation of action plans • Use internal benchmarks and best practice – show success cases • Inspire through workshops and seminars • Integrate with internal systems and processes • Follow up!

49. Focus on key areas High Score Low Impact on PSI

50. Based on key areas, plan actions and set targets