IAF Certification/ Registration Bodies’ Member Satisfaction Program

1. IAF Certification/ Registration Bodies’Member Satisfaction Program September 19, 2003 Final Report Summary

2. Agenda • Research Objectives • Executive Summary • ABs: Interacted and Satisfaction With • Accreditation Bodies’ Performance Analysis • Mutual Recognition (MLA)

3. Research Goals • Measure CRBs’ overall satisfaction with Accreditation Bodies. • Identify areas warranting improvement by Accreditation Bodies. • Examine the perceived value of MLA (Mutual Recognition)

4. Research Design • Web-based interviewing: • An email invitation is sent to CRBs which contains the URL where the web survey is administered. • Reminder emails are sent to non-respondents. • Respondent names and email addresses are provided by IAF members to Burke. • Survey is conducted in English, French, German, Spanish, Korean, Mandarin and Japanese.

5. Executive Summary • This 2003 program is the first comprehensive assessment of CRB perceptions worldwide. • Contact information was provided by over 40 Accreditation Bodies and the survey was conducted by the IAF. • Email invitations were sent by Burke with a message from the IAF asking for their participation. • In addition, two email reminders were sent.

6. Executive Summary • Key Findings • We sent out 704 invitations. Of these, 99 completed the survey. This yields a response rate of 14%, fairly typical for an online survey. • Those CRBs completing the survey worked with 31 of the 43 Accreditation Bodies listed. On average, CRBs have been accredited by 3.36 ABs. • We asked respondents to name the two primary ABs with whom they work and we then gathered information about these two organizations. • We gave respondents the opportunity to rate other ABs at the end of the survey.

7. Executive Summary • Key Findings (cont.) • Overall Satisfaction with ABs: • Two-thirds of respondents (64.8%) are at least somewhat satisfied with the Accreditation Bodies with whom they work. • However, only 13.7% were “very” satisfied. • Typically, we would expect 60 - 65% to be very satisfied. • The proportion of CRBs who express dissatisfaction (32.8%) is very high.

8. Executive Summary • Key Findings (cont.) • How to Improve AB Satisfaction Levels: • We have conducted over 1,500 similar programs. • Historically, organizations would focus on customer complaints to direct improvement efforts. • The problem with this approach is that these issues might not drive customer/ stakeholder satisfaction or purchase. • Organizations then moved (10-15 years ago) to key driver analysis. Identify those issues driving customer satisfaction and focus on those. • The problem with this approach is that an organization might already perform well on several of the key drivers and thus focusing on them is a waste of those resources.

9. Executive Summary • Key Findings (cont.) • How to Improve AB Satisfaction Levels: • We combine these approaches: we focus on key drivers of satisfaction but only on those where customers rate your performance low. We call these issues “target” issues. • Improvement on target issues represents an organization’s best chance of improving overall satisfaction. The target issues identified include: • Completes accreditation procedure in a timely manner • Responds quickly to requests for extensions of scope • Has open, clear communication with me • Treats me as a valued customer • Provides services that are a good value for the money

10. Executive Summary • Key Findings (cont.) • How to Improve AB Satisfaction Levels: • Several other issues should be monitored as they are nearly target issues themselves: • Has a witness process that adds value • Allows the CRBs to participate in the governance of the Accreditation Body • Has reasonable office audit fees

11. Executive Summary • Key Findings (cont.) • Mutual Recognition (MLA): • 58.6% of CRBs believe having one accreditation body recognized worldwide is very valuable and 31.3% feel it is somewhat valuable. • This differs across regions: • North (84.6%) and South America (100%) and Australia (78.6%) believe one accreditation body recognized worldwide is very valuable. • In Asia (29.4%), Europe (53.1%) and South Africa (50%), the value is seen by fewer CRBs. • Somewhat fewer (71.7%) saw the value (very or somewhat valuable) of an IAF accreditation mark recognized worldwide.

12. Executive Summary • Key Findings (cont.) • Mutual Recognition (MLA): • While 61% see the MLA as at least somewhat valuable, over a third (35%) do not see the value. • More CRBs (45%) are dissatisfied with the progress of the ABs in achieving mutual recognition using MLA than are satisfied (40%).

13. Executive Summary • Conclusions • It should be remembered that only 14% responded to our invitation to participate in the survey. They may, or may not, be representative of the entire CRB population. • This survey especially cannot be used to evaluate individual Accreditation Bodies as the base sizes per AB are just too small. • Among those who participated, the levels of dissatisfaction would be a matter of grave concern were this program sponsored by a commercial firm in a competitive market.

14. Accreditation Bodies Worked With and Evaluated

15. Q.1: Number of Accreditation Bodies By Whom They’ve Been Accredited Base: All Respondents: 99 Mean = 3.36

16. Q.3: Overall Satisfaction With Accreditation Bodies Base: Total Accreditation Bodies Rated: 131

17. Performance Attribute Analysis 1. What is important to our customers? Stated Importance Customer importance ratings indicate stated attribute importance Attribute Leverage Attributes that are found to be drivers of satisfaction via correlation 2. How are we doing? Attribute Performance CRBs' attribute ratings evaluating performance. 3. So what do we do now? Issue Prioritization Contrasting attribute leverage with attribute performance to identify improvement opportunities

18. Attribute Performance Please rate how important you feel each attribute is to you. Not At All Very Important Important 1 2 3 4 5 6 7 8 9 Then rate how satisfied you are with each Accreditation Body’s performance on that attribute. Very Very Dissatisfied Satisfied 1 2 3 4 5 6 7 8 9

19. Q.4: Attribute Leverage- Most Important Performance Issues - Base: Total Accreditation Bodies Rated: 131

20. Attribute Performance Deficiencies 2. How are we doing? • Attribute performance ratings show how Accreditation Bodies perform on each attribute in the eyes of its customers. • We provide the mean rating to give an overall assessment of AB performance. • The primary metric, however, is a “Deficiency” score which indicates the percentage of CRBs who give a 1-4 rating (on the 9-point scale) for their Accreditation Body’s performance. Attribute Performance CRB attribute ratings evaluating Accreditation Bodies’ performance. Showing proportion of customers getting less than they would like (dissatisfied).

21. Q.4: Accreditation Body Performance- Proportion of CRBs Giving a Bottom 4 Box (1 - 4) Rating –Issues With the Highest Levels of Dissatisfaction Base: Total Accreditation Bodies Rated: 131

22. Fewer Customers Perceiving Deficiency More Customers Perceiving Deficiency AB Performance Performance Issue Classification & Prioritization-The Targeting Matrix - Important Strengths Must maintain: continuous improvement candidates ABs are under-performing on these impactful attributes TARGET ISSUES Stronger Focus on these issues Attribute Impact on Satisfaction Work here last; check to see if “over-delivering” or if able to redirect resources Reduce deficiencies, but lower impact on satisfaction; less “bang for the buck” Less Critical Issues Less Important Strengths Weaker

23. Opportunity Identification 5 Completes accreditation procedure in a timely manner 10 Responds quickly to requests for extensions of scope TargetIssues Higher Important Strengths 19 Has open, clear communication with me 1 25 21 Treats me as a valued customer 11 22 Provides services that are a good value for the money 26 12 24 6 17 4 2 13 3 27 Has reasonable office audit fees 28 15 23 Attribute Impact On CRB Satisfaction 29 9 18 14 8 20 Allows the CRBs to participate in the governance of the Accreditation Body 7 16 Has a witness process that adds value Less Important Strengths Less Critical Issues Lower Lower Higher Proportion of CRBs Giving a 1-4 Rating Base: Total Accreditation Bodies Rated: 131

24. Mutual Recognition: MLA

25. Q.10a: Value of One Accreditation Body That Is Recognized Worldwide Base: All Respondents: 99

26. Q.10b: Value of an IAF Accreditation Mark That Is Used Worldwide Base: All Respondents: 99

27. Q.11: Value of Current IAF MLA (Mutual Recognition Arrangement) Base: All Respondents: 99

28. Q.12: Overall Satisfaction With the Progress of the Accreditation Bodies in Achieving Mutual Recognition Using the MLA Base: All Respondents: 99

29. Action steps to move forward

30. Action steps moving forward • Communicate the results to the GA • Establish team to address issues • Communicate the study results to all members • Communicate appreciation to the CRBs • Communicate the establishment of the team to the CRBs • Proactive Communiqué • Seek input from members • Identify corrective actions • Communicate – develop consensus • Execute • Resurvey