Michigan Quality Leadership Award New Examiner Training 2014

1. Michigan Quality Leadership Award New Examiner Training 2014

2. Agenda • Welcome and Introductions • Begin with the End in Mind • Code of Ethical Conduct • Key Factors • Six-Step Process • Evaluating Items • Preparing for March 12

3. Clever Introductions • Find a partner to introduce to the class. • Share name, affiliation, Baldrige-like experience. • Share another item of your choice, or complete this sentence: My zodiac sign is ______ and it will help me be an awesome Examiner because ……

4. Examiner Roles and Responsibilities for Today’s Learning • Take an active role in your own learning by asking questions, sharing information, viewpoints, and roles during the training program. • Take an active role in helping others to increase knowledge and develop skills.

6. Application Submission Judges’ Meeting Not selected Independent Review Consensus Review Feedback Report to Applicant Selected Site Visit Judges’ Meeting Not selected Judges Name Applicant an Award Recipient Feedback Report to Applicant Award Process Cycle Selected

7. Begin with the End in Mind • Feedback Report • Constructive, actionable feedback that helps applicant • Identify and leverage Strengths • Prioritize and address Opportunities for Improvement (OFIs)

8. Baldrige Criteria Framework: A Systems Perspective

9. Ensuring Examiner Ethical Behavior A key success factor for the Award program is promoting and ensuring ethical behavior. Code of Ethical Conduct Conflict of Interest

10. Code of Ethical Conduct Four Principles: • Protect the Integrity of the Award Process. • Exhibit Professional Conduct at All Times. • Protect the Promise of Confidentiality. • Protect the Program’s Intellectual Property.

11. Conflict of Interest Statement • Upon notification of application, immediately review for conflicts of interest. • Sign and email back the Conflict of Interest statement. • Call Geri Markley, Michigan Quality Council, with all questions.

12. Exercise: Code of Ethics In your small table groups: • Assign roles. • Discuss your assigned ethics scenario. • Prepare to share your consensus response.

13. Scenario 1: the answer is …… Examiners should not exploit their selection to the Board of Examiners. There are no “secrets” for receiving the award and no guarantees for receiving a site visit. Also, the program prefers to understate the “winning” aspect of the process; it encourages a focus on continued self-improvement and the sharing of best practices to help organizations achieve performance excellence. Examiners may not use the MQC logo in any advertising/promotion (and business cards may not include “MQLA examiner” or the MQC logo).   Principles: Protecting the integrity of the award process, the program’s intellectual property, and the reputation of the BOE.

14. Scenario 2: the answer is …… Any assistance with an organization’s application would result in a conflict of interest for an examiner. The examiner would be considered an employee of the organization. If the organization reapplies, the examiner would not be permitted to evaluate the application. Examiners advising or participating with an organization in preparing an award application shall not reveal or discuss that participation with other examiners during training or at any other time.  Principles: Protecting the promise of confidentiality and the integrity of the award process.

15. Scenario 3: the answer is …… No. At no time during Independent Review or Consensus Review should an examiner contact the applicant. Examiners shall not communicate with the applicant organization or in any manner seek additional documentation, information, or clarification. This includes Internet searches or other external sources of information. Examiners shall safeguard the confidences of all parties involved in the judging or examination of current or former applicants. Principles: Protecting the promise of confidentiality and the integrity of the award process.

16. Scenario 4: the answer is …… No. Examiners must avoid disclosures that may in any way influence the award integrity or process, currently or in the future, and treat as confidential all information about the applicant. While this information has been publicly released, it is not part of the application, and, therefore, it is not relevant to the evaluation. While potentially interesting, this information should not be discussed and must not be considered in the evaluation. The program shares with the team leader any known information that could affect selection as a role model, and the team leader seeks clarification during the site visit. Principle: protecting the integrity of the award process.

17. Scenario 5: the answer is …… Yes. Examiners may use the Internet and other sources to familiarize themselves with common or generic terminology. Examiners shall not communicate with the applicant organizations or in any manner seek additional documentation, information, or clarification specific to the applicant. This includes Internet searches or other external sources of information. Principle: protecting the promise of confidentiality.

18. Master List of Key Factors • “Facts” about the organization • Indicates relevance and importance • Not processes or evaluation of processes • Mainly from the Organizational Profile

19. Examiners View Applicants through Two Lenses: Criteria and Key Factors

20. Examples of Key Factors • Size a) 39 employees located in one building b) 73,000 employees located in twelve states • Strategic Challenges a) Heavily regulated and intensely competitive b) Rapid expansion from acquiring many small competitors • Products a) Highly technical and changes often b) Has not changed in 37 years

21. Key Factors • How are key factors used? • Why are they important? • Case Study Key Factors

22. Exercise: Key Factors In your small table groups: • Assign roles. • Identify key factors for your assigned section of the Organization Profile. • Prepare to share your key factors.

23. Six-Step Item Evaluation Process

24. Independent Review Steps • Ground yourself in the criteria. • Select the relevant 4-6 key factors for the item. • Read the relevant section of the application. • Draft around 6 strengths & OFIs. • Write two feedback-ready comments. • One strength • One OFI • Determine the scoring range and score for the item.

25. Step 1: Ground yourself in the Criteria item.

26. Step 1 – Ground Yourself in Criteria Requirements • Read the Criteria item to understand the Item requirements. • Look at 1.1a(1) – Senior Leadership (p.7) How do SENIOR LEADERS set your organization’s VISION and VALUES? How do SENIOR LEADERS DEPLOY the VISION and VALUES through your LEADERSHIP SYSTEM, to the WORKFORCE, to KEY suppliers and PARTNERS, and to CUSTOMERS and other STAKEHOLDERS, as appropriate? How do SENIOR LEADERS’ actions reflect a commitment to those VALUES?

27. Step 2: Select the relevant key factors for the item.

28. Step 2 –Select the Relevant Key Factors for the Item • Review master list of key factors; determine the attributes of the organization that would influence its responses to the item requirements. • Select the most relevant 4-6 key factors for that item. Example: 1.1a(1) would likely have a key factor related to the organization’s vision and values.

29. Step 3: Read the relevant section of the application.

30. Step 3 – Read the Relevant Section of the Application • Identify the processes or approach the applicant uses to meet item requirements. • Flag, mark up and/or take notes as needed. • Item 1.1a(1) may describe an approach used to set vision and values, how they are deployed, etc.

31. Step 4: Draft around six strengths and opportunities for improvement.

32. Step 4 – Draft Around Six Strengths and OFIs • Select relevant key factors for each approach. • Provide brief statement of approach. • Provide evidence. • Select appropriate evaluation factors (ADLI). • Provide reference to specific areas of the item (e.g., a(1), b(3)).

33. Step 5: Draft feedback-ready comments.

34. Step 5 – Draft one each Feedback-Ready Strength and OFI • Use Comment Guidelines to help craft comments that capture your findings and analysis. • Include these key elements: • Brief “nugget” • An example or two • Why this feedback is important

35. Step 6: Determine the scoring range and score for the item.

36. Step 6 – Determine Scoring Range and Score for the Item • Review balance of observations. • Use scoring guidelines (pp. 32-33). • Determine “best fit” scoring range. • Determine score.

37. Analyzing a Process Item The purpose of process items is to permit diagnosis of your organization’s most important processes: the ones that contribute most to organizational performance improvement and to key outcomes or performance results.

38. Assessing Process Maturity • Process = the methods the applicant uses to address item requirements in Criteria categories 1-6. Four factors are used to evaluate process maturity: • Approach • Deployment • Learning • Integration

39. Approach Approach refers to • Appropriateness of methods • Effectiveness of methods • Degree to which the approach is systematic (repeatable and based on reliable data and information) • Degree to which the approach addresses basic, overall, or multiple requirements of the criteria

40. Criteria Item Format

41. Item 1.1 – Approach (p. 7) a.(1) How do SENIOR LEADERS set your organization’s VISION and VALUES? How do SENIOR LEADERS DEPLOY the VISION and VALUES through your LEADERSHIP SYSTEM, to the WORKFORCE, to KEY suppliers and PARTNERS, and to CUSTOMERS and other STAKEHOLDERS, as appropriate? How do SENIOR LEADERS’ actions reflect a commitment to those VALUES?

42. Deployment Deployment refers to the extent to which • The approach is applied consistently • The approach is used by all appropriate work units • The approach is applied to all appropriate areas, customers, suppliers, partners, etc. Tip: Key factors are important in assessing deployment.

43. Learning Learning refers to • Refining the approach through cycles of evaluation and improvement • Encouraging breakthrough change to the approach through innovation • Sharing refinements and innovations with other work units and processes in the organization

44. Process Maturity - Learning • Three dimensions of organizational learning: • Continuous improvement of existing processes • Innovation leading to breakthrough changes • Knowledge-sharing of such improvements and innovations

45. Integration Integration refers to the extent to which • The approach is aligned with organizational needs. • Measures, information, and improvement systems are complementary across processes and work units. • Plans, processes, results, analyses, learning, and actions are harmonized across processes and work units to support organization-wide goals.

46. Process Maturity - Integration

47. Alignment and Integration

48. Capturing Observations

49. Ground yourself in the criteria item. Select the relevant key factors for the item. Read the relevant section of the application. Draft around 6 strengths and opportunities for improvement (OFIs). Draft feedback-ready comments (Two for IR). Determine the scoring range and the score for the item. How do I evaluate Process Items? By using the Six-Step Process!! Independent Review

50. Exercise: Evaluate a Process Item (Steps 1 - 4) In your small table groups: • Assign roles. • Complete Steps 1 - 4. • Ground yourselves in the Criteria by summarizing • Identify 4 – 6 most important Key Factors • Review the applicant’s response • Identify around six observations with evidence • Prepare to report out.