Welcome, Examiners!

1. Welcome, Examiners! Washington State Quality Award Examiner Training 2010

2. Thank you!! • THANK YOU for your time, your efforts and expertise! • WSQA’s mission: Improving the way we live, learn, and work in Washington by helping organizations improve through the use of the Baldrige Criteria for Performance Excellence

3. Objectives For This Class • Learn to examine a WSQA application through all stages of the assessment process • Do this with confidence • In yourself as an Examiner and • In the assessment process

4. Agenda: Session 1 • WSQA organization overview • Baldrige criteria overview • WSQA Assessment process • Online scorebook preview • Independent review, step-by-step • Preparing • Key Factors • Process review • Results review Mostly presentation Mostly exercises

5. Agenda: Session 2 6. Ethics, code of conduct 7. Consensus, step-by-step 8. Key Themes 9. Site visit preview 10. Final scorebook preparation 11. “Lite” vs. “Full” 12. Summary Mostly exercises Mostly presentation

6. Introductions • Name, organization, what you do • Questions and expectations? • What do you like to do when you’re not working?

7. Workbook contents Table of Contents Session 1 Slides TAB 1 Session 2 Slides TAB 2 Ethical Standards and Conflict of Interest Determination TAB 3 Considerations for Small Organizations TAB 4 Comment Guidelines and Scorebook Editor TAB5 Case Study TAB 6 Scorebook and Feedback Report TAB7 Independent Review Blank Worksheets TAB 8 Sample Evaluations TAB 9 Homework Instructions and Code of Conduct Exercise TAB10

8. Signed Ethical Statements • Collect Signed Statement

9. For a Successful Course… • Be an active participant • Give only constructive feedback • Be courteous to your colleagues • Honor diverse opinions • Seek to understand • Ask questions • Have fun!!

10. 1. WSQA organization overview 10

11. What is WSQA? • Created by State law to: • Promote excellence • Recognize achievement • Educate and train • Patterned after the Baldrige National Quality Award • Awards presented annually by the Governor • Awarded to private, public, and not-for-profit organizations in manufacturing, service, education, and healthcare

12. WSQA Vision WSQA Mission Be recognized as a premier resource for advancing organizational excellence. Make Washington State a better place to live, learn, and work by helping organizations achieve superior results through the use of the Baldrige Criteria for Performance Excellence.

13. Funding WSQA is a 501(c)(3) non profit organization supported by • Corporate donations • Individual memberships • Workshop , collaboratives and conference fees • Many volunteer hours

14. Learning Opportunities • WSQA offers workshops • Introduction • Self Assessment • Case Study • Preparation for Application • Annual conferences • Consulting • Performance Excellence Collaborative • Become an Examiner

15. Who is WSQA? • Examiners • Lite Assessment • Full Assessment • Certification follows assessment • Senior Examiners • Level 3 Team Leads • Scorebook Editors • Panel of Examiner & Process Development (PEPD) Mentors • Panel of Judges • Board of Directors • Executive Director and support team

16. 2. Baldrige criteria overview

17. Objectives • Learn a little Baldrige history • Get a high-level understanding of the criteria • What’s in the booklet? • YOUR EXAMINER TRAINING INCLUDES ATTENDANCE TO INTRODUCTION TO PERFORMANCE EXCELLENCE WORKSHOP!

18. Baldrige award history • Created by law in 1987, named for Malcolm Baldrige, Secretary of Commerce from 1981-1987 • Purpose is to improve American productivity and quality • Originally just for manufacturing, now includes education, healthcare, nonprofit organizations • Public-private partnership

19. Public / Private Partnership Private Private Private Private Private Private Public Public Public Affiliated Affiliated Foundation Foundation Foundation Foundation Foundation Foundation Foundation Foundation Foundation Foundation Foundation U.S. Dept of Commerce U.S. Dept of Commerce U.S. Dept of Commerce U.S. Dept of Commerce U.S. Dept of Commerce U.S. Dept of Commerce ASQ Board of Examiners Board of Examiners Board of Examiners Board of Examiners Board of Examiners State Award Programs NIST Board of Overseers Board of Overseers Trade Associations Award Recipients

20. Baldrige Framework - A Systems Perspective Leadership Triad Organizational Profile: Environment, Relationships, and Challenges • 2 • Strategic • Planning • 5 • Workforce Focus • 7 • Results • 1 • Leadership • 3 • Customer and Market Focus • 6 • Process • Management Results Triad 4 Measurement, Analysis, and Knowledge Management Supportive Underpinning

21. Core Values and Concepts • Agility • Management by Fact • Systems Perspective • Organizational & Personal Learning • Focus on the Future • Societal Responsibility • Visionary Leadership • Valuing WF Members and Partners • Managing for Innovation • Focus on Results and Creating Value • Customer-Driven Excellence 21 21

22. Criteria Characteristics Non-prescriptive and adaptable Created and improved through a consensus-building process with the user communities Capture the leading edge of validated management practice Changes incremental in the short term, but significant over the long term Support improvement in overall organizational performance practices and capabilities Focus on sustained improvement in results Facilitate communication and sharing of best practices 22

23. Six-Step Evaluation Process

24. Ground yourself in the Criteria Item

25. **Item Question Organization** Glossary

26. Understanding the Questions • The quality of the feedback provided depends on understanding of the meaning and significance of the questions • The Criteria book provides aids to understanding • A Glossary of Key Terms is provided; Terms with definitions are identified with CAPs • Open your criteria, find a few Key Terms and look up the definition • Notes are provided to clarify key requirements, give instructions, and indicate important linkages • Read N1 on Item 1.1 (page 7) to find an expectation concerning the applicant’s Vision • Category and Item Descriptions discuss the purpose for the questions and comment on expectations for answers • Read the first Comment in Section 1.1 and consider how it will affect the applicant’s discussions in 1.1

27. 3. WSQA Assessment Process Objective: Introduce the Assessment process

28. Read Criteria Read Applicant Response Analyze Response Record Analysis Meet with co-examiners to consolidate analysis Visit the Applicant to clarify/verify Co-write an analysis report What Examiners Do…

29. Stages of the Assessment Process

30. Preparing for independent review • Read the criteria • Read the application • Familiarize yourself with the online tool http://www.wsqa.net/extraining.php • Check in with Team Lead or Mentor after preparing a few Key Factors

31. Select the relevant Key Factors for the Item

32. 4. Independent Review:Key Factors Objectives: 1. Know how to prepare for independent review 2. Be able to identify and record Key Factors

33. Key Factors topics • Role and importance of the Organization Profile • What are Key Factors? • Role of Key Factors in the assessment process

34. Organization profile • Introductory section of the Criteria and the application • WSQA criteria requires applicant to complete the entire Baldrige profile • Applicant describes • Organizational environment • Organizational relationships • Competitive environment • Strategic context • Performance improvement system

35. Organizational profile • Not evaluated or scored as are categories • Examiners use as a basis for evaluation • What do we need to know and understand about the applicant? • What’s important to the applicant? • Applicant can use as an initial self-assessment to identify gaps/conflicts in knowledge • Evaluation: How well does the applicant address its Key Factors?

36. What is a “Key Factor”? • “Significant attributes of an organization that influence the way the organization operates” • Facts about the organization and key strategic decisions made that shape the organization

37. What is a “Key Factor”? • Examples • Mission, vision, and values • Employee/staff profile • Customer and market segments and customer requirements • Competitive position and critical success factors • Strategic challenges • Governance structure • These are all key factors because they are facts or attributes that affect the way the organization operates.

38. What is a “Key Factor”? • Non-examples • Complaint management process • Strategic planning approach • These are not key factors; rather, they are processes that the applicant uses in managing its work

39. How to identify key factors • Responses to the Organizational Profile • Responses to the Criteria Items—it is important to consider key factors wherever they are found in the application. • Eligibility forms

40. Key Factor Format • Not full sentences • No need to specify “the applicant” • First word is Bold • Example: Workforce: 1500 ees, 80% technical, 20% administrative, 30% minority, 25% work remotely

41. Exercise 1: Key Factors You will need: • Flip chart and marker • Case study application • Criteria Steps: • For assigned Section of the Organization Profile, identify 2-3 key factors • Review Key factors in the Case Study Scorebook for your section to see identify similarities.

42. 5. Independent review:Process Items Objective: Be able to diagnose a process item for independent review 42

43. Read the relevant section of the application

44. Read the application • Begin Analysis of the Response • Which questions are the applicant answering? • Which evaluation factors [Approach, Deployment, Learning, Integration] are present in the response? • Make notes on your findings • Sticky notes, highlighter, comments in the margins • Identify any results that could be generated by the activity and reported in Category 7 • If you find misplaced information (key factors, responses to other criteria items), put references to those responses in the appropriate section of the applicants response

45. Independent review • Select the most relevant Key Factors (from list) • Read the criteria for the category • Read (and annotate) the relevant section of the application • Identify and record the applicant’s processes • Record observations. Note missing responses and significant observations • Recommend a scoring range for the category

46. Draft around six strengths and opportunities for improvement

47. Independent review steps - process • Review related criteria • Read the category under review, take notes • Select relevant key factors from list • Identify & record processes • Record observations on worksheet under strengths and Opportunities • Indicate • Approach • Deployment • Learning • Integration

48. Identify and record processes • “Process” includes • Inputs • Steps (related activities) • Time frames • Outputs • End users (internal/external customers) • Standards • Key measures to evaluate and improve the process • Not always formal or named – you might have to read between the lines • Look to criteria for guidance

49. Identify and record processes “Systematic” refers to processes that • Are well-ordered • Are repeatable • Use data and information so learning is possible • Build in the opportunity for evaluation, improvement, and sharing

50. Identify and record processes • “Effective” refers to how well a process or measure addresses its intended purpose • Determining effectiveness requires evaluating • How well the approach is aligned with the organization’s needs • How well it is deployed, and/or • How the evaluation of the outcome of its measures are used, i.e., for learning and improvement