New Examiner Orientation

Baldrige National Quality Program New Examiner Orientation Welcome!

Activity Instructions • At your tables, please introduce yourselves • by sharing your names and your industry/sector. • Elect a scribe and a reporter. • Identify something all of you have in common • outside of being a member of the Baldrige Board • of Examiners. • Share your commonality with all of us in five • minutes. • 5 minutes

Course Objectives • After the New Examiner Orientation course, you • should be able to do the following: • Describe the Baldrige culture and the process for evaluating Award applications. • Describe ethical Examiner behaviors. • Define the concept of key factors. • Describe ADLI and LeTCI evaluation factors. • Describe the attributes of a well-written comment. • Discuss scoring. • Support the Baldrige Award process.

Independent Review Module Objectives • Identify the importance of the Independent Review phase of the Award process. • Use analysis skills while completing the Independent Review.

Independent Review BenefitsActivity Instructions • Your challenge is to work together at your tables to • come up with ways that the Independent Review • benefits the applicant’s ultimate feedback report. • Elect a spokesperson and a scribe. • 3. After two minutes, share a top benefit of the • Independent Review phase of the Award process. • 2 minutes

Bloom’s Taxonomy

Analysis “To analyze” is to • Divide a whole into its component parts. • Identify or separate ingredients of a substance. • State the constituents of a mixture. Examine, Study, Investigate, Break Down

Key FactorsModule Objectives • Describe the concept of key factors. • Identify key factors in your assigned application. • Provide applicant-specific evaluation feedback.

What Is a Key Factor? Definition Key factors are significant attributes of an organization that influence the way the organization operates. Examiners will use key factors to focus their assessment on what is important to the applicant.

Key Factors–Customers Examiners use key factors to • Provide more accurate and meaningful feedback to the applicant and understand the Criteria areas of greatest importance to the applicant • Begin to understand the key points to investigate during the Consensus Review and Site Visit Review phases of the Award process

What Is a Key Factor?

Key Factors Activity Instructions • Work with those at your table. Elect a scribe and a reporter. • Review your table’s assigned section of the Nightingale College of Nursing (NCON) Organizational Profile. • For your table’s assigned section of the Organizational Profile, identify areas of similarity and differences in one another’s key factors. • Achieve consensus on a list of key factors for your assigned section of the Organizational Profile. • Capture your ideas on chart paper. 15 minutes

Using ADLI to Evaluate Process Items–Module Objectives • Use the approach, deployment, learning, and integration (ADLI) evaluation factors while assessing an applicant’s responses to Process Items. • Use the ADLI evaluation factors throughout the Independent Review phase of the Award process.

ADLI Activity Instructions • Consider your organization or an organization that you know well. • Identify one process that responds to Category 5, Item 5.1 (Workforce Engagement). • Capture on a pad of paper the approach/process, the extent of deployment, what learning/cycles of improvement have taken place, and how the approach is integrated. • Share your story with a partner from your table. 12 minutes

NCON Item 5.1 Observations Pair-Share Activity Instructions • Form a pair with someone at your table. • As pairs, review the Criteria requirements for Item 5.1. (5 min.) • As pairs, review key factors (KFs) from the NCON Case Study Scorebook for Item 5.1. (5 min.) • Review one another’s NCON 5.1 observations. (10 min.) • As pairs, provide feedback to one another. Each of you should be able to provide your partner with an interpretation of what your partner said in his/her observations. (5 min.) • As pairs, rewrite one observation so that it conveys the intended thought. Use the guidelines from the “What Is an Observation?” handout. (10 min.) 35 minutes

NCON Observations Debrief Table Activity Instructions 1. As a table, elect a scribe and reporter. Discuss the questions: • What did you learn about writing observations? • What will you do differently, if anything, to create meaningful, well-written observations? • Prepare to report top “lessons learned.” 10 minutes

Using LeTCI to Evaluate Results Items—Module Objectives Identify the levels, trends, comparisons, and integration (LeTCI) evaluation factors for Results Items. Use the LeTCI evaluation factors to effectively assess an applicant’s responses to Results Item requirements.

Results Evaluation Factors—LeTCI Activity Instructions Return to the process you used earlier today while assessing an organization you know well using the ADLI evaluation factors. Share the “story” of workforce-focused organizational results (Item 7.4) using the LeTCI evaluation factors. You have five minutes of individual preparation time. Take five minutes each to share the organization’s results “story” with your partner using the LeTCI evaluation factors. 15 minutes

Ethics EnergizersModule Objectives Describe the Baldrige Examiner Code of Ethical Conduct. If faced with an ethical dilemma, choose the actions most aligned with the Code of Ethical Conduct. Demonstrate ethical behavior while serving on the Baldrige Board of Examiners.

Code of Ethical ConductActivity Instructions Work with your tablemates to discuss the ethical situation on the next slide. 2. As a table, discuss what actions are most aligned with the Code of Ethical Conduct. 5 minutes

Ethical Situation #1 The product development cycle of the application I am reviewing is difficult to understand. It is only 6 months long! And the cycle time for the manufacturing process itself is supposedly 32 hours with 100% inspection. My organization takes twice as long with only random sampling! I think I should contact a friend of mine who is an expert on Lean manufacturing. She has studied some of the best organizations, and perhaps she can help me understand if this is really possible. I figure if she says that this just isn’t doable, then I may just call the Official Contact Point to get clarity and perhaps let him know my concerns. What do you think?

Item-Level Scoring—Module Objectives Describe the Scoring Guidelines for Process and Results Items, which are used during the Independent and Consensus Review phases of the Award process. Use the Scoring Guidelines to effectively score Process and Results Items.

Scoring Activity Instructions Form a pair with someone at your table. Referring to your evaluation of Nightingale College of Nursing on the Independent Review Worksheet for Criteria Category 5, Item 5.1, share the score you assigned for this Item and your rationale with your partner. Using the Scoring Guidelines, read the scoring descriptors for the scoring ranges you and your partner selected for the Item, as well as the ranges above and below these. 15 minutes

Consensus Review Module Objectives Describe the synthesis step in the Consensus Review phase of the Award process 2. Synthesize observations for one Item into “around six” core ideas

Bloom’s Taxonomy

Consensus Review Activity #1 20 minutes

Code of Ethical ConductActivity Instructions Work with your tablemates to discuss the ethical situation on the slide in the front of the room. As a table, discuss what actions are most aligned with the Code of Ethical Conduct. 5 minutes

Ethical Situation #2 I received a mailing from someone who claims to have been a Baldrige Examiner for five years. For only $500, we can attend his one-day seminar, “How to Win the Baldrige Award.” His brochure promises ten secrets for winning. Because his approach to winning the Baldrige Award is so successful, he says that he has been given permission to use the MBNQA logo on his publications. He guarantees that our organization will receive at least a site visit, and the brochure implies that he has terrific connections. I definitely think we should sign up, don’t you?

Comment-Writing Module Objectives Describe the elements of a well-written comment. Write comments that add value to the applicant’s feedback report.

Find the Elements of a Comment XYZ uses the Comprehensive Complaint Management System (CCMS) (Figure 3.2-3) to log and track the resolution of customer complaints received through personal conversations, phone calls, e-mails, or survey responses. Data entered into the system are analyzed for root causes to determine a solution and are communicated to the appropriate departments, with additional training provided as necessary. These data are reviewed monthly at corporate performance meetings and again during year-end strategic planning. As the result of these reviews, XYZ improved the process in 2007 with the addition of the Handle It Now process to allow all frontline customer service staff members to resolve complaints on the spot. This process has been implemented in all 25 XYZ branches.

Find the Elements of a Comment ABC provides no comparative data for workforce results relative to workforce engagement and satisfaction. Without appropriate comparative data, ABC may have difficulty addressing its strategic challenges of a shortage of qualified licensed professionals and the influx of private urgent-care centers.

Find the Elements of a Comment XYZ uses the Comprehensive Complaint Management System (CCMS) (Figure 3.2-3) [Approach] to log and track the resolution of customer complaints [Criteria] received through personal conversations, phone calls, e-mails, or survey responses. Data entered into the system are analyzed for root causes to determine a solution and are communicated to the appropriate departments, with additional training provided as necessary. These data are reviewed monthly at corporate performance meetings and again during year-end strategic planning. As the result of these reviews, XYZ improved the process in 2007 with the addition of the Handle It Now process [Learning] to allow all frontline customer service staff members to resolve complaints on the spot. This process has been implemented in all 25 XYZ branches. [Deployment]

Find the Elements of a Comment ABC provides no comparative data for workforce results relative to workforce engagement and satisfaction [Criteria]. Without appropriate comparative data[comparisons], ABC may have difficulty addressing its strategic challenges of a shortage of qualified licensed professionals and the influx of private urgent-care centers [applicant-specific].

Your Challenge Your challenge is to practice constructing a strength comment for one core idea generated by those at your table.

Activity: Comment Writing Practice 30 minutes

Reflection Take a few moments to reflect on your personal learning from today’s session, and make a few notes on your next steps. What have you learned regarding your strengths? What did you discover that may be opportunities for your development? How will you address your opportunities for development?

New Examiner Orientation