Introduction to the New Examiner Preparation Course

1. Introduction to the New Examiner Preparation Course

2. Introduction At the end of this unit, you will be able to • Know the names of the training team and your table mates • Describe your role and responsibilities in training

3. Uncommon Commonalities Warm-Up • Use less than 30 seconds each to introduce yourselves to your table mates • Within five minutes from the start time, identify one thing your group members have in common that is unique to the entire class • Report out

4. Malcolm Baldrige National Quality Program Department of Commerce ….. Technology Administration Foundation for the Malcolm Baldrige National Quality Award Board of Overseers National Institute of Standards & Technology Baldrige Award Winners Cooperating Organizations - Professional Societies - Trade Associations - State and Local Networks • Board of Examiners • Judges • Senior Examiners • Examiners • Alumni Examiners • Contractor • ASQ

6. Introduction At the end of this unit, you will be able to • Know the names of the training team and your table mates • Describe your role and responsibilities in training

7. Malcolm BaldrigeNational Quality Award New Orientation - Process Items

8. At the end of this unit, Examiners will be able to: • List and describe the information asked about in the Process Categories • Apply the evaluation process to Process Items • Describe the format for good Process summary notations and comments • Demonstrate how to score a Process Item • Write a Process comment using the format

9. 2004 - Process Items 2003 - Approach/Deployment Items

11. Item 3.1 Vocabulary • How • Stakeholder • Learning • Segments

12. Additional Vocabulary • Key • Partners • Systematic • Action plans • Alignment

13. The Item Evaluation Process

14. The Item Evaluation ProcessStep 1

16. The Item Evaluation Process Step 2

17. Key Factors Exercise

18. 3.1 Key Factors • Largest public school district in state with 84,169 students, in 8 cities, 68 buildings, 750 square miles • Key Student segments regular Ed, special Ed, ESL, ESP, LCC, and NCS (and higher Ed is a customer) • 4 stakeholder groups – parents, taxpayers, school board, and businesses (Chamber of Commerce/Bd. of Econ. Development) • Strategic challenge – community – related: engage parents, the community and businesses in collaborative learning efforts • Strategic challenge – education and learning: agile and responsive to changing performance expectations, e.g., NCLB, addressing pockets of poverty • DLT manages through Perf. Excellence System (Figure P.2-1)

19. The Item Evaluation Process Step 3

20. The Item Evaluation Process Step 4

21. Verb & Descriptive language from the Criteria Example from appl. methods/ activities Transition by/through/ using Subject from the Criteria or Application + + + Process Summary Notation Basic Format

22. Process Summary Notation Basic Format OFI Verb and Descriptive language from the Criteria + + Intro. statement Subject from the Criteria It is not clear how/apparent The applicant does not report/address how

24. Summary Notation Exercise

25. The Item Evaluation Process

27. Approach • Methods • Appropriateness • Effectiveness • Repeatable • Based on data

28. Deployment • Approach is applied consistently • Approach is applied to most important Item requirements • Approach is used by all appropriate work units

29. Learning • Refining the approach • Encouraging breakthrough change • Sharing

30. Integration • Aligned with organizational needs • Measures, information, … are complementary across processes and work units • Plans, processes, … are harmonized across process and work units

31. The Item Evaluation Process Step 5

33. Steps in the Scoring Process 1. Review Criteria requirements and summary notations 2. Note balance and importance of strengths and OFIs 3. Read the description of the scoring ranges in the Scoring Guidelines 4. Determine the scoring range most descriptive of the organization’s achievement level 5. Read range descriptions above and below to determine where, within the range, the score would fall 6. Verify that the balance and substance of strengths and OFIs correspond to the score 7. Record the score

34. Scoring Exercise

35. The Item Evaluation Process Step 6

36. Which do you prefer? • App. listens and learns to determine students’ and stakeholders’ requirements and expectations and their relative importance w. 4-stepSSRDP (Figure 3.1-2). SSRDP gathers info on requirements/expectations from key student segments, and other key stakeholders through surveys and other segment-specific methods with varying frequency from daily to every 4 years (Figure 3.1-3). • App. listens and learns to determine students’ and stakeholders’ requirements and expectations and their relative importance w. 4-stepSSRDP (Figure 3.1-2). SSRDP gathers info on requirements/expectations from key student segments, i.e., current students, former students, alumni and prospective students (home schooled and adult students) and other key stakeholders (parents, school board, taxpayers, and business) through surveys and other segment-specific methods with varying frequency from daily to every 4 years (Figure 3.1-3).

37. Use single complete thought 1-3 complete sentences Address central Criteria requirement Relevant and important to applicant based on KFs Non-prescriptive Non-judgmental Draws linkages to the Organizational Profile or Other Items Does not conflict with other comments Process Comments General Guidelines

38. Process Comment Basic Format Verb & Descriptive language from the Criteria Example from appl. methods/ activities Transition by/through/ using Subject from the Criteria or Application + + + + Scoring Guidelines language + so-what + grammar and punctuation check

39. Applicant determines the student segments and markets that its educational programs will address through an effective, systematic, 4-step Segmentation Process. That process, managed by the Research and Knowledge Management Department, involves: 1) gathering, analyzing, and integrating information and data using a variety of listening/learning methods (Fig. 3.1-3); 2) validating existing segments and identifying emerging segments and market opportunities; 3) communicating final segmentation throughout the district; and 4) organizing information, data and knowledge by segment. The segmented information is used to meet organization, information, and NCLB requirements and is integrated with the district’s organizational needs identified in other Criteria categories, e.g., strategic planning (Fig. 2.1-1), relationship management, and curriculum design (Figure 6.1-1).

40. Comment Writing Exercise

41. At the end of this unit, Examiners will be able to: • List and describe the information asked about in the Process Categories • Apply the evaluation process to Process Items • Describe the format for good Process summary notations and comments • Demonstrate how to score a Process Item • Write a Process comment using the format

42. Evaluating, Scoring, and Writing Comments for Results Items

43. You will be able to • Explain the purpose of results • Describe where results are found • Describe the information requested in Results Items • Indicate that results respond to “what” questions • Apply the evaluation process to a Results Item • Describe the elements of a good Results comment • Write Results comments that include those elements

44. Results Items • Answer “what” questions • Link to important processes and action plan requirements • Cover students/customers, products/services, market/financial, HR, operational, governance/social responsibility

45. Organizational Performance Results

46. Point Values for Results Items

47. Key Terms • Results • Summative assessment • Formative assessment • Measures and indicators

48. Key Terms, con’t. • Current levels of performance • Trends • Comparisons • Segments

49. Item Evaluation Process

50. The Item Evaluation ProcessStep 1