Writing Well Written Comments

1. Writing Well Written Comments

2. How do good comments add value? Organization Profile Applicationcontent Criteria The most valuable Comments/Key Themes

3. The Structure of Well Written Comments Well written comments incorporate criteria language, ADLI/LeTCI, scoring language, and application language.

4. Category 1-6 Does the comment link to the Criteria? Does it consist of 2-3 grammatically correct sentences (about 1000 characters)? Does it use examples from the application that illustrate the OFI/Strength? Does it include a so what if it is an OFI? Is the statement non-judgmental and non-prescriptive? Does the comment link to the scoring guidelines? Well Written Comment Guidelines

5. Well Written Comment Guidelines • Tie your comments to the basic, overall, or multiple level Criteria requirements that are most important to the applicant. • Write process comments so they contain: a subject identified from the criteria, the application, or the scoring guidelines; verb(s) and requirements from the Criteria; examples from the application, and figure numbers, as appropriate. • Use the evaluation factors (ADLI or LeTCI), and scoring language, to clearly articulate the areas of the applicant’s response to clarify the strength or provide insights that will help the applicant improve overall organizational effectiveness and capabilities. • Draw linkages between an Item and the applicant's Organizational Profile. • Ensure that the comment does not contradict other comments in the same or other Items or in the Key Themes Worksheet.

6. The Style of Well Written Comments • Use active voice and present tense (e.g., 'completes' rather than 'is completed'). • Use a polite, professional, and positive tone. • Use vocabulary/phrasing from the Criteria and the Scoring Guidelines. • Describe what is missing if something "is not clear". “Not clear” comments must be resolved if the applicant has a site visit. If they don’t do it based on site experience then say they don’t do it. • In lieu of the applicant’s name, use such phrases like "the applicant" or "the organization" to refer to the applicant in the independent and consensus scorebooks. Use the applicant's name in final feedback reports. • Use the applicant's language/terminology when appropriate.

7. “Don’ts” to avoid in comments • Avoid parroting • Avoid being prescriptive • Avoid going beyond the criteria

8. “Do’s” for well-written comments (refer to comment guideline in workbook)

9. Example Strength Comment • The applicant identifies their core competenciesas the participating dentist network; expert claims administration; extraordinary customer service and performance culture. A SWOT analysis is used in the annual strategic planning process to identify core competencies. Correlation analysis is performed on satisfaction survey data and is used to determinekey drivers of customer satisfaction. Initiative action plansaccentuate core competency strengths. Deploymentis evidenced by the use of bi-annual CAG meetings, ensuring alignment with the missionand the integrationof data for easy access across all departments and customer segments • Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

10. Well Written Comments Don’t…. • Go beyond the requirements of the Criteria or assert your personal opinions. • Be prescriptive by using "could", "should", or "would". • Be judgmental by using terms such as "good", "bad", or "inadequate". • Tell the applicant that they aren’t doing enough of something • Comment on the applicant's style of writing or data presentation. • Use jargon or acronyms unless they are used by the applicant • Only “Parrot” the application or the Criteria. Provide only enough language to add clarity—seek to add value rather than restate information.

11. How Could This Comment Be Improved? Application language: • Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: • Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the Define step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. This is proof that the organization has a great systematic approach to the SPP and extensive cycles of learning in 1997, and 2004.

12. How Could This Comment Be Improved? Application language: • The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: • Our Store is in the beginning stages of building an approach to setting a vision for customer satisfaction, this includes the manager working very hard to instill the beliefs in leadership and responsibility. Everyone in the store receives a reward and this will ensure that everyone will have ownership in the organization and the ability to evaluate their own work, which should improve performance overall.

13. How Could This Comment Be Improved? Application language: • Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: • Customer relationships are built through a variety of methods including Patient enrollment, Patient-Family Advisory Boards, and community health activities. Surveys are conducted randomly after patients are discharged. The Patient-Family Advisory Board is used to determine issues that lead to poor relationships and implements new relationship-building processes.

14. How Could This Comment Be Improved? Application language: • Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: • Through the SPP, customer groups are annually identified using the MVV as a focus. The SWST uses demographic data to identify gaps, disparities and identify potential customers. This process has been deployed throughout the organization, and through multiple cycles of refinement the customers needs in the service area have been met.

15. Well Written OFI Comment Guidelines • May be used to give credit to the applicant for some areas that HAVE been addressed • “While the applicant does x, y, z, it is not evident that a, b, and c are performed.” • May begin with a statement about what is missing: • “There is no evidence that…” • “There is no indication of how…” • “It is unclear…” • “It is not apparent that there is a …” • Must tie to criteria language, (ADLI), and/or to Key Factors that the applicant has identified in the Organizational Profile • Must include a “so what” (Why does the applicant care about this? Why would they take action?) • Usually these come from Key Factors / Org Profile. The closer the ‘so what’ can be to identifying the applicant’s specific needs, the better.

16. Example OFI Comment • Although cross functional teams are chartered and aligned with the SPP, there is no indication of processes used by the teams or otherwise by the applicant to design and innovate its work processes to meet key requirements. There is no indication of how design and innovation processes address opportunities for agility and improvements in cycle time and productivity.It is also unclear how the team work process improvementapproachis evaluated forannual process changes or improvements. An annual evaluation of strategic teamalignment and improvements may provideefficiency and effectivenessin theaccomplishment of strategic objectives, initiatives, and action plans. • Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

17. How Could This Comment Be Improved? Application language: • Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: • Although Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors, there is no evidence of how blind spots are identified, nor how time horizons are set and how the SPP addresses those horizons. If the applicant implemented a more extensive benchmarking process that identified blind spots and included time horizons, there may be more of an opportunity for this organization to have a more successful Strategic Planning Process.

18. How Could This Comment Be Improved? Application language: • The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: • Although the manager has worked hard to instill the belief that leadership and responsibility should be present at all levels, there is no indication of a systematic approach, nor is there deployment at any level, no integration, nor are there any cycles of learning.

19. How Could This Comment Be Improved? Application language: • Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: • There is no systematic approach to building customer relationships, or in conducting patient surveys. And although there is a Patient-Family Advisory Board there is no approach described for how issues are determined and how new relationship-building processes are developed.

20. How Could This Comment Be Improved? Application language: • Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: • Although customer groups are identified annually in the SPP, and the Spirit Team (SWST) identifies gaps, disparities, and identifies potential customers, there is no indication how this information is deployed throughout the organization and how the information is used to meet the unmet needs of customers in the area.

21. Comment Writing Exercise • Each person at the table select an independent review item (Cat 1-6 strength or OFI) • Write a well written comment • Swap scorebooks • What feedback would you give to this examiner • Use the comment checklist for examples • Practice giving feedback to each other • Report out the most significant key learnings • (If time swap with to another person at the table)

22. Category 7 Are specific results (numbers) used rather than generalities? Have trends, comparisons and/or appropriate benchmarks been considered? Does the comment include figure numbers? Have we summarized multiple charts with a consistent theme (trends, missing, comparisons, etc.) Category 7 Comment Guidelines

23. (Well Written Result Comment Example There is a lack of comparative data in a number of the Results sub-categories; as an example, no comparative data is available for measures such as Discharge Weight of ELBW Infants (7.1-4), Improvement % of ELBW Infants (Figure Figure 7.1-5), % of Breast Milk Available (Figure Figure 7.1-6), Outpatient Lactation (7.1-7) and Catheter Related Bloodstream Infections (7.1-8), Survival to Discharge by Rhythm (Figure Figure 7.1-17), and Well Child Visits Figure (7.1-19). The lack of comparative results may impact the value of Excellence - setting and achieving the highest standards.

24. How Could This Comment Be Improved? The applicant’s market share results for key service lines is strong, exceeding all comparative data across Discharges; Inpatient Surgeries; the applicant's Heart Services; and Neurosurgery Market. The applicant enjoys market dominance in these service lines and has shifted its focus to adolescent care and services to new geographic markets. The applicant has received numerous recognitions and awards for their leadership in these markets and service lines. These successes are important to their goal of achieving national recognition as a premier hospital, a tertiary and quaternary hub, and preferred provider of regional health care services.

25. How Could These Comments Be Improved? • The applicant's Net Income from Operations (7.3.8) is above the comparison median and trending up over the past three years and overall applicant’s turnover rate has been decreasing. • The applicant’s Net Sales between 2006 and 2008, have increased by 10% in 2006 to over 12.2% in 2008. The applicant is consistently above the goal of 9% for all three years, but is below the Baldrige comparison of 15% for all three years. • Although the turnover rate for RNs has been increasing between 2006 and 2008. The turnover rate for RNs was 7.7% in 2006 and 10.3% in 2008. There have been substantial improvements in physician satisfaction from 13% to over 25% during the same time period.

26. Comment, scoring considerations • For LITE, only one score for entire results Category 7 • Propose a score for results related to your categories • Be prepared to reach team consensus for all of Category 7 • Using scoring language can help the applicant understand comments and score

27. Category feedback • Check and provide feedback on whether Cat. Lead’s comments: • Include criteria requirements • Provide meaningful feedback • Address ADLI or LeTCI • OFI’s provide relevant “so what” (at least some of them) • Results cite figure numbers/titles and specifics • References to application are correct • Recommended score reflects comments in content and balance (strengths:OFI’s ratio) • Back-up does this for full, team does this for lite

28. Information resources • WSQA comment guideline handout • Your back-up, team lead, PEPD mentor • Reference materials in • Your workbook: comment and scorebook examples • http://www.wsqa.net/extraining.php • Baldrige web site http://baldrige.nist.gov/Examiner_Resources.htm

29. Six-Step Evaluation Process

30. Managing your time • Put aside time in your calendar (refer to time commitments) • Work with your lead & team • Set meetings early • Coordinate • Communicate • Stories to tell?

31. Managing your time • Schedule work hours • Break tasks into little pieces • Reward yourself for wins • Move quickly past setbacks • Call your lead • Call your back-up

32. Software • Link to software http://www.wsqa.net/extraining.php • Username and Password: firstlast name • Link for software manual and taped webinar http://www.wsqa.net/Extimetable.php Click on scorebook, select Key factors, then select Item evaluation Save Frequently, Save OFTEN, Save A Lot!-save in software- not on your browser

33. THANK YOU!! Your support and participation as Examiners helps us all by helping WSQA fulfill its mission!