BENCHMARK REPORT

1. BENCHMARK REPORT

2. Agenda • Beginner Level Basics • Introduction • Generating reports • Interpreting reports • Beginner to Intermediate Level Details • Other report options • General tips • Resources REMINDER: the information generated by this report is only as accurate as the data entered into the NCIR!

3. Introduction

4. Benchmark: A benchmark is a standard of excellence or achievement that similar things are compared to or measured against (in order to evaluate performance or level of quality).

5. NCIR Benchmark Report: The NCIR benchmark report assesses the immunization records of a particular group of your clients and tells you HOW MANY, WHAT PERCENT, and WHICH of these clients have met and not met a particular “benchmark” for immunizations.

6. Key concept: • Select a group of clients to assess. • Select the criteria for meeting the benchmark (BM). • The report determines which of these clients have met the benchmark criteria and which have not. • within any age or birth date range selected clients Up-to-date with: specific vaccine(s) By: specific age (for childhood vaccines) ( ( ( ( clients who have not met benchmark clients who have met benchmark

7. Key concept: • Select a group of clients to assess. • Select the criteria for meeting the benchmark. • Report determines which of these clients have met the benchmark criteria and which have not. • within any age or birth date range select clients select benchmark criteria

8. Why generate a report? number and percent • To learn the number and ercent of clients up-to-date (UTD) with a particular vaccine or group of vaccines: • by a specific age (for childhood immunizations) • by the date the report was run • For a list of clients that indicates (for each selected vaccine) whether the client was/had: • UTD by benchmark age; late UTD; due (and if so, number of doses received); refused list of clients

9. Why generate a report? number & percent Info list of clients

10. Beginner Level Generating Reports

11. To open: • Click “benchmark report” (on the left side of the NCIR screen, under the “reports” section) Note: Benchmark reports must be generated from an “Administrator” account.

12. These are the main sections of concern for beginner level reports. Beginner Reports select clients 3 5 Select Evaluation Date 6 select benchmark criteria

13. Select: group of clients to assess (use only one of the three options) 3 • The 1st/default option generates results that can be confusing.* Until comfortable with report, use another option. • Birth date range: If you have a specific birth date range, enter dates or select using calendar icon. • Age range: If you don’t need a specific birth date range, the age range option is simpler. Be sure to specify “days,” “weeks,” or “years” in the dropdown menus. *For more info on this option, see the benchmark report manual.

14. Select: evaluation date 5 • Type/select the current date (the date that you are running the report). You must enter an evaluation date or the report will not run. • A date that is different from the report generated date can impact the results (designed for advanced level use).* *For more information on evaluation dates, see the benchmark report manual.

15. Select: benchmark criteria 6 • The report will review the immunization status of each client to see if it meets this BM criteria that you selected. Column: Vaccine Rows: Age/Age Group In cell: Number of doses required to meet BM

16. Select: benchmark criteria 6 • Childhood benchmarks (BM): when you want to know if a group of clients are UTD with vaccines routinely given in childhood. • As childhood vaccine recommendations are based on age, the BM report includes age and immunization criteria. (Aka: to meet a childhood benchmark, clients must be UTD by a particular age) Childhood benchmarks

17. Select: benchmark criteria 6 • Adolescent/Adult Benchmark: when you want to know if a group of clients (within any age or birth date range, but usually clients over age 6) are UTD with vaccines routinely given in adolescence or adulthood. • There is no age criteria for these vaccines. Clients meet the benchmark if they were UTD when the report was generated. Adolescent/Adultbenchmarks • Note: Both males and females will be assessed for HPV.

18. Select: benchmark criteria 6 • You can select a full row: Click on the first cell of the row that you want to select. The row will become highlighted. In this example: all assessed clients who are UTD with 4DTaP, 3HepB, 3Hib, 1MMR, 3Polio, 4Pnuemo, & 1Var by 24 months of age will meet this benchmark.

19. Select: benchmark criteria 6 • You can select a single cell: Just click on it. In this example: the report will determine which of the assessed clients were UTD with Hep B at 16 months. • You can select multiple cells within the same row, but you cannot select cells in more than one row. In this example: clients UTD with 1Mening and 3 doses of HPV will meet BM

20. Select: benchmark criteria 6 • To deselect: Just click on the cell and it will no longer be highlighted. In this example: clients must have completed the 4:3:3:1:3:1 series by 24 months of age in order to meet the benchmark

21. Click generate (only one time, please) and then wait… To process:

22. Report will take a few minutes to complete. Click “refresh” occasionally to determine report status. To view report: • When status is “DONE,” click on blue benchmark link to view report.

23. For beginner level reports Interpreting Report Results

24. So, this is the example criteria that was selected, and now we will look at the report that would be generated from this criteria. Beginner Reports Clients: age 24-36 months 3 3 5 Report run date: 1/9/13 6 6 Benchmark: UTD with 4:3:3:1:3:4:1 series by 24 months

25. Key sections of report • List of selected criteria for report (“filter conditions”) • Number & percent Information • List of clients report criteria number and percent Info list of clients

26. Selected criteria for report • Each selection you made for the report is listed here • Tells you exactly what the report means “Report generated” and “Evaluation” dates are the same Selected Clients: 24-36 months of age Selected benchmark immunizations” 4:3:3:1:3:1 by 24 months

27. Understanding Report Criteria Clients: Benchmark: Clients: Benchmark:

28. Number & percent info • The primary outcome of the report. • Found on the last two lines of the “filter conditions” section. • The first number is the number of total clients that were assessed. number and percent info

29. Understanding number and percent info: 4 of 10 clients (40%) UTD by 24 months 5 of 10 clients (50%) UTD by the report generated date (the 4 who met the benchmark and 1 late) • Line 1: UTD with BM vaccines by BM age • Line 2: UTD with BM vaccines as of date report generated (includes clients on the first line plus those who were late UTD) Misleading. Pretend this says: “Including late UTD”

30. For Adolescent/Adult BM: • Adolescent/Adult benchmarks have no age criteria • Clients cannot be late UTD • The two number and percent lines are identical 5 of 12 clients UTD/ met all BM criteria Clients cannot be late UTD. So, including late UTD clients, the same 5 of 12 (42%) met all BM criteria

31. Client list • Lists clients who did not meet all benchmark criteria (Note: you can choose which clients are listed- discussed on following slides). report criteria Number and percent Info . List of Clients

32. Understanding the client list Met benchmark • Y : met all benchmark criteria for that vaccine Column: Selected BM Vaccine Parenthesis: number of doses required to be UTD) Did not meet benchmark • N C: UTD (“completed”) vaccines, but late UTD • N # : Not UTD (and the # of doses received) • N : Not UTD (did not receive any doses of vaccine)

33. To view a specific record: • This list is an easy way to quickly review client names, addresses, and phone numbers and see if any are obviously missing or wrong (e.g. out of state). • You can view a specific client record by clicking on the link on the client’s last name.

34. Reviewing client information • You can view client info but you cannot make changes. • When you close out of the screen you’re still able to view report. Gender is coded as “U” (unknown). Missing address Client comments

35. Updating client information • Please correct or update any info that is incorrect or missing (Note: Changes will not show up on a report until the next day). • Steps: copy the Client ID and use it to search for the client through “manage clients.” Make edits.

36. Saving/exporting report • Be sure to save the report! Once you run the next report, any previously run report will disappear! • In order to save the report you must first export it as text or a spreadsheet or display it as a PDF • Filter conditions can only be saved and viewed in PDF format. • You can use a spreadsheet if you want to sort or further filter the client information. *See user manual for details on why you want to export information and what you can do with it.

37. To re-open a report: • Once you leave the BM screen you can still return to the last report that was generated. • Click on “check benchmark.”

38. Beginner to Intermediate Level Other options when generating the benchmark

39. Again, here is the screen where you can build the report. Screen used to build report 1 2 3 4 5 6

40. Sections that can be skipped but may at times be useful. Screen used to build report 1 2 4

41. Select: client population 1 Dropdown Menu • Only relevant if your practice is associated in the NCIR with multiple sites/clinics and you want to assess clients from only one of those sites. • If the above applies, select the 1st option and specify a site using the dropdown menu. • Otherwise, leave the 2nd/default option selected.

42. Select: client population (LHDs only) 1 • Local Health Dept. accounts display two extra client population options: • 3rd option: Assesses only people who are listed in the NCIR as residents of your county (whether or not they are “active” clients). • 4th option: Assesses your “active” clients (regardless of their county of residence) AND people who are listed in the NCIR as residents of your county,

43. Select: clients displayed on the client list 2 • This section determines which clients are displayed in the client list. It has no other impact. • 1st/default option: Generates a list of clients who did NOT meet the benchmark. • 2nd option: Generates a list of clients who DID meet the benchmark. • 3rd option: Generates a list of all assessed clients (those who met benchmark and those who did not).

44. Select: how to assess refusals 4 • You can run the report so that clients who refused a vaccine are assessed as UTD with that vaccine.* • 1st/default option: only considers vaccinated clients as UTD. • 2nd option: considers both vaccinated clients and those who refused the vaccine as UTD. *Based on “client comments” recorded in the NCIR. For more info, see user manual.

45. General Report Tips

46. Refusals • One refusal of a vaccine is counted on the benchmark report as one “dose” of that vaccine. • Example: If a client has 2 refusals for HepB, then the client would meet the benchmark for 1Hep but would not meet a benchmark that was run for 3Hep. • You can now record multiple client refusals for the same vaccine in “client comments,” as long as each entry has a different “applies-to” date. • You may want to review previous refusals and add “as-of” date.

47. Refusal example You want to know: • Clients: age 13-19 • Benchmark: UTD with 3HPV

48. If refusals do not meet benchmark

49. If refusals meet benchmark

50. Refusals on client list An “R” in the client list indicates that the client is considered UTD for all doses of the vaccine as a result of refusals. Note: “R”s only display on reports run for childhood BMs. For adolescent/adult BMs, UTD as a result of refusals is indicated the same as in UTD as a result of vaccination (“Y”).