Hiring, Training, and Evaluating Supervisors

1. Hiring, Training, and Evaluating Supervisors Supervisor Metrics Grant Benson Institute for Social Research, University of Michigan IFD&TC, Miami Florida, May 17, 2005

2. Monitoring Supervisor Performance • Measurement Objectives • Key Performance Indicators • Production Costs • Production Goals • Production Quality • Staff Satisfaction • Respondent Satisfaction / Interview Verification

3. Measurement Objectives • Provide Supervisors with Guidance • Meet Client Objectives • Control Costs • Achieve Response Rates • Obtain High Quality Interviews • Increase Staff / Interviewer Satisfaction • Clearly Establish Expectations • Reward Positive Behavior

4. Key Performance Indicators – Cost • Out-of-Carrel Time, by Shift • Can be indexed by efficiency rates or approximated with minutes between non-productive calls • Efficiency rate = Total interviewing production minutes / (Average interview length * total interviews) • Minutes between non-productive calls = (Total interviewer minutes – Total time in interviewing) / (Total attempts – Attempts yielding interviews) • If shift-level time interviewer time information is not available, may be modeled on calls per hour per active interviewer • ((# iwers * shift length in minutes) – interview minutes) / (total attempts - # interviews)

5. Shift-Based Time Between Non-Productive Calls, Detail

6. Key Performance Indicators – Cost • Project Scheduling and Staffing • Requires that interviewer time is projected on on-going basis, not just once or twice a month • May require using part-timers or arranging for interviewers to be “on call” • % Lines Worked: Lines available during a shift divided by number of lines worked • Calls Per Available Line: Number of dials divided by the number of available lines

7. Survey-Based Evaluation of Staffing Levels, Detail

8. Key Performance Indicator – Production • Completed Interviews, by Shift • Requires realistic goals on a shift level • Should include related results, including screening • Missed Appointments, by Shift • Should be set to zero on hard appointments, with small tolerance for time to call by soft appointments • Indicative of interviewer prioritizing and scheduling of breaks • Rates, by Shift and Interviewer • cooperation rates and refusal conversions

9. Shift-Based Performance Indicators, Detail

10. Key Performance Indicator – Quality • Interviewer Monitoring Rate • Number of interviewers monitored on shift • Overall monitoring rates met • Average Interviewer Performance Score • Comparative score • Longitudinal improvement • Missing Data Rate • Can be calculated on key variables by shift interview was conducted

11. Interviewer-Based Quality Indicators, Detail

12. Key Performance Indicator – Interviewer Satisfaction • Trained, Reliable Interviewers Are Essential to Any Call Center • Survey the interviewers on a regular basis • Measure job satisfaction, understanding of job expectations, supervisor support, and on-the-job training • Post survey results and be responsive to short-comings

13. Sample Interviewer Questions • “How satisfied am I that I fully understand what is expected of me as an interviewer?” • “How satisfied am I that I have received all the tools and training I require to do the job that is expected of me?” • “How satisfied am I with the support I receive from senior management?” • “How satisfied am I with the guidance/direction I receive to do my job?” Source: Deborah Morden (2002) Performance Measures for Contact Centers

14. Summary • Supervisor Metrics Should be Used to Establish Clear Expectations • Metrics Should be Specific and Limited • Metrics Should be Public • Metrics Should Carry A Specific Reward / Retraining Component With Them

15. Sources • Deborah Morden (2002) “Performance Measurements for Contact Centers” • Stephanie Chardoul, M. Patricia Maher, and Kathleen Santos (2005) “Centralized Telephone Interviewing Projects Monitoring / Quality Control Summary” • Grant Benson and Lloyd Hemingway (2004) “The SMS2 Management Manual” • Robert M. Groves et al. (2003) “Using Process Data from Computer-Assisted Face to Face Surveys to Help Make Survey Management Decisions”