National RUSA Institute: “Serving 21st Century Users: Opportunities & Challenges”

1. Reference Assessment Programs: Evaluating Current and Future Reference ServicesINTERVIEWS & FOCUS GROUPS National RUSA Institute: “Serving 21st Century Users: Opportunities & Challenges” Marie L. Radford, Ph.D. Associate Professor, School of Information & Library Science, Pratt Institute October 12-14, 2000 Omni Inner Harbor Hotel Baltimore, MD

2. Introductions • Presenter- Marie L. Radford, Ph.D. Associate Professor, School of Information & Library Science Pratt Institute, 200 Willoughby Ave. Brooklyn, NY 11205-3897 mradford@prodigy.net (718) 636-3512 (o) (718) 636-3733 (fax) • Workshop Participants

3. AGENDA • Interviews • Types of interviews • Questioning types • Listening effectively • Advantages & Uses • Disadvantages • Planning & Conducting Interviews • Focus Groups • Advantages & Uses • Disadvantages • Planning & Conducting FGs • Analysis • Reporting Results • Limitations

4. INTERVIEWS What is an Interview? “An interview is a conversation involving two or more people, which is guided by a predetermined purpose.” (Lederman, 1996, p. 5)

5. TYPES of INTERVIEWS • Person-to-Person with Individuals • Telephone • Focus Groups

6. KEY COMPONENTS • Good Questions • Good Listening Skills • Good Interpersonal Skills

7. TYPES OF QUESTIONS • 4 valuable types for information gathering interviews • OPEN • “What is it like when you visit the library?” • DIRECTIVE • “What happened when you asked for help at the reference desk?” • REFLECTIVE • “It sounds like you had trouble with the online catalog?” • CLOSED • “Have I covered everything you wanted to say?”

8. NEUTRAL, LEADING, AND LOADED QUESTIONS • NEUTRAL • “What are your impressions of the M. L. Radford Memorial Library?” • LEADING • “You don’t like the librarians at the M. L. Radford Memorial Library, do you?” • LOADED • “How many other unreasonable requests have you made to the reference librarians?”

9. DO’S Be clear Be focused 1 idea per question Use specific questions Give sufficient time PRETEST! DON’TS Hypotheticals Wandering interviewee Too many choices Ambiguous or abstract questions Leading or loaded questions Questioning Do’s & Don’ts

10. DO’S Pay attention Listen actively Invite talk Open mind Supportive atmosphere Show acceptance & understanding Paraphrase Note NV cues DON’TS Rush to fill silences Give opinions Be defensive Forget to listen for feelings Hesitate to clarify Glaze over Effective ListeningDo’s & Don’ts

11. In-Person Interviews: Advantages & Uses • Face-2-face interaction • In-depth info. • Understand experiences & meanings • Highlight individual’s voice • Preliminary information to “triangulate” • Control of sampling • Greater range of topics

12. Telephone Interviews • 3 Types of Sampling 1. All who phone are asked to participate in a short interview immediately 2. Phoning users are asked to participate in a call-back interview 3. Random dialing of community members (samples non-users) • Disadvantages • Not all users have phones • Nonverbal cues missing • Follow-up may be difficult

13. Interviews: Disadvantages • Time Factors • Varies by # & depth • Transcription: 1.5 hrs. per hr. • Lots of prep. & administration • Cost Factors • Higher the #, higher the cost • Training interviewers • Tape purchase & transcription • Additional Factors • Self-report data • Errors in transcription or note taking possible

14. Library Applications • Academic • Reference Encounter • Public • Family Place Project

15. ACADEMICThe Reference Encounter(Radford, 1999) • Interviews & observation • 3 sites • Community College • Undergraduate College • Research University • Qualitative methodology to capture complexity & depth • 27 pairs of librarians/users • Analysis • Critical Incident • Paired Perceptions • Categories

16. PUBLICFamily Place Project(Middle Country Public Library, Centereach, NY & Libraries For the Future, NYC) • Interviews & questionnaire • 5 sites • 3 Urban • 1 Suburban • 1 Rural • Qualitative methodology to evaluate change process • Analysis • Critical Incident • Categories

17. Phase I: Planning • Determine purpose of interviews in overall assessment • Formulate Q’s & pretest • Identify: Interviewees &Interviewers (& train ‘em) • Choose notes or tape recorder • Decide when & where • Decide f-2-f or telephone • Plan schedule of interviews • Choose method of data analysis • Identify time & private place

18. Phase II: Conducting Interviews • Be on time & don’t waste their time • Obtain permission to use info. (report and/or publication) & if taping • If taping check equipment & have back-up • Create safe climate, assure confidentiality • Be prepared, flexible, & stay on task • Listen & know when to probe • Accept that some interviews won’t go well • Thank them!

19. Focus Group Interviews • What is a Focus Group? “A focus group is an in-depth, face-to-face interview of a group of 8 to 12 people representing some target group and centered on a single topic.” (Zweizig, Johnson, Robbins, & Besant, 1996)

20. Focus Groups: Advantages & Uses • Advantages of individual interviews plus… • Take less time • Creates synergy: • Comments stimulate others • Unexpected insights • More complete info. • Elicit strong & common opinions • Less inhibiting, less formal • Access needs of under-served or underrepresented groups • Control of sampling

21. Focus Groups: Disadvantages • Skilled moderator required • Participants may be too quiet or too outspoken • Analysis, summarization & interpretation of responses may be difficult • Loads of planning & administration time • Transcription of tapes is time consuming & costly

22. ACADEMIC Assessment of AV Delivery Service Undergraduate College Focus Groups of students, faculty, users and non-users of service Used to develop survey to evaluate and improve service PUBLIC Preparation for service excellence training Urban library system Focus Groups of staff (mixed librarians & non librarians) and users Used to develop survey to inform training Library Applications

23. Phase I: Planning • Determine role of focus group study in overall assessment • Identify: • target group(s) • moderator & assistant • Decide # of groups (3-4 groups per targeted group) • Develop interview guide (& pretest) • Plan schedule (1.5-2 hours each), when, & where • Choose notes or tape recorder • Choose method of data analysis • Con’t...

24. More Planning • RECRUIT VOLUNTEERS • Identify pool of potentials • Plan for 8-12 per group (over-recruit and remind) • Offer reward if possible (is not $, free ILLs, photocopying, food) • Try to get representative members • Form homogeneous groups: • Academic (e.g., faculty, undergrads, grad. students) • Public (e.g., adults, teenagers, non-users)

25. Moderator Characteristics • Excellent communication skills • Experience in group dynamics • Knows when to probe • Restrained • Ability to: • Involve all participants • Ask neutral questions • Quickly establish rapport • Summarize areas of agreement/disagreement • Direct but not control discussion

26. Phase II: Conducting Focus Groups • Be on time, don’t waste time • Bring/check supplies (flip chart, markers, masking tape, etc.) • Obtain permission to use info. (report and/or publication) & if taping • If taping check equipment & have back-up • Begin by creating safe climate • More...

27. Conducting Focus Groups Continued • Use effective listening • Help quiet people talk, limit talkative people • Design well constructed guide, including: • Introduction (purpose, ground rules) • Ice breaker or warm-up set of questions • Relevant major/probe questions • Summary or closing • Thank them!

28. Interviews & Focus GroupsPhase III: Analysis • Listen/transcribe tapes, review notes • Ways to analyze data: • Code data into pre-determined categories • Use data to identify categories • Use data as basis for summary statements “capture the essence” • Interpret the data- intensive analytic method • Compare all interviews/focus groups • Compile and summarize • Look for trends or problems to study • Do not overgeneralize from results!

29. Interviews & Focus GroupsPhase IV: Reporting Results • Parts of Report • Statement of purpose • List of issues purpose • Explanation of how data was collected and analyzed • Summary of findings: What was said on each issue • Sample quotes (anonymous) • Interviewer’s or moderator’s impressions • Recommendations • Short term (low hanging fruit) • Long term

30. Limitations:Interviews & Focus Groups • Difficult to generalize • May not be representative • Analyzes perceptions, not facts • Subjective analysis (reliability requires more than 1 person) • Raw data could be misleading • Results can be limited by: • Poor/inexperienced moderator/ interviewer • Poorly constructed discussion guide/interview questions • Compensate by combining methods (survey/questionnaires/ observation)

32. Summary & More Information • Where to get more information • List of books, articles, in packet.