Focus Groups as Assessment Tools

1. Focus Groups as Assessment Tools Marie L. Radford, Ph.D., Associate Professor, Rutgers University mradford@scils.rutgers.edu LAMA/MAESUsing Measurement Data for Library Planning & Assessment Committee “Got Data, Now What? Analyzing Focus Groups and Usability Study Results” ALA, Chicago, June 26, 2005

2. Participants • How many have conducted FG & have data to analyze? • How many are planning FG? • How many are considering FG? • How many are skeptical of value of FG?

3. AGENDA • Phase 1: Planning • Phase 2: Conducting Focus Groups • Phase 3: Analysis of Focus Group Data • Phase 4: Reporting Results • Using Focus Group Results in Strategic Planning

4. What is a Focus Group? “A focus group is an in-depth, face-to-face interview of a group of 8 to 12 people representing some target group and centered on a single topic.” (Zweizig, Johnson, Robbins, & Besant, 1996)

5. Advantages of Using FG for Strategic Planning • In-depth understanding of experiences & meanings. • Better data – synergy created: • Comments stimulate others • Unexpected insights • More complete info. • Elicit strong & common opinions • Less inhibiting, less formal. • Access needs of under-served or underrepresented groups.

6. Disadvantages • Lots of planning & administration time! Skilled moderator required • Participants may be too quiet or too outspoken • Analysis, summarization & interpretation of responses may be difficult • Tape transcription - time consuming & costly

7. Limitations • Difficult to generalize • May not be representative • Analyzes perceptions, not facts • Subjective analysis (reliability requires more than 1 person) • Compensate by combining methods (surveys/observation)

8. Phase I: Planning • Crucial phase! • Communicate to stakeholders • Determine role of FG in overall assessment • What info. is needed? • Identify key issues • Choose moderator & assistant (see handout)

9. More Planning • Identify target group(s) • Decide # of FG, 3-4 per targeted population • Identify pool • Recruit volunteers • Plan 8-12 per group (over-recruit & remind) • Offer reward (e.g., free copycard, food) • Seek representative members • Form homogeneous groups: • Academic (e.g., faculty, undergrads, grad. students) • Public (e.g., adults, teenagers, non-users)

10. Yet More Planning… • Design FG guide, include: • Introduction (purpose, ground rules) • Ice breaker or warm-up set of questions • Relevant major/probe questions (4-5) • Summary or closing • Consider modified FG (see handout) • Plan schedule (1.5-2.5 hours each) • When & Where • Choose notes or tape recorder • Choose method of data analysis

11. Phase II: Conducting Focus Groups • Bring/check supplies • Flip chart, markers, masking tape, etc. • If taping check equipment & have back-up • Start on time, don’t waste time • Begin by creating safe climate • Obtain permission to use info. & if taping • Help quiet people talk, limit talkative people • Thank them!

12. Phase III: Analysis • Listen/transcribe tapes • Review notes • Ways to analyze data: • Use data to identify categories • Repeated reading/listening • Code data into pre-determined categories • Use data as basis for summary statements “capture the essence” • Modified FG already identifies priorities

13. Analysis, Cont. • Compare across focus groups • Compile & summarize • Look for trends or problems to study • Do not overgeneralize from results!

14. Phase IV: Reporting Results • Parts of Report • Bulleted executive summary • Statement of purpose • List of issues • Method (keep it brief!) • How data collected • How analyzed • Describe participants

15. Parts of Report, Cont. • Summary of findings: • What said each issue • Consensus? • Comparison of groups • Sample quotes (anonymous) • Few, well chosen, representative • Interviewer’s impressions • Recommendations: • Short term (low hanging fruit) • Long term

16. Library Strategic Planning Examples • Academic • Rutgers University Libraries • Major part of strategic planning initiative. • Focus groups of students & faculty. • 3 Campuses. • 3 Moderators, 3 Assistants. • Recruitment difficulties. • Results extremely useful.

17. Brooklyn Public Visual & Performing Arts Library Inform building project. Large urban library system. FG of community groups, artists, art educators. Report used in informing decision making, fund raising, & strategic planning. Public Library Example

18. Presenting to Administrators • For written or oral presentation • Brevity is important • Put your best foot forward! Highlight: • Positive results • Strategic directions • Major concerns • Begin & end with quotation • Bulleted executive summary may be all that is read (!)

19. Using Results for Strategic Planning • Allow sufficient time to review results • Use missions, goals for guidance • Look at short & long term recommendations • Prioritize recommendations

20. Using Results Cont. • Gain consensus • Establish task forces or assign responsibilities • Establish objectives, target dates • Establish regular (6 mo. or annual) review process • Create assessment plan

22. For More Information… • See list of books & articles. • Questions?? • Thank you!