Research Methods LIS 51 0

1. Research MethodsLIS 510 • October 20, 2009

2. What is Research? Who does it? Where? How? Why? Basis for your info worlds project, info behavior & info science in general

3. Research • Where does research happen? • University • Laboratory • Field – homes, schools, grocery stores • ?? • Who does research/what types? • Market • Historical • Experimental • ??

4. Doing Research (McGrath, 1994) Systematic use of some set of theoretical & empirical tools to try to increase our understanding of some set of phenomena or events In the social & behavioral sciences, the phenomena of interest involve states & actions of human systems—of individuals, groups, organizations, & larger social entities—& the by-product of those actions

5. Research Process - Main Features • Some content that is of interest (substantive domain) • Some ideas that give meaning to that content (conceptual domain-theory) • Some techniques or procedures by means of which those ideas and contents can be studies (methodological domain) WHY IS THIS FAMILIAR??? ALREADY BEEN DOING IT! YOU’RE ALL OLD HANDS AT IT NOW!

6. Research (Vogt, 1993) Basic (Ivory Tower/Academy): Primary goal - advancing knowledge & theoretical understanding of relations among variables Applied (Real World): Apply results to specific problem - effects of law enforcement methods on crime rates. RQs determined by policy makers et al., who want to help

7. Stages of the Research Process • Formulate a RQ or hypothesis (theory helps) • Connect it to reality (lit review, workplace) • (Seek $$ support) • Design the study – choose method(s) • Collect data (always takes longer…) • Analyze & interpret data (ditto) • Write-up findings & disseminate (“so what”) SIMPLE, RIGHT ?????????????

8. Types of Methods • Qualitative, Ethnographic, Naturalistic • Inductive • Working expectations • Generate theory • From particular  to general • Observation • Interviews • Focus groups • Journals or diary keeping • Think-aloud protocols • Case study • ???

9. Types of Methods, con’t • Quantitative (Deductive) • From the general to the particular • Hypotheses, theory driven • Surveys / questionnaires • Experiments – lab or field • Content analysis • Statistical analysis • Transaction log analysis • Experience sampling method • Bibliometrics (content & citation analysis)

10. Research Tenets • All methods are valuable: enable gathering of evidence in systematic way; yet, all have weaknesses/limitations • Determine what kinds of evidence we can gather but also what we cannot • Some are more difficult to use in particular situations than others • Time factors, Cost factors, Access factors • Multiple methods (Triangulation) allows the strength of some methods offset the weaknesses of others

12. Back to Naturalistic Inquiry • AKA “qualitative research,” “ethnographic research,” “constructivist inquiry” • Used synonymously for paradigm characterized as noncontrolling, humanistic, holistic, inductive, & (yet) scientific • Enables researchers to understand the relationship between thought & behavior by immersing themselves in the real drama of social life

13. Naturalistic Inquiry, cont’ • Primary data collection instrument is the researcher • Researchers can observe behavior, listen to what people say, & learn the rules for what it takes to think & act as a native might • Field researchers can observe what people do & then compare these observations & measures to what people say they do • Did that happen in the Info Ground work?

14. Researcher’s Notebook (VIR) • Field Notes: describe who, what, where, when & how; contain contextual data. • Method Notes: describe how method is working/be improved, suggestions for other techniques, etc. • Theory Notes: contain researcher’s ideas & Qs about why particular behaviors & activities are occurring. Later, re-written as extensive memos that connect thoughts on different phenomena & used for theory building emphasis placed on identifying negative cases/anomalies that refute/extend study’s theoretical framework.

15. Interviews: Different Types • Informal conversation: Spontaneous generation of Qs • General interview guide: Interview is guided by checklist of issues to be addressed • Standardized open-ended interview: Interview follows a list of Qs—all phrased the same for all participants • Advantages & Disadvantages of Each!

16. Interviewing: The Imp of Wording • Truly open-ended Qs: Let participants talk in their language as much as possible • No dichotomous Qs: No questions that require a “yes” or “no” response • Singular Qs: One question = one idea

17. Interviews – Wording, Cont’ • Clear Qs: Making sure the participant understands the Qs right away • Terms of art: Avoid using jargon, language of IB research; use everyday language instead • Neutral Qs: Interviewer stays completely neutral, no matter what the participant says • Rapport: Ability to convey empathy and understanding without judgment

18. The Interview Process: Recommended Procedure • Always start with an intro that includes: • Introducing yourself • Purpose of interview • Explaining what type of Qs you will ask • Estimating length of interview • Assuring participant that his/her identity will be anonymous

19. The Interview Process: Recommended Procedure, Cont’ • During interview, follow script & only ask Qs to elicit elaboration (but revise script as needed) • Thank participant at end of interview & ask if s/he would like to add anything (when best data come out)

20. Thinking of Qs… Critique the following interview Qs for their type and advantage/disadvantage: • Do you like to search the Web? • Is Firefox your favorite browser? • Do you access the Web at home or at school? • What cognitive processes do you go through?

21. Thinking of Qs, Cont’ • Why didn’t you print out this last page? • To what degree do professor’s teaching style and difficulty of the requests affect your relevance judgments? • From among the tasks you perform, which one do you like best? • Do you think that the Web is good for surfing but not for searching? • What are your favorite info sources?

22. Short on Time? • Use what you got! Collaborate! • Pretest, pretest; prize flexibility and beware Qs that lead to confounded, multiple replies (“what’s the reason you’re here/using this today?”) • Quick, great Qs: • Will you use [X] again? • Will you tell anyone about [X]? What? • The one-shot Q

23. Observation • Enables the researcher to discover the here-and-now interworkings of the environment via the use of the 5 human senses • Structured vs Unstructured • Participant vs Unobtrusive (different degrees of)

24. Observation Tips • Determine best way to conduct your observation (unobtrusively, as participant…) and prepare observation checklist based on your objectives/research purpose • Do homework: gather background info before arriving on scene. Ask who might be there and why. • Dress to fit in. Arrive early. Position yourself to best advantage.

25. Observation Tips, Cont’ • Draw lay-out of site, gather documents, etc. • Complete observation checklist: who is present & why (roles), activities are occurring, time of day, other contextual factors • Keep your note book and transcribe asap

26. Think Aloud Protocol • Gather user’s perceptions & cognitive processes when interacting with a system • User asked to elucidate upon what s/he thinking when accessing & viewing different screens or pages and form their impressions • Video/audio-record session; transaction log • Follow-up interview to review transaction log

27. Data Analysis • Data analyzed as collected by developing & revising codebooks based on units of analyses & emergent themes (*anomalies*) • Classic approach: • Glaser & Strauss's (1967) constant comparative where coding of data combined with generation of theoretical ideas • 3 stages: open coding, selective coding, axial coding

28. Data Analysis, Cont’ • Analyze all transcripts/notes using codebooks supplemented by re-writing of theory notes as comprehensive memos, diagramming or flowcharting concepts, etc., for theoretical connections • Inter-coder reliability tests (robustness) • Different software: N5, Atlas-ti, …

29. Validity & Reliability • Validity – is it accurate? • Reliability – is measurement the same or similar every time? ?? Possible to have reliability without validity In Qualitative Research, use “Trustworthiness” to gauge the quality of the research—see Naumer & Fisher paper

30. Ethics • IRB/Human Subjects Division • No harm should come to participants • Participants should not be deceived • Coercion should not be employed • Participation is voluntary • Data should be confidential -------------------------------------------- • Are electronic discussions private? • Should people be informed before studying their messages?

31. The Impact of Free Access to Computers & the Internet at Public Libraries http://cis.washington.edu/usimpact Institute of Museum & Library Services Bill & Melinda Gates Foundation

32. Introduction • Aim: Identify impacts of public access computing (PAC) through public libraries • Since 1984 public libraries have provided • Free access to the Internet & computers • Digital resources • Databases • Networked & virtual services • Training • Technical assistance “Access to computers and the Internet provided by public libraries, including “digital resources, databases, networked and virtual services … and to the training, technical assistance, and staff knowledgeable in technology services.” (IMLS RFP, 2007)

33. Public Library Distribution Central & Branch Libraries http://plgdb.freac.fsu.edu/imf.jsp?site=geolib ALUW Winter Quarter Meeting February 19, 2008 IMLS/UW Expert Committee Meeting 33

34. Library Statistics (NCES 2005) http://nces.ed.gov/pubs2008/2008301.pdf 9200 Public Libraries 290M People Served 1.4B Library Visits/yr 185K Public-use Internet Terminals 11 Terminals per Library 375M e-Resource Uses/yr

35. imPACt Project Goals • Develop & apply robust indicators demonstrating impact of PAC in public libraries to • Individuals, Families, Communities • Over short, medium & long-term • Document results from presence/absence of PAC at public libraries • Inform policy makers • Governmental • For-profit • Findings will support policy changes oriented toward increasing government/for-profit monetary and political support • Draft evaluation tools & techniques for future use

36. Telling the Story of 21st Century LibrariesIn Ways that Communicate with Policy Makers Largest study of U.S. public libraries Beyond Boxes: Computers, Internet, Training, Services 8 Domains: Civic Engagement eCommerce eBusiness Education eGovernment Health Employment Social Inclusion

37. Mixed Method Study • Nationwide telephone survey (N≈4,500 users & non-users) generated population estimates for different types of PAC users and ways they benefit across 8 domains • Nationwide online survey administered through public libraries (N≈45,000) used same instrument, supplementing it enabling analysis of user outcomes by library attributes • 4 case studies (300+ interviews) provided contextual influences on PAC outcomes and holistic understanding of how it impacts people’s lives

38. Data collection coverage Case studies: 4 libraries; 300+interviews Telephone survey: n=4,532 PAC users & non-users Web survey:400+ libraries; 45,000 completed surveys

39. Creating Public Value through Public Access Computing Moore, M. H. (1995). Creating public value: strategic management in government. Cambridge, Mass: Harvard University Press.

40. Last Words • Triangulate (methods, researchers, theory) • Use what you have! Keep it simple (KISS), collaborate • Research is always dirty; rarely goes as planned • Pretest • Keep good notes • Have fun!