1 / 50

IS 4800 Empirical Research Methods for Information Science Class Notes April 4, 2012

This class notes provide an overview of different research methods including quantitative, qualitative, and mixed methods approaches, along with steps in the research process and examples of qualitative studies.

Download Presentation

IS 4800 Empirical Research Methods for Information Science Class Notes April 4, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. IS 4800 Empirical Research Methods for Information Science Class Notes April 4, 2012

  2. Overview of Research Methods • quantitative • Descriptive • demonstration • correlational • experiments • qualitative • ethnography • case studies • mixed

  3. Steps in the research process • Identify a phenomenon of interest • Iterate: • Investigate current state of knowledge (lit. review ?) • Narrow down your interest to a research question or hypothesis • Identify research method to employ (survey, experiment, ethnography, case study)

  4. Steps in the research process (cont.) • Operationalize research question or hypothesis • Define the source of your data • Sample population and recruitment method (if relevant) • And/or the location/activities to be observed • Define variables and/or data collection methods and instruments • Identify how the analysis will be carried out ----YOU NOW HAVE A RESEARCH PROPOSAL ------

  5. Steps in the research process (cont.) • Carry out your observations • Analyze your data • Draw conclusions, write up the results Example: Freshman retention – a major concern of every University !! A quantitative study: midterm exam question A qualitative study (we will discuss that)

  6. Quantitative vs. Qualitative • Goals • Kinds of data • Data collection methods • Kinds of analyses • Kinds of explanations • When used in system development lifecycle?

  7. Goals of qualitative research • Identify/compare patterns of behavior • Analyze beliefs/subjective experience • How people view a situation or experience • Distinguish multiple perspectives • Compare beliefs/self-reports to actual behavior • DESCRIBE a complex phenomenon • EXPLAIN why things happen

  8. Examples of qualitative studies Case studies of failed system development or deployment Ethnographic studies of workgroup practices prior to introduction of new technology 8

  9. Some qualitative methods in brief • Case study research • Explore in depth one activity or project • Limited in time and place • Data collection by observation, interviews, artifacts . . . • Goal: tell a coherent “story” with lessons learned • Most common methodology for IT empirical studies • Ethnography • Observation in natural setting • Observer may become part of the group to experience directly how its members interact • Goal: identify patterns of interaction (power structures, problem-solving/goal achievement) 9

  10. Some qualitative methods (cont.) • Grounded theory research • Data collection from artifacts and/or interviews • Develop a set of categories and a model telling how they relate to each other • Goal: explain the meaning of what is observed • Involves an bottom-up iterative process of data collection/theory formation 10

  11. Advantages/disadvantages of quantitative studies + Systematic rules and procedures already worked out, and can be followed + Traditional, accepted as “proof” - Closed-ended questions may lead to ignoring important factors and relationships - Quantitative methods cannot handle phenomena that are difficult to turn into variables

  12. Advantages/disadvantages of qualitative methods + more innovative and creative + capable of addressing issues that do not lend themselves to being described by variables - conclusions may be less credible Lecture 1 - Introduction 12

  13. Mixed Methods Pragmatic philosophy – find out whatever you can using whatever methods are possible Involves both qualitative and quantitative elements  (at least 2 stages of research) Advantages/Disadvantages + combines structure and flexibility - requires more time and resources 13

  14. Examples of Mixed Method Designs Pattern 1: “instrument” data followed by in-depth interview to get insight on the reasons for the observed relationships and capture any insights you overlooked in study design Pattern 2: exploratory study followed by survey or experiment to generalize the results – representative of a long-term research program 14

  15. Applications of the Methods Which Method would you choose? • Studies of computer/supported learning • Studies of IT impacts in medicine • Computer-supported collaboration (in general) • Case study or ethnography for groupware • Grounded theory study of chat groups

  16. Goals of qualitative research • Identify/compare patterns of behavior • Analyze beliefs/subjective experience • How people view a situation or experience • Distinguish multiple perspectives • Compare beliefs/self-reports to actual behavior • Ideally, propose a model to describe or explain

  17. What is a model ??What counts as evidence ??Elements/types of qualitative models • Define a taxonomy – useful aggregation of data • Rogers’ adopter categories • Innovators • Early adopters • Early majority • Late majority • Laggards

  18. Elements/types of qualitative models • Define properties that affect outcomes • Perceived attributes of innovation that help explain their different rates of adoption • Relative advantage • Compatibility • Complexity • Trialability • Observability • Explains individual’s likelihood of adopting

  19. Elements/types of qualitative models • Identify a relationship of interest • Rogers: homophily/heterophily and diffusion • Homophily is the degree to which two or more individuals who interact are similar in certain attributes such as beliefs, education, socioeconomic status, etc. • Although homophily enables better communication, innovation requires heterophily, at least regarding knowledge of and experience with an innovation, and it is likely that heterophily in that area co-occurs with other important differences. • This explains why some groups are slower to adopt new behaviors

  20. Elements/types of qualitative models • Characterizing a process as a sequence of steps • Rogers: the innovation/diffusion process • Knowledge • Persuasion • Decision • Implementation • Confirmation • Characterizing the “shape” of a process • Rogers: the S-shaped curve

  21. Collecting data for quantitative studies Identify/define the variables (and their coding) Design the “instrument” – a measurement process or technique Types of instruments: a survey (paper, phone, Web) a form for observer to fill in (experiment or field study) a prototype system (with interaction capture) Field testing/validating instruments part of quantitative methods 21

  22. Qualitative Data and Collection Methods • Direct observation • Participant observation • In-depth interviews • Focus groups • “Artifacts” – usually text or Databases

  23. Direct Observation • May be in person or use audio or videotape, observe through a 1-way mirror • Unlike participant observation, often focused on specific events (how many, how often, by whom, observe patterns – for example, interruptions at a meeting)

  24. What to observe • Spatial relations • Activities • Communication • Verbal • Other • Tasks • How work is allocated

  25. How to be an effective observer • Preparation • Stay in the background • Be factual and objective in your notes • (interpretation comes later) • Taking notes: • Hand written usually • Type in to computer later • EXPANDING NOTES (ASAP)

  26. “(Participant) observation”: in natural setting • “Participant” observation occurs when you interact casually and/or form relationships with informants • How much you actually “participate” depends on the goals of the study.

  27. Participant observation • Advantages: • Offers insights into complex behavior • Identify the “right questions” for further study • Verify/correct self-reports • Disadvantage: • Time consuming • Data collection is difficult • Problem of subjectivity

  28. How to operationalize • Field notes • Text • Diagrams, maps • Can result in numerical data • Interviews (interviewer more clueful) • Focus groups (facilitator more clueful)

  29. What to observe • Spatial relations • Activities • Communication • Verbal • Other • Tasks • How work is allocated • See Table 3 in reading

  30. Ethics • Do not disrupt the activity your are observing versus • Do not mislead • No formal rules about disclosing your role as a researcher when engaging in casual conversation – article suggests a point where you want to ask specific question • Disclosure includes: right of refusal, confidentiality

  31. Protecting confidentiality when data is unique • Separate identify info from field notes entered into the computer • People, organizations/companies, should be given fictitious names

  32. How to be an effective observer • Preparation • Stay in the background • Be factual and objective in your notes • (interpretation comes later) • Taking notes: • Hand written usually • Type in to computer later • EXPANDING NOTES

  33. Tips • Leave space • Take notes strategically • Use abbreviations • Cover a range of observations: Body language, etc.

  34. Tips • Leave space • Take notes strategically • Use abbreviations • Cover a range of observations: Body language, etc.

  35. Participant observation: in natural setting • “Participant” observation occurs when you interact and/or form relationships with informants • Demanding and time-consuming • How much you actually “participate” depends on the goals of the study. • Subjects may “forget” you are a researcher

  36. How to operationalize direct/participant observation • Field notes • Text • Diagrams, maps • Can result in numerical data • Interviews (interviewer more clueful in P.O.) • Focus groups (facilitator more clueful in P.O.) “Water cooler” effect

  37. Participant observation • Advantages: • Offers insights into complex behavior • Identify the “right questions” for further study • Verify/correct self-reports • Disadvantage: • Time consuming • Data collection is difficult • Problem of subjectivity

  38. Ethics of direct observation/ participant observation • Do not disrupt the activity your are observing versus • Do not mislead • No formal rules about disclosing your role as a researcher when engaging in casual conversation – some authors suggest a point where you want to ask specific question • Disclosure includes: right of refusal, confidentiality

  39. Protecting confidentiality when data is unique • Separate identify info from field notes entered into the computer • People, organizations/companies, should be given fictitious names

  40. In-depth interview/focus group • Probes the interviewee(s) views of the phenomenon of interest • Interviewer/facilitator should be neutral • Data collected: transcript, audio/video recording, notes

  41. In-depth interview/focus group Interviewer should: • Start with some open-ended questions • Follow up by asking “how” and “why” • Keep the discussion on track

  42. Documents • Memos and meeting notes • Transcripts of conversations or speeches • Manuals and policy handbooks • Newspapers and magazines • Internet-based research • Email • Web sites • Blogs • Especially important in case studies

  43. Qualitative Data Analysis by John V. Seidel • Description of how to go about analyzing transcripts of interviews, documents, and/or field notes. • Focus on “coding” • First identify “events” • Assign terms that represent concepts of interest • Organizing codes into a scheme • Building qualitative models using the coding scheme as the model vocabulary • Focus on iterative nature of QDA

  44. Two perspectives on coding • Objectivist perspective • Condensed representation of facts • Can be subjected to hypothesis testing • Strong burden of consistency/completeness • Heuristic perspective • Signposts pointing to things you care about • Foundation for further analysis

  45. Elements/types of qualitative models • Examples from Rogers’ theory of innovation diffusion • VCR’s • Cell phones • Metric system • Seat belts in cars • Dvorak keyboard

  46. Three analogies to explain this • Jigsaw puzzle analogy • A little data and a lot of right brain • Multi-threaded DNA (patterns among the patterns)

More Related