MIS 650 Project Seminar

1. MIS 650Project Seminar Qualitative Methods Introduction to Qualitative methods and Grounded Theory MIS 650 Qualitative Research

2. Dept. of Information Systems Doctoral Program in IS & ND “Qualitative” Methods • Differ from Quantitative methods in that the data used is not numerical measures but words; the goal is often not to prove relationships but to derive arguments that relationships exist • Rely on interpretation • Such methods do not depend on amassing large samples, using statistics, or computing or counting things, but assembling arguments, examples, cases, etc. MIS 650 Qualitative Research

3. Dept. of Information Systems Doctoral Program in IS & ND “Qualitative” Methods’ Agenda: Increasing the Quality of the Discourse One of the goals is to increase our knowledge of the world through an indirect means: by increasing the quality of the discourse that is essential to knowledge generation. The reasoning is thus: Better (more, more relevant, more useful) knowledge requires better theories. These theories arise from the crucible of discussion, debate and discourse. Improving the raw material of this discourse means improving the process of creating theories; theories arise from our experiences, which are socially constructed. MIS 650 Qualitative Research

4. Dept. of Information Systems Doctoral Program in IS & ND Increasing the Quality of the Discourse Impressions, Thoughts, Felt Ideas Debate, Discussion Dialectic Discourse Expressed Ideas Essential Criticism Higher Quality Ideas Critical Commentary Experience MIS 650 Qualitative Research

5. Dept. of Information Systems Doctoral Program in IS & ND The Flow… Stage Objects Criteria Idea Theory Testing Creation Development of Theories Concepts Arguments Data, Methods Good Ideas Good Arguments Correct Theories Grounding Theory Strength- Correspondence ening to Reality MIS 650 Qualitative Research

6. Dept. of Information Systems Doctoral Program in IS & ND The Flow… Ideas arise from researchers’ and others’ experiences. The key is to have useful experiences. The resultant ideas are said to be “ground- ed” in these experiences. We collect case examples of the concepts and thoughts we want to think about in or- der to express these thoughts better. At this stage what we want is a wide variety of experiences to be able to ex- press ourselves better. Idea Creation Concepts Good Ideas Grounding Stage Objects Criteria MIS 650 Qualitative Research

7. Dept. of Information Systems Doctoral Program in IS & ND The Flow… At this stage What we desire is to have “mean- ingful” dis- cussions that produce robust argu- ments. We do this by (1) exposing the ideas to the strongest non- superficial tests we can find. (2) Structuring the debate to cover as much as possible. Stage Objects Criteria Theory Development Arguments Good Arguments Theory Strength- ening MIS 650 Qualitative Research

8. Dept. of Information Systems Doctoral Program in IS & ND The Flow… Testing of Theories Data, Methods Correct Theories Correspondence to Reality Only when we have strong ideas with reasonable argu- ments can we begin to pro- pose tests of correspondence of those ideas to the real world. It is only at this stage that the expensive proof procedure is worth putting into play. Stage Objects Criteria MIS 650 Qualitative Research

9. Imaginings about this or other external or internal worlds Dept. of Information Systems Doctoral Program in IS & ND Basis of Qualitative Methods Comments or interpretations or translations of this or other worlds Positivism assumes the world as an objective reality to be discovered, measured and thus understood. Actual Events Interpretivism assumes the world is socially (or otherwise) constructed and this must be understood first and only later measured and discovered MIS 650 Qualitative Research

10. Dept. of Information Systems Doctoral Program in IS & ND Challenges of Qualitative Methods • Data are complex, multifaceted • Analysis is strongly influenced by data type and, in fact, may be contemporaneous with data collection • Not the same as “easy” quantitative methods • The search is for “meaning” • Because the data are words, they (the data) may possibly “speak for themselves” (but not always!) MIS 650 Qualitative Research

11. Dept. of Information Systems Doctoral Program in IS & ND Where/When to Use Qualitative Methods • Case Studies create and expose a lot of qualitative data • Content analysis uses “content” (i.e., words) • Grounded theory, to create theory • Where the words of research sources can serve as examples, cases, or insights • Where the researcher must “interact” with the data to derive hypotheses or conclusions MIS 650 Qualitative Research

12. Dept. of Information Systems Doctoral Program in IS & ND Uses of Qualitative Methods • To determine the mental processes of users of IT in a variety of cultural settings • To understand the value of IT use or IT artefacts to members of different cultures • To understand how people fit IT into their daily lives in meaningful ways • To see what strategies people engage in to understand IT themselves MIS 650 Qualitative Research

13. Dept. of Information Systems Doctoral Program in IS & ND Dangers of Qualitative Data • “Thin” case studies • Treating words as content, expression or naïve theory as data • Drawing conclusions of a statistical nature based on small samples • Researcher-driven conclusions (i.e., interpreting the researcher rather than the data sources’ inner states). MIS 650 Qualitative Research

14. Dept. of Information Systems Doctoral Program in IS & ND Reliability and Validity Issues • Source: Kirk, Jerome and Marc Miller Reliability and Validity in Qualitative Research. Newbury Park: Sage, 1983. • What reality is being probed? • What does reliability mean with samples of 1? • What is an appropriate test of validity for idiosyncratic experience? • Different sub-disciplines face these issues differently MIS 650 Qualitative Research

15. Dept. of Information Systems Doctoral Program in IS & ND Reliability and Validity Issues • Validity is really a challenge making certain that you have the right name for what you are observing. • Reliability is the serious scientific problem. • There are three sorts of reliability: quixotic: a single method of observation yields an unvarying measurement diachronic: observation stability through time (test-retest) synchronic: similarity of observations in same time period (split halves, inter-rater) MIS 650 Qualitative Research

16. Dept. of Information Systems Doctoral Program in IS & ND Meeting Reliability Challenges • Document your procedure • Tell how decisions involving the research -- and in particular data gathering and recording -- were made • Describe how the researcher was prepared • Sources of reliability: Bias (researcher effects; surface biases, reactivity of insiders) Replication (a real problem; see above; strive for disconfirmation) Specification (context, conditions of research) MIS 650 Qualitative Research

17. Dept. of Information Systems Doctoral Program in IS & ND Two Examples of Qualitative Methods • Narrative Analysis: understanding individuals and their times through stories that they tell • The Ethnographic Method: understanding people and their ethos through a study of their culture They are similar in their goals and methods but focus on different aspects of people. Both face problems with validity and reliability and meet the challenge in different ways. MIS 650 Qualitative Research

18. Dept. of Information Systems Doctoral Program in IS & ND Qualitative Method Example: Narrative Analysis • Source: Catherine Kohler Riessman, Narrative Analysis, Newbury Park: Sage, 1993 The analysis of stories, first-person accounts of personal experience: “talk organised around consequential events.” A teller in a conversation takes a listener into a past time or “world” and recapitulates what happened then to make a point, often a moral one. MIS 650 Qualitative Research

19. Dept. of Information Systems Doctoral Program in IS & ND Qualitative Method Example: Narrative Analysis How people make sense of their world The analysis of stories, first-person accounts of personal experience: “talk organised around consequential events. A teller in a conversation takes a listener into a past time or ‘world’ and recapitulates what happened then to make a point, often a moral one. An individual’s data The experience is real for that person The events are important to the teller The events behave according to a naïve “physics”, “sociology”, etc. that tells about the teller The teller is aware of the audience There is a structure to the story that is important MIS 650 Qualitative Research

20. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis Issues: 1. Realists take issue with correspondence with “truth” 2. Related to other textual approaches such as semiotics, hermeneutics, discourse analysis, deconstructionism 3. Highly concerned with “how protagonists interpret things” 4. Highly subjective. MIS 650 Qualitative Research

21. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis: Research Model Reading(5) Analyzing(4) Transcribing (3) Telling (2) Attending(1) Primary Experience MIS 650 Qualitative Research

22. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis: Research Model Reading(5) Retelling as the agent of the text, reinterpretation into other frameworks Analyzing(4) Creation of a “metastory”, a “false document” Transcribing (3) Representation, fixing, culling, loss of information Telling (2) Performance, description, organisation, taking into account cultural context, interaction, dialogue Attending(1) Reflect, remember, recollect into observations, choose from primary experience MIS 650 Qualitative Research

23. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis: Technique What is a narrative (…”and then what happened?”); expectations of listeners are cultural and there are brackets; there are many different kinds of structures (not an exhaustive list, just exhausting!): Labov’s paradigms: All narratives have six elements: abstract, orientation, complicating action, evaluation, resolution, coda. Burke’s dramatism:Act (what), scene (when/where), agent (who), agency (how done), purpose (why). Gee’s poetics: poetic units, stanzas, strophes, punctuation. MIS 650 Qualitative Research

24. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis: Truth Narrative is a “root metaphor” (Sarbin): “it is an achievement that brings together mundane facts and fantastic creations; time and place are incorporated.” Narrative can 1. Recreate experience via order 2. Constitute reality: telling makes things real 3. Provide plans or ideologies to live by Narratives don’t speak for themselves; they speak about the teller. We aim not for TRUTH but for “believability, enlargement of understanding rather than control” MIS 650 Qualitative Research

25. Dept. of Information Systems Doctoral Program in IS & ND Narrative Analysis: Validation • Historical truth of narrative is NOT primary interest; we aim for trustworthiness. There are four criteria: • Persuasiveness, plausibility (subject to rhetorical style) • Correspondence to informants’ experience (expect problems as narratives are movable feasts) • Coherence (global=relative to narrator’s goals; local=internal structural consistency; themal=content) • Pragmatic use by others in the future (!) MIS 650 Qualitative Research

26. Dept. of Information Systems Doctoral Program in IS & ND Qualitative Method Example: Ethnographic Method • Source: Stewart, Alex, The Ethnographer’s Method, Newbury Park: Sage, 1998 Characteristics of the method: 1. Participant Observation 2. Holism: researcher constructions “society” and the data have “breadth” 3. Context Sensitivity: comprehensive data 4. Socio-cultural description (social relations) 5. Theoretical Connections to Anthropology MIS 650 Qualitative Research

27. Dept. of Information Systems Doctoral Program in IS & ND Mapping to “Conventional Values” Epistemic Value Conventional equivalent Underlying question Research pro-cess challenge Veracity Validity (exclu- ding external) Verisimilitude of depiction Field conditions, researcher’s constraints Objectivity Transcendence of perspectives Reliability (exclu- ding consistency Context sensitivity Reactivity,lack of res’rch context Perspicacity Generalizability, external validity Applicability of insights elsewhere Inability to create insights, invalid taxonomies MIS 650 Qualitative Research

28. Dept. of Information Systems Doctoral Program in IS & ND Research Coping Tactics Veracity Objectivity Perspicacity 1. Prolonged fieldwork 2. Search for discon- firming observations 3. Good participative role relationships 4. Attentiveness to context 5. Multiple modes of data collection 1. Trail of Ethno- grapher’s path* 2. Respondent validation 3. Feedback from outsiders 4. Interrater checks on indexing, coding 5. Comprehensive data archive 1. Intense consider- ation 2. Exploration Very helpfulquestionable use * “Dance card”, network, path MIS 650 Qualitative Research

29. Dept. of Information Systems Doctoral Program in IS & ND Four Phases of Qualitative Research Invention: ”Getting in and Getting Along”--Finding the field Copping direction (networking), copping a look (first viewing), copping a taste (early episodes of non-intrusive assessment) Discovery”Getting Data” -- Collecting the data Scoring a chance (permission), scoring the facts (gathering data, separating data from noise, discovering new facts) Interpretation“Getting it straight” --reading the field Checking the validity; checking the reliability (strength of the data) Explanation: “Getting out and getting even” -- settling accounts Splitting up (severing relationships), splitting the take (negotiating costs and benefits with studied group), splitting the scene. MIS 650 Qualitative Research

30. Dept. of Information Systems Doctoral Program in IS & ND Qualitative Research: Summary GoalsMethodsResults Understanding Enlightenment Creativity Divergence Exploration Intense interaction Personal Exploration Involvement Observation Comparison “Thick” description Teaching, learning “Trail of experience” Lots of data on small amounts of experience “Qualitative research is defined by the location of hypothesis-testing activity in the discovery, rather than the interpretation phase.” --Kirk and Miller MIS 650 Qualitative Research

31. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory • Why “ground” your research • You might want to create a new direction for research • Theory might be restricting your “search” • The “reality” in question might be in question! • This might be part of “action research” • What methods are used in Grounded Theory • Open coding, axial coding, selective coding • Where are you when you finish? • You have a “prototheory” MIS 650 Qualitative Research

32. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Philosophy Meanings Meanings are important (perhaps paramount or even the entirety of the theory) Theory should fit the reality it is meant to explain, i.e., be “grounded” in that reality. Theory is meant to do more than explain; it is to guide action. Reality Action/ Intervention MIS 650 Qualitative Research

33. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Open Coding • Disaggregate data into conceptual, labeled units • Compare derived code labels and group them into larger categories • Avoid existing sets of labels to avoid contamination and counter-interpretation effects • You are looking for significant themes. MIS 650 Qualitative Research

34. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Open Coding • Disaggregate data into conceptual, labeled units • Paragraphs, sentences, phrases, fragments • Compare derived code labels and group them into larger categories • Categories are “from the text itself” • Avoid existing sets of labels to avoid contamination and counter-interpretation effects • You are not “proving” the categories’ validity, just trying to establish them. • You are looking for significant themes. MIS 650 Qualitative Research

35. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Axial Coding • Look for relationships among various categories • Rearrange this into a hierarchy and allow subcategories to emerge • Identify what is happening and why, explore the context for events, segregate causes and outcomes of campaigns, and seek co-occurrences. • Verify derived elements as hypotheses to be verified with later sources; seek both positive as well as negative cases or episodes MIS 650 Qualitative Research

36. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Axial Coding • Look for relationships among various categories • You are attempting to build a theory-generator here C1 C2 C3 C4 C5 C6 C7 C8 C9 C1 C2 C3 C4 C5 C6 C7 C8 C9 Precedes, Causes, Depends upon, Is necessary for, Is incompatible with, Is part of, etc. MIS 650 Qualitative Research

37. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Axial Coding • Rearrange this into a hierarchy and allow subcategories to emerge (possibly networks) MIS 650 Qualitative Research

38. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Axial Coding • Identify what is happening and why, explore the context for events, segregate causes and outcomes of campaigns, and seek co-occurrences. • Verify derived elements as hypotheses to be verified with later sources; seek both positive as well as negative cases or episodes Examples, counterexamples MIS 650 Qualitative Research

39. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory-Techniques: Selective Coding • Derive the new theory based on the major relationships among categories • Constantly compare incoming data with existing categories and data (constant comparison) • Sampling note: You are not seeking “representative sampling of respondents or informants”; you are seeking broad representation of ideas, meanings, thoughts, and expressions to let them emerge. Thus rather than finding typical respondents, you are seeking to have as broad a range of respondents as possible. MIS 650 Qualitative Research

40. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: References From ISWorldNet (http://www.isworld.org/isworld/isworldtext.htm) http://www.aukland.ac.nz/msis/isworld/grndrefs.htm Basic/Classic: Glaser and Strauss (1967), Strauss & Corbin (1990), Glaser (1992) [these latter two differ about what GT is] IS Classic: Orlikowski, Wanda. “Case Tools are organizational Change: Investigating Incremental and Radical Changes in Systems Development,” MIS Quarterly, 17 (3): 309-340, 1993 Pandit, Maresh R. “The Creation of Theory: A Recent Application of the Grounded Theory Method,” The Qualitative Report, 2(4), 1996. (http://www.nova.edu/ssss/QR/QR2-4/pandit.html) MIS 650 Qualitative Research

41. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Example A doctoral student is interested in what how users of CASE (Computer Aided Software Engineering) actually use CASE in their work. She decides that “use” and “meaning of use” are highly related and uses Grounded Theory to build her theory. She begins with simple questions asked of a few referred users; questions are such as “How do you use CASE?” “Why do you use it that way?” “How did you learn to use it like that?” Note the generality of the questions Is she using a pre-existing theory that is being tested?How is her initial sample located?What does she do with the answers to the questions? MIS 650 Qualitative Research

42. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Example (2) After the first set of interviews, she uses open coding to create a series of fragments such as “Took a course”, “Learned from my colleagues”, “I just copy everyone around here”, “The course wasn’t very good.” Then she categorized the comments (axial coding) into a set of categories and generated a new series of (finer) questions about training, policy, and style. Then she selected other users who were different from the first set (used a different tool in a different firm) and asked these questions of them. What is the purpose of open coding? What is the alternative? Why did she use fragments as opposed to paragraphs or sentences? What else can she do with open coded fragments?What is the purpose of categorization?Why is she seeking different kinds of users? What happens to reliability? MIS 650 Qualitative Research

43. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Example (3) These users’ answers were openly coded to create more fragments (on cards) and as the number of cards increased, she established a table relating the fragments (now categories) to one another in pairs or more complicated structures. Why is she creating a “database” of fragments? Why have so many?What is the purpose of the table? What kinds of structures is she looking for? MIS 650 Qualitative Research

44. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Example (4) Gradually a set of structures arose that indicated to her the varieties of use of CASE tools. In some cases, the use was motivated by others; in some cases the users created their own use. But it turned out in all cases that the uses were not the set of uses for which CASE was intended! What kind of picture is emerging? How can we say that is “theory”? Whose theory is it?What is her major finding?Why would a more positivist approach been more difficult? MIS 650 Qualitative Research

45. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Example (5) Using selective coding and constant comparison, as the set of respondents grew (and as the number of contrasted sources of information from the respondents grew), she became more convinced that the categories she derived had good reason to exist in the causal structure she was watching emerge. However, at no time did she try to prove her emerging theory. Why is she “constantly comparing”? What is her goal here? How does she know how many respondents to interview?What makes her convinced about her causal structure?Why doesn’t she use this method to prove any theory? MIS 650 Qualitative Research

46. Dept. of Information Systems Doctoral Program in IS & ND Grounded Theory: Diagnostic Bill is attempting to discover theory about why/how NPOs employ IS consultants. He has noted how NPOs differ from “normal” businesses and has a theory about normal business. Using this theory, he generates a series of questions which he asks of a few NPO managers. Based on these answers, he creates a series of categories and notes that the categories are very similar to those of normal businesses. He then asks for referrals to other NPOs and uses these categories to see if the other NPOs are in fact similar. When he sees they are, he starts to ask questions of a causal nature (“Do you think that X causes you to use consultants of a certain type in a certain way” and in this way builds up a theory. MIS 650 Qualitative Research

47. Dept. of Information Systems Doctoral Program in IS & ND Action Research - Example* Results of research into collaborative technologies (such as E-mail) have had inconclusive or contradictory results. In this study, 107 people in two NZ organisations collaborated in twelve groups over 10 to 45 days over email-based systems. Participants were interviewed and their messages were analysed for content. Consistent with a social influence model, participants felt that the e-mail intervention increased individual knowledge learning about other departments and departmental heterogeneity. * Koch, Ned. “Sharing Interdepartmental Knowledge using Collaboration Technologies: Action Research Study”, Journal of Information Technology Impact, 2(1): 5-10, 2000 MIS 650 Qualitative Research

48. Dept. of Information Systems Doctoral Program in IS & ND Action Research - Example (2) Note the elements of action research: (1) There was a technological intervention that was actually employed by working groups (2) The groups were engaged in “process improvement tasks” (3) The researcher actually provided the mediation (I.e., was part of the system). * Koch, Ned. “Sharing Interdepartmental Knowledge using Collaboration Technologies: Action Research Study”, Journal of Information Technology Impact, 2(1): 5-10, 2000 MIS 650 Qualitative Research