Qualitative Document Analysis

1. Qualitative Document Analysis In this session, I adopt a rather eclectic view of `document’. In addition to typical sources (e.g. media reports, government papers, minutes of meetings, company reports), I include documents that are read as part of the literature review and also the working documents that become your thesis. My rationale for this is that similar issues and skills are involved in the `analysis’ of all of them. Hugh Willmott Research Professor in Organizational Analysis Cardiff Business School Home Page : http://dspace.dial.pipex.com/town/close/hr22/hcwhome

2. Outline of Session • Analysing secondary sources • Analysing literature • Analysing your text Textual Analysis. `Textual analysis involves mediation between the frame of reference of the researcher and those who produced the text. The aim of this dialogue is to move within the “hermeneutic circle” in which we comprehend a text by understanding that frame of reference from which it was produced, and appreciate that frame of reference by understanding the text. The researcher’s frame of reference becomes the spring board from which the circle is entered, and so the circle reaches back to encompass the dialogue between the researcher and the text’ (J. Scott (1990), A Matter of Record: Documentary Sources in Social Research, Cambridge: Polity cited in Mason, 2002, p. 110) Exercise: Consider how reading a journal article on your research topic illustrates the idea of `mediation’ and `dialogue’

3. Forms of Textual Analysis – Quantitative and Qualitative `Quantitative Researchers try to analyse written material in a way that will produce reliable evidence about a large sample. Their favoured method is content analysis in which the researcher establishes a set of categories and then counts the number of instances that fall into each category. The crucial requirement is that the categories are sufficiently precise to enable different coders to arrive at the same results when the same body of material (e.g. newspaper headlines) is examined’ `Qualitative Researchers analyse small numbers of texts and documents for a very different purpose. The aim is to understand the participants’ categories and see how these are used in concrete activities like telling stories, assembling files or describing “family life”. The constructionist orientation of many qualitative researchers thus means that they are more concerned with the process through which texts depict “reality” than with whether such texts contain true of false statements’ D. Silverman (2005), 2nd ed., Doing Qualitative Research; A Practical Handbook, London: Sage, p 160

4. Analysing secondary sources

5. Secondary Sources: An Alternative to Observation and Questioning • Secondary sources are those which already exist, as contrasted with primary sources which you collect/ generate/ construct • Typically, secondary sources include media reports, company histories and records, minutes of meetings, diaries, letters, photographs, etc [Why exclude books, articles etc from this list?] • Secondary sources are examined in order to gain knowledge of, and insight into, phenomenon under study. Style as well as content may be analysed • Secondary sources are particularly valuable when situations or events cannot be investigated by direct observation or questioning, or where reliance upon the public record alone (rather than, say, interviewee accounts) is preferred

6. Types of Documents • Personal (e.g. diaries) v. Official (e.g. press releases) • Restricted access (e.g. minutes) v. open access (e.g. company reports) Organizational documents may include company reports, memoranda, manuals, policy proposals, website information, accounting records, strategy documents, sales brochures, , etc Government documents may include departmental reports, parliamentary reports, etc. Much of this is becoming more readily accessible through websites and freedom of information act.

7. Source: J.Mason (2002), 2nd ed. Qualitative Researching, London: Sage, Ch 6 Characteristics of Documents • They are `constructed in particular contexts, by particular people, with particular purposes, and with consequences – intended and unintended’, p. 110 • The researcher may wish to • investigate why the documents were prepared, who prepared them, under what conditions and according to what conventions • know how the documents have been received, and what they have been used for

8. See Alan Thomas, Research Skills for Management Studies, London: Sage, 2004, pp 197 et seq Criteria for Evaluating Documentary Sources (1) • Is it authentic? The contents of the statements of corporate leaders (e.g. CEOs, Vice-Chancellors) that appear in annual reports might be analysed as part of a study of their communication styles. But can it be assumed that the leader, rather than a member of the PR office, has drafted the statement? Does it matter? • Is it credible? Is the author of the document an impartial and/or expert witness? Consider the annual report example given above. When the statement is optimistic, is the organization doing well, or is the statement intended to reassure nervous investors or current and potential stakeholders with an attempt to instil confidence? See next slide

9. An Example – Performance Records • …accusations of “massaging” of performance records have been levelled at the police service, the health service and elsewhere. Falsification of corporate accounts has given rise to some well-publicised scandals….Business leaders can rarely be considered as disinterested reporters of their careers and their organizational experiences. The analyst must assess the likelihood of their having overdramatized their role in the events they recount and must be sensitive to the social and political perspective from which they view the world’ • (Alan Thomas, Research Skills for Management Studies, London: Sage, 2004, p 200, emphasis added) • On what basis is the assessment of the analyst to be made? • How does the analyst develop `sensitivity to the social and political perspective…’? • What about the `social and political perspective’ of the researcher?

10. See Alan Thomas, Research Skills for Management Studies, London: Sage, 2004, pp 197 et seq Criteria for Evaluating Documentary Sources (2) • Is it representative? Has there been selectivity in what is recorded (e.g. successful ventures) in the documents, and what is preserved (e.g. minutes of meetings)? How much confidence can be placed upon the integrity of the data set? What about corroboration? • What does it mean? What does the document’s content mean to its author and/or its intended reader? To what extent, for example, is it an articulation of current conventions? Are these questions any less relevant for other documents, such as journal articles and thesis chapter drafts? See next slide

11. What Meaning is to be Attributed to Documents? …it is important to realize that documentary reality does not consist of descriptions of the social world that can be used directly as evidence about it. One certainly cannot assume that documentary accounts are “accurate” portrayals in that sense. Rather, they construct their own kinds of reality. It is, therefore, important to approach them as texts. Texts are constructed according to conventions that are themselves part of a documentary reality. Hence, rather than ask whether an account is true, or whether it can be uses as “valid” evidence about a setting, it is more fruitful to ask ourselves questions about the form and function of texts themselves’ (P. Atkinson and A. Coffey, `Analysing Documentary Realities’ in D. Silverman (2004), 2nd ed., Qualitative Research: Theory, Method and Practice’, p. 73 See also extract from Atkinson and Coffey’s article. Click here or see next slide Exercise: Can you summarise Atkinson and Coffrey’s stance in your own words? Can you provide a brief illustration, using a document that is to hand? Does the same argument apply to their text?

14. Analysing literature

15. See Alan Thomas, Research Skills for Management Studies, London: Sage, 2004, pp 72 et seq Reviewing Literature: What is the Point? Three Views `The purpose of a literature review is to establish the current state of knowledge in the field. It will therefore be a significant contribution to the dissertation or thesis and will usually be included in it as a prelude to the report of the empirical work’ Thomas, 2004, p. 73, emphasis added [The purpose of the literature review is] to demonstrate skills in literature searching; to show command of the subject area and understanding of the problem; to justify the research topic, design and methodology’ C. Hart (1998), Doing a Literature Review: Releasing the Social Science Imagination, London: Sage, p13, emphasis added `I object to the practice of simply backing up with a truckload of stuff and dumping it on unsuspecting readers, which seems to me what most traditional reviews accomplish. That is more likely to create an obstacle that gets in the way, rather than paves the way, to reporting what your have to contribute…By all means, flag important citations to the work of others. But do so sparingly and only as the references are critical to helping you to analyse and to situate your problem and your research within some broader context’ H. Wolcott (2001), 2nd ed., Writing Up Qualitative Research, London: Sage, p 74 and 75, emphasis added

16. Reviewing Literature – Some Key Elements • Your research may comprise a number of areas. A review must pay adequate attention to each of these • Your research methodology forms part of a literature that will require some critical reviewing • You should seek to achieve a tight coupling between your empirical work and your literature review. • The areas of literature that you review must be shown to be directly relevant to the topic of, and approach to, your empirical study • Avoid mere descriptive summarising; be focused and critical

17. Reviewing the Literature: Two Types of Review • Exploratory Review • examines the extensiveness (size) of the literature • examines how field has been addressed (scope) – angles, disciplines, themes, theories, key contributions? • not an exhaustive treatment of sources • Synoptic Review • concise and thorough review of all material that is assessed to be relevant to your study • Topic content and methodological approaches • critical evaluation of established literature in terms of how it could be extended both in content and method • identification and exploration of `gaps’

18. Reviewing Literature: Searching and Handling • Sources most relevant for preparation of reviews • Review papers or books that survey a topic (may indicate a dominant approach rather than what is relevant for you) • Star papers that are repeatedly cited (may indicate a dominant approach rather than what is relevant for you) • Model papers (may also be `star’ papers that are deemed exemplary within their field and may provide a benchmark for your own study) • Recommended literature from `experts’ in the field (likely to be highly selective, so be cautious) • Electronic data bases (watch out, as they can be overwhelming and incomplete)

19. Approaching Texts: Kinds of Reading • Rapid scanning – form an impression of the potential relevance of the paper – high, low, doubtful or potential • Selective reading – paying attention to only those sections, chapters or pages that contain relevant material • Top and tailing – reading of abstract, introduction and conclusion only, perhaps as a basis for deciding whether to undertake rapid scanning or close reading of the entire paper • Notes/ Bibliography reading – reading the references or the footnotes to gain a quick sense of where the author is `coming from’. Sometimes the most revealing and insightful points can be tucked away in the references • Close reading – attentive and repeated reading Tip. It may be beneficial initially to undertake rapid scanning of all sources, even when your intention is to undertake a close reading

20. Analysing Your Text

21. Some Pertinent Questions and Approaches to Writing • How do I make my text accessible and convincing? • What kinds of arguments can I build from my data? • Where does theory come into my argument? • What is the purpose of the argument? Arguing Evidentially – supported by evidence; how do you justify the relevance and credibility of the evidence presented?Arguing Interpretively – shown to be meaningful and reasonable; what standards of meaningfulness or reasonableness are you invoking?Arguing Illustratively – demonstrate the case through an appealing or persuasive exampleArguing Reflexively – acknowledging the problems of argumentation but commending a particular interpretation on the basis of the value of its particular contribution Adapted from J. Mason (2002), 2nd ed., Qualitative Researching, London: Sage, Ch 9

22. Producing Text: Starting and Editing • To give yourself the best chance of producing something worthwhile, create the conditions that are most conducive to writing. Exclude or minimise possible sources of distraction • To get you started, a good, focussing sentence to open up with is `The purpose of this chapter/paper/thesis…But, if this beginning is too demanding, just start your draft with whatever is possible, even if its relevance is not immediately apparent. Often introductions are written after the substantive sections of a text. • To avoid excessive redundancy and meandering, it may be useful to identify a set of key points and arguments that you want to make; or even create a table of contents that sets out on one page the intended structure of the text. It is likely that this will be elaborated and revised in the process of writing – perhaps as it becomes clear that the material requires two chapters, not one. • Review your document regularly to assess whether you are still `on track’, or perhaps to gain a clearer sense of the (new) track. • Be sure to give emphasis and space to key points where you make a distinctive contribution

23. Source: H. Wolcott (2001), 2nd ed., Writing Up Research, London: Sage Writing as Analysing Text (1) • Think of writing as an integral part of the research process, not as something that comes with `writing up’ `Hear this. You cannot begin writing early enough. And yes I really mean it. Would that mean that someone might write a first draft before venturing into the field to begin observations or interviews? Absolutely.’ H. Wolcott (2001), p. 21 `Writers who indulge themselves by waiting until their thoughts are clear runt the risk of never beginning at all…Writing is not only a great way to discover what we are thinking, it is also a way to uncover lacunae in our thinking’ (ibid: 22) • When you are writing your drafts of chapters (or conference papers) you will be working on a document.In effect, you will be analysing its content and style in relation to the (shifting) objectives you have ascribed to it • Writing can be especially anxiety-provoking if you are hoping/expecting to get it right first time. Just get it down. Worry about getting it right later. • Many people work through numerous drafts. That means undertaking recurrent analyses and critiques of the text-in-progress

24. Writing as Analysing Text (2): Tightening Up • Reviewing Content and Style • content is paramount but is rendered accessible by style; be mindful of your target audience • invite others to review your work; present it at seminars,etc; subject your work to peer review • Revising and Editing • revision addresses content – focus, qualification, major cuts to remove diversions, expansions to incorporate complexity etc • Logical sequence; systematic development • editing addresses style – economy of expression; avoiding repetitions and discrepancies; paraphrasing of quotations; restructuring sentences; use of punctuation, etc • Unnecessary words, passive voice, overused phrases, overused punctuation or other devices

25. Revisiting the Evaluation of Documents (see earlier slide) • Is it authentic? • to what extent do you claim ownership of the document (e.g. thesis)? • Is it credible? • on what basis do you assert and defend the credibility of the document? • Is it representative? • in what sense is it representative of a passable thesis? • What does it mean? • What is the significance of the document, and for whom?

26. (For Week 8)`Institutions and Technology’ : Using and Producing Documents for Analysis Questions for discussion: • How does the analysis of documents figure in this paper? • What kind of use is made of documents? • What other kind of textual analysis might have been conducted? • How do you evaluate the authenticity and credibility of the documentary sources and the article itself as a document? • In what respects can the article be viewed as un/representative of a contribution to an academic journal? • What is the significance or meaning of the article as a document, and for whom? T.Bridgman and H.Willmott (2006), `Institutions and Technology; Frameworks for Understanding Organizational Change – The Case of a Major ICT Outsourcing Contract’, The Journal of Applied Behavioural Science, 42, 1 : 110-126 Click here for access to the article.

27. Final Thought : In a Fog? `It is not unusual to begin in a fog. A certain amount of wandering around is inevitable before it is possible to find one’s bearings and gain a sense of direction. It is necessary to tolerate uncertainty at the beginning of a project, starting out with a broad view, scanning for a range of possibilities and then narrowing down to a specific focus’ (Thomas, 2004, p. 70)

28. Additional Recommended Reading • J. Scott (1990) A Matter of Record: Documentary Sources in Social Research, Cambridge: Polity Press • L. Pryor (2003), Using Documents in Social Research, London: Sage • A. Fink (1998), Conducting Research Literature Reviews : From Paper to the Internet, London: Sage • C.Hart (1998), Doing a Literature Review: Releasing the Social Science Imagination, London: Sage • C. Hart (2001), Doing a Literature Search: A Comprehensive Guide to the Social Sciences, London: Sage • L.Richardson and E.A. St Pierre (2005), `Writing: A Method of Inquiry’ in N.K. Denzin and Y.S. Lincoln, eds., Handbook of Qualitative Research, London: Sage