Research in the real world: the users dilemma

1. Research in the real world: the users dilemma Dr Gill Green

2. Overview of the Lecture • Context for the examination of research approaches • Examine aspects of the Qualitative Research: Defining, Attributes, Features &Types • Examine aspects of the Quantitative Research: Defining, Attributes, Features &Types • Reflect and summarise on each approach

3. The users dilemma • How do users get what they want? • Traditional view of software development • Analysis – specification-development-implement-signoff • Happy users • So why do researchers report that 70% of system implementations fail • How do users know what they want? • The Marco Polo effect • How do you describe something you have never seen before • The gambler effect • How do you speculate how you may like to do things in the future • The tigger effect • How easy is it to ask for the wrong thing

4. Some hard words • Ontology – What is • epistemology – what it means to know • Why is this important? • You need to know for yourself how you interpret your world?

5. Theoretical perspectives and what they teach us • Positivist • Reality consists of what is available to the senses • Inquiry should be based upon scientific observation and empirical action • Principles are shared between Natural and human sciences and deal with facts not values • Interpretivist • Reality is a shifting state culturally derived and historically situated • Inquiry deals with the actions of individuals in social settings • Principles suggest the emergence of unique individual qualitative aspects

6. What is Qualitative Research “Qualitative research is a process of enquiry that draws data from the context in which events occur, in an attempt to describe these occurrences, as a means of determining the process in which events are embedded and the perspectives of those participating in the events, using induction to derive possible explanations based on observed phenomena.” (Gorman & Clayton, 1997)

7. What happens in Qualitative Research? • Data taken from context in which events occur • Data collection first hand • Attempt to describe occurrences • Focus on process not snapshot • Immersion leading to insight • Induction

8. Qualitative Research: Induction • Use of “bottom-up” approach to analyse and interpret data • Research based on observed data • “Grounded” theory • that is based on established theories

9. Qualitative Research: Attributes 1 • Assumptions • social construction of reality • primacy of subject matter • complexity of variables • difficulty in measuring variables • Purpose • contextualisation • interpretation • understanding participant perspectives

10. Qualitative Research: Attributes 2 • Approach • Theory generalising • Emergence and portrayal • Researcher as instrument • Naturalistic • Inductive • Pattern Seeking • Looking for pluralism and complexity • Descriptive • Researcher Role • personal involvement and partiality • empathetic understanding

11. Key features of Qualitative Research (Hittleman & Simon) • Data is collected within its natural setting. Main data collection instruments are the researchers themselves • Data are not numerical • Focus on the process of an activity, not just its outcomes • Data analysed in non-numerical manner. Outcomes generate debate rather than verifying a predicted outcome

12. Qualitative Research: Why is it important in IT • Many of techniques and methods can be applied to the requirements engineering process • Helps to place user at centre of design process • Enables triangulation with quantitative methods

13. Doing Qualitative Research • Many ways of collecting and analysing data • Historical • Correlational • Developmental • Descriptive • ...

14. Qualitative Research: Overview of Techniques • Observation • Interviewing • Questionnaires • Group Discussion • Historical Study • Content Analysis • Ethnographical Research

15. Qualitative Research Summary • Increased knowledge of qualitative research • Awareness of qualitative approaches relevance to computing

16. Quantitative Research: What is it? • The aim of quantitative research is not simply to state that something has a relationship with something else, but to state causality

17. Quantitative Research • Focuses on numerical and statistical data • Positivist approach • Recognising only positive/measurable facts and observable phenomena • Empirical “scientific” approach • Relying on experimentation and not untested theory • Searches for causality and effect

18. Quantitative Research :Deduction • Top-down approach • The inferring of particular instances from a general law • Working something out from something else - Sherlock Holmes style

19. Attributes of Quantitative Research 1 • Assumptions • objective reality of social facts • primacy of method • possible to identify variables • possible to measure variables • Purpose • generalisation • prediction • causal explanation

20. Attributes of Quantitative Enquiry 2 • Approach • Hypothesis based • Manipulation and Control • Uses formal instruments • Experimentation • Deductive • Component analysis • Seeking norms and consensus • Reducing data to numerical indices • Researcher Role • detachment and impartiality • objective portrayal

21. Features of Quantitative Research 1 • Tests for cause and effect • X causes Z to happen • Y does not cause Z to happen • Not simply that something has a relationship with something else • Involves empirical studies • Uses numerical and statistical techniques

22. Features of Quantitative Research 2 • Assume primacy • Researcher defines the research activity • Relationships are measured • Causal explanations are made

23. Quantitative Research : Descriptive Statistics • Allows summaries of large quantities of information • Should be easily comprehensible for reader • Presentation is vital • long strings of numbers… • tables, charts, graphs • numerical techniques • concise, appropriate text

24. Quantitative Research : Inferential Statistics • Procedures for making generalisations about characteristics of a population based on information taken from that population • Powerful • estimation • hypothesis testing • Methods and rules for organising and interpreting data

25. Quantitative Research: Why is it important in IT • Establishes metrics • Report on process and system efficiency concerns • Predict outcomes from developments • Improve development and operational processes • Basis for managing risk • Analysis of incidents • Identify causal relationships • Plan

26. Quantitative Research Summary • Quantitative research is based on scientific inquiry • Offers numerous techniques for data analysis • Searching for causality and prediction

27. Some questions to answer for next week • Can you identify your epistemological stance? • Have you identified a theoretical perspective • Is your approach deductive or inductive • Have you considered research methodology