Data Generation, Complexity and Synthesis

Data Generation, Complexity and Synthesis Prof. Rob Kitchin, NIRSA, NUI Maynooth

Introduction • Generating data is about creating empirical evidence • It is not a neutral, passive exercise • It is generation, not collection; creation not harvesting • It is theoretically informed • It involves assumptions about the world (ontology) • It involves assumptions about how the world can be validly measured (epistemology) • It involves assumptions about what academic research is for (ideology) • It necessitates a series of informed choices about mechanisms to elicit useful data (methodology) • It is a complicated process • It involves synthesis; the drawing together of lots of data to create an overview; a theoretically informed, empirically grounded story • It involves carefully thinking and planning, and sound execution; it should not be an ad hoc process

Starting with questions • Research is about answering questions • There are lots of different types of questions • What is A like? What is interacting in the urban environment like for disabled people? • What does A mean? What do we mean by `access for all' and `barrier free environments'? • Is A like B? Are the planning needs of disabled people the same as the able-bodied community? • Is A different from B? Do planners and disabled people agree on how the urban environment should be designed? • Is A better than B?Is the urban planning for disabled people better in Labour council areas than Conservative run areas? • Are A and B related? Is there a relationship between the size of a town and the quality of urban planning for disabled people?

Starting with questions • Does A affect B? Does poor design and low accessibility of an environment decrease usage by disabled people? • Does A cause B? Do disabled people use this area more as a direct result of its re-development to make it more accessible? • Is A located where B is min/max? Are special day care centres located in the areas of primary need? • How are A and B minimised simultaneously? Can we maximise the accessibility of the environment whilst minimising the expense? • Why does A support B?Is a disabled access project receiving funding from the state for political reasons rather than genuine commitment to the access needs of disabled people?

Framing questions • How we set about answering our questions is not as simple as it at first might seem • Answering questions is embedded in theoretical assumptions and choices • Philosophical theory – ideology, ontology, epistemology • The specifics and mechanics of knowledge production • Positivism, Phenomenology, Marxism, etc. • Conceptual theory – substantive focus (e.g., governance - coalitions, regimes, regulation, actor-networks, etc.) • Philosophical and conceptual theory are highly inter-related • The rejection of theory – e.g., collect data and let it speak for itself - is in itself a theoretical position; empiricism

Framing questions Empiricism Empiricism refers to the school of thought where facts are believed to speak for themselves and require little theoretical explanation. Empiricists hold that science should only be concerned with objects in the world and seek factual content about them. Normative questions concerning the values and intentions of people are excluded from study as it is claimed we cannot scientifically measure them. A source of primary data is closed-question questionnaires Positivism Positivists argue that by carefully and objectively collecting data regarding social phenomena, we can determine laws to predict and explain human behaviour in terms of cause and effect. Positivists reject normative and metaphysical questions that cannot be measured scientifically. ·Positivism differs from empiricism because it requires propositions to be verified (logical positivism) or hypotheses falsified (critical rationalism) rather than just simply presenting findings. ·Sources of primary data are closed-question questionnaires and surveys

Framing questions Marxism Marxists suggest that society is structured so as to perpetuate the production of capital. Marxists are concerned with investigation of the political and economic structures that underlie and reproduce capitalist modes of production and consumption. ·To do this Marxists suggest that we need to consider how conditions might be under different social conditions to highlight how society operates. ·A source of primary data is observation but also re-examines secondary data sources with analysis consisting of determining the dialectical (how one affects the other) relationship between societal structures and individuals Phenomenology Phenomenology rejects the scientific, quantitative approach of positivism. Instead phenomenology suggests that we concentrate upon understanding rather than explaining the world. The goal of phenomenology is to reconstruct the worlds of individuals, their actions, and the meaning of the phenomena in those worlds to understand individual behaviour, without drawing upon supposed theories. A source of primary data are in-depth interviews with people who have experienced the phenomena in question

Framing questions • Thinking through philosophy … • Naturalist/Anti-naturalist? • Value-free/Action-orientated? • Objective/Situated? • Truth or truths? • Realist/Anti-realist? • Structure/agency? • …

Answering questions • Questions are answered through a four-stage process • data generation (e.g. interviews, questionnaires, surveys) • data analysis (e.g., statistical test, discourse analysis, etc) • data synthesis (pulling and weaving together different analyses) • data interpretation (what does the analysis tell you) • These constitute methodology – a set of ordered techniques • Methodology must match ontological and epistemological assumptions • It might be necessary to employ more than one methodology to provide a sufficient answer to a question

Example study of poverty Empiricism: Facts about poverty would be collected and presented for interpretation by the reader. (e.g. Indices of poverty - social welfare recipient, housing tenure, etc.). Positivism: Poverty is explained through testing a hypothesis by collecting and scientifically testing data related to poverty (e.g. statistically testing whether poverty is a function of educational attainment). Phenomenology: To understand poverty it is suggested that we need to reconstruct the world of people who are poor (e.g. we need to try and see the world through the eyes of a poor people). This might be attempted by talking to them about their life experiences. Marxism: Poverty is explained through the examination of how poor people are exploited for capital gain (e.g. Examining whether poor people are poor because it is in the interests of capital to retain unskilled, low wage jobs rather than distribute fully corporate profit).

Answering questions validly • All good studies aim to be valid and reliable • Validity concerns the soundness, legitimacy and relevance of a research theory and its investigation • Validity relating to theory concerns the integrity of the theoretical constructs and ideas that support and provide the foundations for empirical research • Validity relating to practice concern the soundness of the research strategies used and the integrity of the conclusions that can be drawn from a study

Practical validity • Construct and analytical validity both relate to the methodological integrity of a study • Construct validity concerns whether data generation techniques are sound, measuring the phenomenon they are supposed to without introducing error or bias; they are telling you about what you want to know • Analytical validity concerns whether the correct method of data analysis has been chosen, leading to results that represent the data truly. Clearly if you have chosen the wrong method of data analysis then you might end up drawing inappropriate conclusions • Ecological and internal validity both relate to the integrity of the conclusions drawn from a study • Ecological validity is concerned with the inferences that can be made from the results of a study • Internal validity concerns whether the results from a study can be interpreted in different ways; can different conclusions be drawn from the same results?

Answering questions reliably • Reliability refers to repeatability or consistency of a finding • Golledge and Stimson (1997) describe three kinds of reliability: • (1) quixotic reliability, where a single method of observation continually yields an unvarying measurement; • (2) diachronic reliability, which refers to the stability of an observation through time; and • (3) synchronic reliability, which refers to the similarity of observations within the same time period.

Answering questions ethically • Research ethics are concerned with the extent to which the researcher is ethically and morally responsible to her/his participants, the research sponsors, the general public, and her/his own beliefs • professional approach to research and focuses upon issues such as privacy, confidentiality, and anonymity • Researcher should weigh carefully the potential benefits of a project against the negative costs to individual participants

Data synthesis • Pulling and weaving together different analyses • Different methods open up different aspects of a phenomenon – methods are oligoptical in nature (partial views from selective locations) rather than panoptical (providing a full view from a gods-eye position) • Different methods can offer results that are often paradoxical, contradictory, puzzling • Is this because the phenomena are paradoxical, complex and messy or because of methodological issues? • In synthesising analyses we need to think carefully about how data were generated and the validity in processing in different ways • We need to think through any puzzling results and what paradoxes and contradictions might mean

Data interpretation • What does the synthesis tell you? • What insights have you gained? • What is the answer to your question? • relationships • themes • Effected by external and internal validity • What are the potential effects of your interpretation? • on theory • on practice and policy • What lessons can be learnt from your research praxis? • how might methodology be tweaked, etc?

Data Generation, Complexity and Synthesis