Graham Smith*, Peter John+, Patrick Sturgis* and Hisako Nomura+ * University of Southampton, UK

1. Deliberation and internet engagement: initial findings from a randomised controlled trial evaluating the impact of facilitated internet forums Graham Smith*, Peter John+, Patrick Sturgis* and Hisako Nomura+ * University of Southampton, UK + University of Manchester, UK

2. Background • Growing interest in democratic innovations – mechanisms that aim to increase and deepen citizen participation in the political decision-making process (Fung 2003; Smith 2009) • These provide opportunities for citizens to discuss matters of public concern • Perceived by policy-makers and reformers as a potential response to widespread disillusionment and disenchantment with the political process

3. The potential of information and communication technology (ICT) • ICTs can challenge traditional barriers to citizen participation associated with time and space • Citizens can engage at their own pace and participation can be ‘scaled-up’ • It is claimed the internet can promote honesty, directness in political exchanges and contact with people from different backgrounds, which is appropriate for an online discussion exercise (see Dahlberg 2001, Witschge 2004 for a discussion)

4. Some concerns about internet and online discussion • Will these new avenues for engagement be dominated by the already politically-interested and engaged, hence reinforcing existing differentials of political power and influence? • Sceptics (including many policy-makers) are concerned by the relatively small numbers of citizens involved and the extent to which they should have a privileged position in the decision-making process • Online discussion also runs the risk that the experience may be diluted. • May lose the intensity associated with non-verbal communication that conveys meaning and emotion

5. The project • One element of the Rediscovering the Civic programme of research • Funded by ESRC/CLG/NWIN • Joint project of University of Manchester (Institute for Political and Economic Governance) and University of Southampton (Centre for Citizenship and Democracy • Variety of experiments (RCTs, design experiments) and other work on aspects of civil engagement (voting, recycling, lobbying, pledging etc) http://www.civicbehaviour.org.uk/research/

6. Randomised Controlled Trials and deliberation • Social scientists have evaluated deliberative forums to see if participants change their views, knowledge and opinions (Drysek 2002, Fishkin 1997, Delli Carpini et al 2004, Price, Nir and Cappela 2006) • Danger of false inferences from study effects • Few randomised control trials testing for the effect of deliberation (exceptions are Farrar et al 2003, Iyengar 2005) • No experiments testing the deliberative potential of online discussion forums

7. Research Questions • To what extent does asynchronous online engagement amongst large numbers lead to shifts in policy preferences? • To what extent can asynchronous online engagement amongst large numbers be termed deliberative? • What is the value of citizens’ interaction compared to the provision of information alone?

8. The RCT design • 6,000 participants from Ipsos-MORI survey panel using random quota sampling. • Randomly allocated to 4 treatment and 2 control groups • 2 x deliberation groups • Hosted on specifically-designed phpBB 3.0.x boards • Access to discussion boards and background information • Goup 1: youth anti-social behaviour (ASB) followed by community cohesion (CC) • Group2: community cohesion (CC) followed by youth anti-social behaviour (ASB) • 2x information-only groups • 2 x control groups • All participants completed three surveys • T1 (before experiment), T2 (after first issue), T3 (after second issue)

9. Design of the experiment

10. Analysis issues • Initial findings here - shifts in opinion for ASB • Intention to Treat (ITT) does not give a full picture of changes • People select into viewing and posting: more intense reaction, but also where selection happens (also at start of the experiment and with online participation) • Propensity score matching (PSM) is not suitable for this experiment – N is too small

11. CACE models • Complier Average Causal Effect (CACE) model: a latent variable approach to the estimation of treatment effects in the presence of non-compliance (Jo and Muthen 2001). • A latent class (or finite mixture) framework is used to estimate the compliance status of those in both the treatment and the control conditions, allowing ‘fair’ comparisons to be made between compliers in the treatment condition and ‘potential compliers’ in the control group. • The problem is the control condition compliance status is unobserved so the model is unidentified. • CACE model achieves identification of the latent compliance classes through application of the ‘exclusion restriction’ • The exclusion restriction relates to the actual and potential behaviour of experimental subjects and how this is associated with the outcome.

12. Effect of online deliberation

13. Policy questions ranked according to effect size

14. Effect of information-only

15. Comparing the use of supporting materials for POST

16. So what predicts who logs in? • Not everyone participates • Next slide runs a regression model to look at bias

18. Conclusions • Online environment ‘works’: there was an exchange of views under the guidance of moderation, and with the aid of written and video materials that participants may access • The is an effect of debating online: people moved opinions endorsing community based explanations • Information not a powerful factor on its own – effect changes for deliberators only • Could be used as a policy tool? • But what about self-selection