Download
1 / 67
670 likes | 783 Views
Margins of Error. John Pullinger , President of the Royal Statistical Society. Public trust and understanding. Bobby Duffy Director, Ipsos MORI Social Research Institute, Visiting Senior Fellow, King’s College London. Focus on understanding and value – but firstly on trust….
E N D
Margins of Error John Pullinger, President of the Royal Statistical Society
Public trust and understanding Bobby Duffy Director, Ipsos MORI Social Research Institute, Visiting Senior Fellow, King’s College London
Scientists and academics win... How much trust do you have in information provided by the following types of people? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 Scientists Academics Accountants Statisticians Economists Actuaries Pollsters Politicians
Trust in scientists vs trust in clergy – a new age of reason? ...would you generally trust them to tell the truth, or not? Base: c.1,000-2,000 Source: Ipsos MORI most years face-to-face in-hom, *2012 ICM telephone ** 2013 IM telephone
Trust in civil servants vs politicians – views have diverged... ...would you generally trust them to tell the truth, or not? Base: c.1,000-2,000 Source: Ipsos MORI most years face-to-face in-hom, *2012 ICM telephone ** 2013 IM telephone % Yes
But government less trusted with our data than online retailers? Base: c. 500 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 Companies such as supermarkets and online retailers collect a lot of data on their customers (for example through loyalty cards). To what extent, if at all, do you trust companies to use the data they collect about you appropriately The government collects a lot of data on citizens (for example through tax returns). To what extent, if at all do you trust the government to use the data they collect about you appropriately?
...not least, debt vs deficit... As you may know there is currently a lot of discussion about our national “debt” and “deficit”. Can you tell me what these words mean when talking about government finances? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 The difference between what government spends and the income it receives each year The total amount of money that the government owes Both mean the same Don’t know
...but public also not so clear when “use it in a sentence”... And can you tell me whether the following statement is true or false? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 “The national debt will always go down if the deficit is decreasing” Those who got definitions right no more likely to get this right Public think 40% of planned cuts already been made...
Basic understanding of numbers is key to statistical literacy – and it is mixed…
Most get very simple questions correct... What is 50 expressed as a percentage of 200? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 2010: 89% correct
...and slightly trickier... What is the average of the following three numbers – 5, 10 and 15? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 2010: 71% correct
...but real difficulties with probabilities... If you spin a coin twice what is the probability of getting two heads? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 2010: 30% correct Strong relationship with education (A-level+), but also big differences by age, younger groups more likely to get right...
A personal optimism bias... What do you think the chance or probability is of the following being injured or killed in a road accident this year (whether as a road user or a pedestrian)? Base: c. 500 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 Mean probability: Someone = 4.1% You = 1.6% Actual probability = c1.2%?
...but focus on negative information Imagine you have a life-threatening illness and your doctor has told you that you need an operation to treat it. How likely, if at all, are you to have this operation if your doctor tells you that... Base: c. 500 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 Avoid targets on “negatives”, even if hit them? Waiting times, immigration...
But does it matter? Do people consider evidence – or think their leaders do?
Principle-based policy-making... Politicians will take decisions partly based on what they think is right, and partly on evidence of what works. Do you think they base their decisions more on what they think is right than on evidence, more on evidence than on what they think is right, or do you think they consider them both in equal measure? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013
...but mirrors people’s own use of evidence People have different attitudes towards statistics. Which of the following do you agree with most? Base: 1,034 British adults aged 16-75 Source: RSS/Ipsos MORI 2013 My own experiences or those of my family and friends are more important than statistics in helping me keep track of how the government is doing Statistics are more important than my own experiences or those of my family and friends in helping me keep track of how the government is doing Both equally Neither/Don’t know
We’re not embarrassed about lack of understanding of numbers... Which of the following things would you feel most embarrassed about admitting to friends and family? Base: 516 British adults aged 16-75, interviews conducted online 9th-15th April 2013 Source: RSS/Ipsos MORI 2013
...and there’s little pride in doing it well Thinking about your child/if you had a child, which of the following would make you most proud? Base: 516 British adults aged 16-75, interviews conducted online 9th-15th April 2013 Source: RSS/Ipsos MORI 2013
We’ve got a long way to go... I keep saying that the sexy job in the next 10 years will be statisticians. And I’m not kidding. Hal Varian, chief economist at Google Statistical thinking will one day be as necessary for efficient citizenship as the ability to read or write HG Wells Value of statistics Number of people reached Quantity of statistical info Media affect Relevance Trust Numeracy VAS = N * [(QSA * MF) * RS * TS * NL] EnricoGiovannini, Former Chief Statistician, OECD
Thank you bobby.duffy@ipsos.com @BobbyIpsosMORI
Understanding and Trust in Statistics Andrew Dilnot CBE, Chair of the UK Statistics Authority
GDP Revisions Available data(output measure) 25 days approx 55 days approx 3 months
Former minister slams 'national catastrophe' of teenage mothers addicted to benefits No set of values for gym-slip mums UK has highest teen pregnancy rate in Europe Teenage pregnancy soars
Norovirus lab reports Source: Health Protection Agency
Norovirus confidence intervals • 1:1500 (1 lab case = 1500 in community). • 2000 lab cases = 3million in community • But maybe 1:140 • (=280,000 cases) • Or maybe 1:17,000 • (=34 million cases) • Community study lab cases… • =1
Lies, damn lies and crime statistics Crime statistics were distorted by politics Police fail to record crime properly
The Guardian, 19 October 2013 Crime falls to new low despite recession and unemployment ...The 6% fall in crime reported in the latest quarterly figures by both the Crime Survey for England and Wales and the separate police recorded crime figures means that crime has now dropped by more than 50 % since it peaked in the mid-1990s...
Public Understanding of statistics in an era of big data Denise Lievesley, Head of School of Social Science and Public Policy, King’s College London
Challenges facing statisticians Humility vs Confidence Relevance vs Autonomy Trust vs Scepticism Measurement Quality vs Pragmatism Purism vs
Humility vs. Confidence Being a statistician means never having to say you’re certain
Humility – being aware of our limitations “Good science should not turn a blind eye to known imperfections – nor should these be concealed from users” Sir Roger Jowell 2007 “The absence of excellent evidence does not make evidence-based decision making impossible: what is required is the best evidence available not the best evidence possible” Sir Muir Gray 1997
ISI declaration on professional ethics 1985 One of the most important but difficult responsibilities of the statistician is that of alerting potential users of their data to the limits of their reliability and applicability. The twin dangers of either overstating or understating the validity or generalisability of data are nearly always present. Confidence in statistical findings depends critically on their faithful representation. Attempts by statisticians to cover up errors, or to invite over- interpretation, may not only rebound on the statisticians concerned but also on the reputation of statistics in general.
Confidence – using the data to make a difference • We need to provide information of high quality, integrity and robustness which can be relied on. • We should be confident about our findings and prepared to account for them.
We need to improve our communication skills and think about impact.We should learn how to tell a story with dataand remember that communication is not what is delivered but what is received. Communication e.g. • Bill Gates has a personal fortune greater than the combined wealth of the 106 million poorest Americans. • The cost of putting all children into school is less than is spent on icecream in Europe each year
Sir Gus O’Donnell(former UK Cabinet Secretary) “I want [the ONS] to be boring, to put out the plain facts, and nothing but the facts, and on clear, predictable deadlines,” he said. It would then be for politicians and government press officers to interpret the figures, he added.
Response of the Royal Statistical Society • it is clearly the task of statisticians to interpret the figures in a statistical context, to facilitate understanding and avoid misunderstanding. • The Code of Practice of the UK Statistics Authority explicitly states that Official statistics, accompanied by full and frank commentary, should be readily accessible to all users and that all UK bodies that are responsible for official statistics should prepare and disseminate commentary and analysis that aid interpretation, and provide factual information about the policy or operational context of official statistics.
