Dr Eric Jensen e.jensen@warwick.ac.uk

1. Researching Science, Media and Public Policy Evaluating science engagement: First thoughts Dr Eric Jensen e.jensen@warwick.ac.uk

2. Overview of Claims of Long-term Impact – Ecsite report ‘Evidence for long-term learning’ ‘There is significant evidence to suggest that Science & Discovery Centres provide lasting benefits.’

3. Overview of Claims of Long-term Impact – Ecsite report • Falk & Dierking (1997) interviewed adults and children aged 9-10 and 13-14 years old about past trips to Science & Discovery Centres. • They found that even after a gap of several years both adults and children could recall many aspects content or subject-related information: • (77% of memories); details of the physical setting (56% of memories); emotional responses to the experience (55%) and details of the social aspects of the visit (47%).

4. Overview of Claims of Long-term Impact – Ecsite report • Of the adults and children interviewed, 80% claimed to have thought about the Science & Discovery Centre experience afterwards. • Similarly Anderson et al (2002) assessed pupils’ memories four to six years after visits to various types of museums and science centres.

5. Overview of Claims of Long-term Impact – Ecsite report • Stevenson (1991) looked at the impact of a major interactive science exhibition immediately after the visit, a few weeks later and then after six months. • It was found that even after six months visitors were able to spontaneously recall details of their experience. • Around 26% of memories – most of them fairly detailed – were spontaneous and most people could spontaneously recall five different exhibits.

6. Overview of Claims of Long-term Impact – Ecsite report Stevenson (1991): • Around half of the memories elicited either spontaneously or with prompts were detailed and clear. • 60% of the exhibit memories were descriptions of what the visitor did at the exhibit and 14% refer to their feelings about the experience. • 26% of the memories reflected visitors’ subsequent thinking about the exhibit’s content suggesting that there was at least some cognitive processing of the experience rather than just the recalling of isolated episodic memories.

7. Overview of Claims of Long-term Impact – Ecsite report CRITICAL RESPONSE: ‘Remembered visiting’ is placed as evidence of ‘long-term impact’. These are not the same thing however!

10. NOW TO LOOK AT A COUOPLE PEOPLE WHO REMEMBERED GOING TO THE MUSEUM!

11. Results: Prior Museum Experience • For those raised locally, some had vague memories of childhood visits to the Fitzwilliam:

12. Results: Prior Museum Experience • One respondent did have distinct memory of her childhood visit to the Fitzwilliam Museum: • This negative memory can also be seen in Tina’s personal meaning map (next).

13. Pre-visit personal meaning map (Tina)

14. Remembering the museum does not equal ‘long-term learning impacts’ So do they reveal long-term positive learning impacts???

15. ANOTHER EXAMPLE

16. Previous Research: What is known? • Study of museum visit impacts commissioned by UK government (DCMS / DCSF). • This 2007 study included one element on outreach programmes, with data collected from ‘community participants’ themselves (first study to do so).

17. Previous Research: What known? • Survey of young people and adults engaged through museums’ “expansion of community provision” asked to respond to a number of statements with three response options: ‘yes’, ‘no’ and ‘don’t know’. • Responses to this survey were very positive (Greenhill et al., 2007, p. 40)

18. Previous Research: What is known? • Large majorities said ‘yes’ to statements: • “Working with the museum has been very inspiring for me” (82%) • “I discovered some interesting things” (93%) • “I feel I have a better understanding of the subject” (84%) • “I could make sense of most of the things we saw and did” (85%) • “Using the museum was a good chance to learn in new ways I had not considered before” (80%)

19. Previous Research: Limitations • Statements used did not arise from respondents themselves, but rather were imposed within a closed-response (yes or no) framework. • As such, the validity of these results could be questioned. • In addition, the research was cross-sectional with no follow-up dimension on the day or later.

20. Zoo Visitor Impact Research Review

21. Previous Evidence on Zoo Impacts: Quality Problems • There is a lack of quality research currently published on the impacts of zoos. • Most of the zoo visitor studies are highly flawed methodologically. • Not falsifiable • Small sample size • Flawed assumptions • Discredited sampling approach • Poor practice in survey design • Validity problems in use of psychometric scales