Presentation Transcript

1. Tobacco Control: Lead GenerationQ1 May-June – Q3 Sept-Oct 2009F2F Evaluation – North West

   9th March 2010
2. Contents Introduction Part 1 – Methodology Part 2 – Results Overall Results North West Results Part 3 – Tests Collateral Roving Lung Age Part 4 – Adviser Attendance Part 5 – Lead Tracking Part 6 – Learnings and Recommendations Part 7 – Appendix Artemis Gender Evaluation – National Results Age Evaluation – National Results Acorn Profiling
3. Introduction This presentation is to review the results of F2F activity run across May-June and Sept-Oct phases 2009, both on a national level and in the North West region First time activity has been run in both time periods Running activity across all 9 SHA regions concurrently continued from Quitting Season Jan-March 2009

4. Part 1: Methodology

5. Methodology The below tables show new processes introduced in each phase All methodology implemented in May-June was carried over to Sept-Oct
6. Methodology

7. Part 2: May-June and Sept-Oct ResultsAll Regions

8. Results Overview Results from each phase have been evaluated based on the following metrics: Total response – all interactions at the F2F activity Valid response – the total number of leaflets distributed Active response – this refers to the number of data capture forms completed Cost Per Response (CPR) is calculated by: Adding all fixed costs* needed to rerun a campaign + variable costs / no. of leads * Fixed costs include training, uniform, collateral and 80% of agency fees
9. Overall results – Core Activity May-Oct 2009Primary KPI’s - Data Capture: Active Response ^ excludes investment costs Cost per day (operational costs only) reduced from £2,603 in May-June to £1,984 in Sept-Oct Overall CPR has been reduced by £13 ^ excludes investment costs. Operational costs include variable costs and fixed costs including 80% fees, training and collateral and uniform
10. Overall Results by Venue Type All venue types reduced CPR in Sept-Oct as a result of cost savings and improved performance from May-June Roving CPR in Sept-Oct was lower than other venue types which contributed to a lower overall CPR Shopping Centres have the highest CPR for the Sept-Oct phase but this is a decrease of £11 from May-June activity – lower site fees across phase were a key factor
11. Average Number of Completed Data Capture Forms by Activity Day Asda continues to lead as the best performing venue type across F2F in both May-June and Sept-Oct Co-op performed least well in both phases but a very low number of activity days, therefore, not affecting overall results Apart from Co-op and Tesco all venue types increased performance in Sept-Oct

12. May-June and Sept-Oct ResultsNorth West

13. Overall results – North West May-Oct 2009Primary KPI’s - Data Capture: Active Response ^ excludes investment costs. Operational costs include variable costs and fixed costs including 80% fees, training and collateral and uniform
14. Results Summary by Venue Type: North West – May-June Retail venues were the best performing venue type in May-June achieving £42 CPR against £52 CPR nationally – only three stores out of the 15 visited performed below 45 leads due to hardened smokers not wanting to or not ready to quit Shopping Centres demonstrated a much lower than average CPR (£46/£60 respectively) – all Shopping Centre locations exceeded targets set – the highest achieving 62 sign-ups
15. Results Summary by Venue Type: North West – Sept-Oct Retail venues had a lower CPR than in the May-June phase (£36 vs. £42) and lower than the national average (£44) in Sept-Oct – 15 of the 20 retail venue types exceeded target – the two highest performing venues achieving 79 leads each Shopping Centres performed well with a £3 lower CPR than national average – 112% of Shopping Centre data capture target was achieved – good footfall in high R&M penetration areas were factors
16. North West Performance by Activity Phase Performance in the North West region has steadily increased each phase – CPR was at it’s highest in Summer 08 as % of data capture target achieved was at 50% and since steadily risen whilst CPR decreased Following cost saving negotiations and target reviews in May-June, CPR in Sept-Oct fell to £38 and116% of target was achieved
17. Average Number of Data Capture Forms Completed per Venue Type Asda was the best performing venue type in May-June (57) and Sept-Oct (59) which reflects national trends Asda, Shopping Centres and Sainsbury’s all performed above the national average in both May-June and Sept-Oct as did Tesco in Sept-Oct Tesco was the only venue type in May-June to perform below the national average and was the lowest performing venue type across both phases – results are based on the same venue that was visited once in May-June and twice in Sept-Oct (Tesco Burnage) – only one of the three days exceeded target

18. Part 3: Tests

19. May-June Regional/National TestsNext Steps Leaflet

20. May-June Tests - ‘Next Steps’ Leaflet (Tested in NW/EoE) ‘Next Steps’ leaflet: Implemented to ensure ‘next steps’ were clearly understood by smokers Replaced Credit Card leaflet in half the PCTs across North West/East of England Control cells used Credit Card leaflet Objective to see if the Next Steps leaflets had higher conversion to appointment/quit
21. Test Results – ‘Next Steps’ Leaflet (NW/EoE) The results to this were not conclusive because the results are not statistically significant Low number of responses from Lead Tracking initiative No quit numbers supplied by North West region 11 events replied from East of England (detailed below)

22. Sept-Oct Regional/National TestsRoving

23. Sept-Oct Test: Roving After successful campaign in London Roving was tested as part of core campaign After liaison with EMO it was agreed to test roving in the East and West Midlands regions Heat maps were used to identify hot spots in and around town centre locations Research was undertaken and local places of interest were chosen based on footfall/dwell time where R&M smokers were likely to be encountered LSSS were contacted for feedback on local smoking hot spots and high R&M footfall areas Test achieved 100% of data capture target overall (101% West Midlands/99% East Midlands) Achieved a lower CPR (£35.68) than targeted for overall Sept-Oct (£43.81) phase due to low operational costs Following the success of the test Roving has now been factored into the venue mix for F2F going forward - 25% of activity in New Year phase is Roving
24. Overall Results – East & West Midlands Roving Activity * London upweight featured as a comparison but was a different brief at a different time of year in a different region and higher budget which enabled bigger team size and better scales of economies ^ excludes investment costs. Operational costs include variable costs and fixed costs including 80% fees, training, collateral and uniform
25. Average Number of Data Capture Forms Completed Per Roving Day Target 37 All days achieved in excess of 90% of target
26. Test Results – Roving West Midlands October 2009 Primary KPI - Data Capture: Active Response ^ excludes investment costs. Operational costs include variable costs and fixed costs including 80% fees, uniform training and collateral
27. Test Results – Roving East Midlands October 2009Primary KPI - Data Capture: Active Response ^ excludes investment costs. Operational costs include variable costs and fixed costs including 80% fees, training and collateral and uniform
28. West Midlands Static v Roving Comparison The number of data capture forms completed during the Roving activity is above 100% and is the second highest performance for this region Roving performed better than all previous phases in terms of CPR (£35) and demonstrated a lower CPR than the static approach during the October phase which was £41 Percentage of data capture achieved against target during October is slightly better than the stand based approach by 2% NB. Previous activity ran during different time of year so like for like comparison is difficult
29. East Midlands Static v Roving Comparison Performance in the number of data capture forms completed during the Roving activity is higher than all previous phases (99%) and slightly better than the static approach in Sept-Oct which was 98% CPR is lower for October’s Roving activity than all previous phases (£36) and is £11 lower than the stand based approach in October

30. Sept-Oct TestsLung Age Tests

31. Sept-Oct Test: Lung Age Lung Age Testing was introduced across all 9 regions hosting core F2F activity in Sept-Oct, in order to: See if undertaking the test acted as a motivator to signing up for LSSS support to quit smoking Build a bank of data for the COPD team at DH 30 of the most experienced F2F marketeers were selected and received training to become ‘level 2’ Community Advisers where they learnt to administer CO and Lung Age tests Upon recommendation, COPD-6 spirometers were purchased for the test Upon starting the test the spirometers began to give unexpected readings: Lung Age younger than the smokers real age – detrimental to F2F objectives as may appear to condone smoking Lung Age readings well in excess of 100 years Due to unexpected readings the test was put on hold except for the South West and North West regions
32. Sept-Oct Test: Lung Age Table below details the number of negative (lower Lung Age) readings and also readings where the Lung Age were over 30 years higher/100 years or over (high readings): General feedback from all teams was that it worked as a motivational tool when the readings were within expected range; it takes longer than a CO test but the result is more tangible The client was encouraged to visit their GP as well as sign up for support if their reading was 30 years higher than their actual age or over 100 years of age If any client was concerned about any of the readings, they were also encouraged to visit their GP
33. Sept-Oct Test: Lung Age Following the initial test using COPD-6 monitors a different Lung Age machine was sourced and tested at four activity days Lung Age machines were configured to prevent a smoker gaining a younger Lung Age than their real age Lung Age machines were easier to use and gave results which worked to support the objectives of the campaign
34. Lung Age Testing – Test Results To assess the impact of Lung Age testing results were compared from venues where Lung Age tests were undertaken with results from the same venues when no tests were performed Of the eight regions that had undertaken Lung Age tests six performed better when Lung Age tests were performed
35. Lung Age Testing – Test Results The data below shows that the % of target data capture achieved was 16% higher when Lung Age tests were performed The pattern is consistent across all the regions except the North East and East of England It is assumed that all clients who have undertaken a test sign up to support Lung Age testing was a contributing factor to the reduction in CPR over the phase

36. Part 4: Adviser Attendance May-June and Sept-Oct

37. Adviser Attendance at F2F Events Advisers are invited to each event as they add a lot of value to the activity through the local knowledge they bring and their experience in smoking cessation Communication has improved dramatically between Carlson and LSSS contacts following the introduction of event overviews and the increased use of the hotline and tobaccof2f@carlson-europe.com email May-June and Sept-Oct: the services were notified about the activity by: An event overview detailing the activity sent to the services two weeks prior to the activity (where possible) A phone call made to each a week before the activity and again the day before Email notification of the activity schedule was sent out prior to the activity on a weekly basis (regional schedules) To encourage attendance at F2F events a competition was run for every adviser that attended an activity day in Sept-Oct The prize was a trip to the UKNSCC in Glasgow 2010
38. Adviser Attendance May-June and Sept-Oct Against National Trend Adviser Attendance rose in the North West from 55% to 67% but nationally attendance has fallen slightly by 2% to 63% Adviser Attendance is higher in Sept-Oct than the national average – the increase could be due to increased communication
39. Comparison Table of Adviser Attendance North West had a lower Adviser Attendance (55%) than the national average in May-June (65%) Attendance in Sept-Oct slightly increased to 67% In May-June Salford & Trafford, T.A.S.K (Rochdale, Heywood and Middleton), Tameside & Glossop and Oldham did not attend one activity day each, FagEnds Liverpool did not attend two days and Manchester did not attend three activity days In Sept-Oct Manchester did not attend seven of nine activity days and Oldham did not attend one activity day

40. Part 5: Lead Tracking May-June and Sept-Oct

41. Lead Tracking Ultimate success criteria for F2F is tracking the conversion of leads into successful quitters Feedback on how F2F leads have performed assist in measuring the effectiveness of F2F activity
42. May-June Lead Tracking Results by Region Of the seven activity days North West responded on 43% were successfully contacted and 3% were converted into quits Total number of quitters was left blank in one feedback report – this could mean no quitters or data not yet available
43. Sept-Oct Lead Tracking Results by Region North West had a higher conversion rate of referrals to successful contact in Sept-Oct than May-June from 43% to 61% but only one service replied with figures The quitter rate is lower for North West (2%) than the national average which was 3% and down 1% to May-June phase
44. Lead Tracking Response Rate May-Oct The No Reply rate has drastically risen from 42.9% in May-June to 95.8% in Sept-Oct 33.3% of leads were followed and quit rates reported in May-June but only 4.2% were reported in Sept-Oct 14.3% of services in North West have capacity issues – this could mean services have not got a system in place to follow up leads or have database issues so cannot track referrals through to quitters

45. Part 6: Learnings and Recommendations

46. Learnings – May-June Shopping Centres proved successful in May-June primarily due to the public being very receptive and a high penetration of R&M smokers All retailer venue types performed fairly well – feedback specifies regional locations and position in store Following cost negotiations and an increase in the percentage of data capture target achieved CPR dropped to £43 in May-June from £48 for New Year 2009 Event overviews and courtesy calls improved information flow and removed negative feedback From testing Next Steps leaflet proved more effective than Credit Card leaflet by filling information gap
47. Actions Implemented into Sept-Oct Activity Roving teams factored into activity to save money on site fees (tests in two regions: West and East Midlands) Deployed Next Steps leaflet across all activity and allowed LSSS to personalise through applying address label Daily targets revised based on all activity results
48. Learnings – Sept-Oct Roving demonstrated a low CPR in the regions tested and is now an integral part of the venue mix A generic approach to Lead Tracking was not generating enough response Although run on a small scale, Lung Age testing had a positive impact on sign ups and anecdotally increased interest in F2F Tesco venues suffered especially from quiet spells Asda stores did exceptionally well – the team mention Lung Age and CO testing helping sign ups as well as a high penetration of R&M smokers Good performance in Shopping Centres and retail venues improved the CPR in May-June and even more so in Sept-Oct
49. Campaign Amendments Already Implemented into New Year 2010 Activity For Sept-Oct activity quick wins were associated with cost savings as opposed to changes in venue selection strategy We implemented: New staffing rates Reduced training costs Included 28 Roving days off-set against budget savings Activity targets were reviewed to ensure that they were achievable and in-line with venue averages to manage expectations against results (whilst still maintaining quality) Following the CPR review the move from hot spot mapping to venue optimisation has been implemented with aim of increasing performance against data capture targets Telephone calls are being made to services to ensure a higher Lead Tracking response rate – individual needs of LSSSs are being established who require longer response times to report on accurate quitting figures
50. Recommendations for Future F2F Events Results from all venue types will be monitored to see if the level of roving should be increased from 25% of activity (in place of more expensive Shopping Centres) Shopping Centres and Retail venues are proving to be a success in the North West region - venue mix should optimised to further reduce CPR With clearer guidance on how to deal with high readings and the implementation Lung Age spirometers, reintroduce Lung Age testing as a regional trial and work with a region to closely monitor the effect on results A tailored approach to Lead Tracking is needed – by demonstrating flexibility to work with services more information should be gathered on the number of quitters recruited through F2F Maintain the current level of communication with services but try to establish why advisers were not able to attend i.e. build knowledge base to inform future decision making
51. Summary CPR in North West dropped from £43 in May-June to £38 in Sept-Oct Phase one of CPR analysis identified cost savings which facilitated drop in CPR for Sept-Oct Phase two of CPR analysis (venue optimisation) should further assist reduction in CPR for New Year 2010 activity Lead Tracking – a 52.9% rise in the No Reply rate since May-June has indicated a new tailor made approach is needed and recommendations have been made Adviser Attendance – increased from 55% to 67% - although this is a vast improvement increased communication in Jan-March 2010 is recommended

52. Part 7: Appendix

53. Artemis COI Artemis is a response and conversion tool that has been used across a wide range of government clients to deliver campaign insight A relatively new service (launched March 07) designed to enable evidential based planning Response and conversion analysis by audience, media channel, vehicle of response, viewed alongside awareness COI Artemis analysis has become integral to DH National Tobacco Control activity. The service has been used since 2007 to: Facilitate better planning – media buying and investment is based on the results of COI Artemis reports; Benchmark campaign performance.
54. Understanding the COI Artemis Response Hierarchy (F2F Model) All interactions at the F2F activity Total Response People who don’t interact with the activity i.e. not interested or non smokers Invalid Response Valid Response Leaflets distributed In depth interactions (not currently measured) Information Response Data capture (request for information pack, for LSSS follow up or joining Together) Active Response Respondent signs up to CRM programme (not currently measured) Contactable Respondent Referral to LSSS for follow up or joining Together Intermediate Conversion active response and intermediate conversion is the same for F2F
55. Gender – May-June There is a significant bias towards female smokers – the biggest variances are in the North West and Yorks & Humber which have a 61/39% and 62/38% split respectively London is the only region to reverse the trend with 52/48% split in favour of males Retail and Shopping Centre locations form the bulk of locations visited which predominantly attract a heavier weight of females As discussed at presentation of Jan-March results, this is not a concern as it won’t affect PSA targets
56. Gender – Sept-Oct There is a significant bias towards female smokers Retail and Shopping Centre locations form the bulk of locations visited which predominantly attract a heavier weight of females
57. Age – May-June As in Jan-March, activity is engaging with more 18–24 years old - 16% of respondents fall into this category, but all groups represented 18-24 represented highest in all regions except London (25 to 29) and North East (50 to 59) 57% of respondents within the 25-49 year old age group
58. Age – Sept-Oct As in May-June activity is engaging with more 18–24 years old - 19% of respondents fall into this category, but all groups represented 58% of respondents within the 25-49 year old age group
59. ACORN Profiling – May-June Acorn is classification that helps understand different types of people in different areas throughout the country Combines geography (we profile people using their postcode information) with demographics, lifestyle, behaviours, attitudes, media etc. For May-June responses we obtained postcodes of people who interacted with F2F and profiled them on a number of levels: 1. Overall level - grouping everybody together and comparing this against the population in England 2. Sub grouping - looking at people who engaged in F2F activity in Supermarkets, Markets and Shopping Centres 3. Sub grouping retail/supermarket by type - Asda, Tesco, Sainsbury's, Morrisons and Co-op  Data was only robust on the overall level. The break down/sub grouping figures are indicative (not a true reflection as we should be looking at audiences over 2,500 people)
60. Overall ACORN Results – May-June
61. ACORN Analysis – May-June May-June results consistent with Jan-March results so activity on track in terms of people engaging with F2F - lower socio-economic groups i.e. single parents/lone parents and struggling families. One group not represented before appears in this analysis across all supermarkets – low income Asian families Two reasons could have impact on this: Hot spot mapping bringing new locations into the schedule Assessment of locations based on hot spots/ethnic make up led to recruitment of a wider mix of ethnic backgrounds so improved performance within these areas (anecdotal feedback)
62. ACORN – Profiling – Sept-Oct For Sept-Oct responses we obtained postcodes of people who interacted with F2F and profiled them on a number of levels: 1. Overall level - grouping everybody together and comparing this against the population in England 2. Sub grouping by region – North West, North East, South West, South East, Yorks & Humber, East of England, West Midlands, East Midlands and London 3. Sub grouping by venue type - Shopping Centre, Retail – Asda, Morrisons, Sainsbury’s, Tesco, Coop, Town Centre, Market 4. Sub grouping by F2F engagement method – Static and Roving
63. ACORN – Overall Results – Sept-Oct
64. ACORN – Analysis – Sept-Oct Sept-Oct results consistent with previous results so activity on track in terms of people engaging with F2F across all regions, venue types and engagement techniques Two main categories - Hard-pressed category – contains the poorest areas of the UK; unemployment is well above the national average; levels of qualifications are low and those in work are likely to be employed in unskilled occupations; household incomes are low; e.g. single parents and pensioners (council terraces); families and single parents (semis and terraces; old people (high rise flats), low income families (terraced estates), large families and single parents (many children), low income (routine jobs), single elderly people (council flats) Moderate means category – contains much of what used to be the country’s industrial heartlands; many people are employed in traditional, blue-collar occupations; other have become employed in service and retail jobs; e.g. young working families, home-owning families, low-income Asian families Rise of a new category – Urban Prosperity category – well educated and mostly prosperous people living in our major towns and cities; e.g. singles and sharers, student flats and cosmopolitan sharers, low income singles (rented flats); mainly London, South East and South West; Shopping Centres