1 / 55

Federal By-Election Post-Test Report 2009

Evaluation of media effectiveness in transmitting election information following November 2009 by-elections in four Canadian ridings. Includes methodologies, findings, aided recall results, and conclusions to optimize future media plans.

richardedavis
Download Presentation

Federal By-Election Post-Test Report 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2009 BY-ELECTION POST-TEST 2009 FEDERAL BY-ELECTIONS POST-TEST

  2. 2009 BY-ELECTION POST-TEST TABLE OF CONTENTS

  3. 2009 BY-ELECTION POST-TEST 1. BACKGROUND, OBJECTIVE, AND METHODOLOGY TELEPHONE INTERVIEWS ONLY

  4. 2009 BY-ELECTION POST-TEST CONTEXT AND OBJECTIVE • The current document is the initial report of the advertising post-test conducted following federal by-elections held on November 9th, 2009. • The four ridings covered by the scope of the research are: • Cumberland–Colchester–Musquodoboit Valley in Nova Scotia, • Hochelaga in the province of Quebec • Montmagny-L'Islet-Kamouraska-Rivière-du-Loup in the province of Quebec, and • New Westminster-Coquitlam in British Columbia. • The main objective of the research was to evaluate which media vehicles (VIC, Reminder Card, newspaper, radio, the Internet) was most efficient in transmitting elements of information about the electoral process: a) the day of the vote; b) location and time to vote; c) advanced polls and; d) ID requirements. • The results of the research will ultimately be used to optimize the media plan in future federal elections. • In addition to the telephone surveys, an additional Internet component was conducted in one of the four ridings to measure the impact of the aided recall of the different elements of the media plan. 1. BACKGROUND, OBJECTIVE, AND METHODOLOGY

  5. 2009 BY-ELECTION POST-TEST MEDIA PLAN OVERVIEW • Overview of the media plan used in the by-election. • The strategy allowed us to evaluate the impact of: • The type of community: Urban vs. Rural ridings • The type of newspapers used: Dailies vs. Weeklies • The number of radio advertising flights: 2 vs. 1 • The detailed media plan for each riding are presented in the Appendix. 1. BACKGROUND, OBJECTIVE, AND METHODOLOGY

  6. 2009 BY-ELECTION POST-TEST METHODOLOGY • Quantitative study • Method: Telephone interviews • Data collection: November 10-19, 2009 • Number of respondents: 800 • Number of respondents per riding: 200 • Response rate: 25% • Margin of error: 6.9% at 95% confidence level • Weighting: Age, gender, Voter turnout 1. BACKGROUND, OBJECTIVE, AND METHODOLOGY

  7. 2009 BY-ELECTION POST-TEST 2. SUMMARY FINDINGS AND CONCLUSIONS TELEPHONE INTERVIEWS ONLY

  8. 2009 BY-ELECTION POST-TEST SUMMARY FINDINGS – AWARENESS • By-election - The awareness level ranged from 80% in Hochelaga to 97% in Montmagny. Past Impact Research post-tests following by-elections usually yield by-election awareness levels of 90% or more. • ID Requirements – 95% claimed to have known about ID requirements even though only half mentioned having heard about it during the recent campaign. 2. SUMMARY FINDINGS AND CONCLUSIONS

  9. 2009 BY-ELECTION POST-TEST SUMMARY FINDINGS – POST-TEST / AIDED RECALL • Of the five elements of the media plan tested, the Voter Information Card obtained the best results in terms of aided recall. Of all those aware of the by-election, 78% recalled having received the VIC at home. The aided recall level is comparable to what is usually observed in by-election post-tests. • The householder (38%), the newspaper (35%), and the radio (31%) received somewhat similar aided recall level. • The Internet banner received the lowest aided recall level. 2. SUMMARY FINDINGS AND CONCLUSIONS

  10. 2009 BY-ELECTION POST-TEST SUMMARY FINDINGS – SOURCES OF AWARENESS • On an unaided basis, the Voter Information Card was key to inform the population about the elements of information regarding the voting process (awareness of the date of the vote, advanced polls, location and hours, and ID requirements). • The unaided score of the VIC varies from 34% (advanced polls) to 56% (voting locations and hours). The score of the VIC is three to four times greater then the following awareness sources. • After the mail (VIC, Householder), the newspaper (weeklies / dailies combined) was the second most important source of information about the voting process. Radio ads were well behind. 2. SUMMARY FINDINGS AND CONCLUSIONS

  11. 2009 BY-ELECTION POST-TEST CONCLUSIONS • RADIO – One flight or two flights? • Results of the survey indicate that a second radio flight helped increase reach (Cumberland vs. the other 3 ridings). However, it did not help recall of the specific elements of information within the ad. • Overall, the performance of radio was stronger in rural ridings (aided recall of the ad / aided recall of the content). 2. SUMMARY FINDINGS AND CONCLUSIONS

  12. 2009 BY-ELECTION POST-TEST CONCLUSIONS • DAILIES OR WEEKLIES - Should the media plan focus solely on dailies, solely on weeklies or on a mix of both? • Results indicate that the situation is not black and white, that it varies from one riding to another. • In Cumberland, a rural riding where dailies were purchased, the media buy was very efficient. Of those who claimed to have seen a newspaper ad, 71% mentioned on an unaided basis, one of the dailies purchased. In Montmagny, also a rural (but geographically spread riding), the use of weeklies also worked efficiently. First of all, the highest aided recall level was observed in Montmagny. Of those who claimed to have seen a newspaper ad, 57% mentioned one of the five weeklies purchased. However, 28% mentioned having seen it in a daily (which were not in the media plan. In the other two ridings, the results were not very conclusive. • RURAL vs. URBAN – Comparable or not? • Results indicate that the information is easier to transmit in rural ridings. • The awareness level of the by-election, as well as that of key information about the electoral process are greater or similar in rural ridings compared to urban ones. The same can be said of the aided recall of the elements of the ad campaign. 2. SUMMARY FINDINGS AND CONCLUSIONS

  13. 2009 BY-ELECTION POST-TEST 3. PROFILE TELEPHONE INTERVIEWS ONLY

  14. 2009 BY-ELECTION POST-TEST PROFILE – GENDER AND AGE 3. PROFILE

  15. 2009 BY-ELECTION POST-TEST PROFILE – EDUCATION 3. PROFILE

  16. 2009 BY-ELECTION POST-TEST PROFILE – ETHNIC / CULTURAL BACKGROUND 3. PROFILE

  17. 2009 BY-ELECTION POST-TEST PROFILE – CURRENT OCCUPATION 3. PROFILE

  18. 2009 BY-ELECTION POST-TEST PROFILE – MEDIA 3. PROFILE

  19. 2009 BY-ELECTION POST-TEST 4. RESULTS TELEPHONE INTERVIEWS ONLY

  20. 2009 BY-ELECTION POST-TEST 4.1 AWARENESS OF THE BY-ELECTION AND VOTE TELEPHONE INTERVIEWS ONLY

  21. 2009 BY-ELECTION POST-TEST AWARENESS AND VOTE • Awareness of by-election – In total, 89% of all respondents were aware of a federal by-election in their riding. The awareness level ranged from 80% in Hochelaga to 97% in Montmagny. Except for one exception in the past (Mount-Royal 1999), the awareness level of an upcoming election has always been above 90%. • The awareness level was greater amongst respondents 35 years and older (92% vs. 82%). • Voter turnout – Of all respondents, 36% voted (weighted data) either via advanced polls or on election day. • The awareness level was greater amongst respondents aged 45 years and older (43% vs. 27%). • Of those who voted, the vast majority (88%) did so on election day. • 98% had the proper IDs at the polls. • Of those who did not have the proper IDs, 3 were vouched for by another elector and 1 had to return for his/her IDs. 4.1 AWARENESS OF THE BY-ELECTION AND VOTE

  22. 2009 BY-ELECTION POST-TEST AWARENESS LEVELS OF KEY INFORMATION • Advanced polls – Of those who knew about the by-elections, 82% were aware of advanced polls. The awareness level was greatest in Montmagny (93%). The proportion was also greater amongst respondents aged 45 years and older (86% vs. 78%), and those who voted (93% vs. 76%). • Voting locations and hours – Of those who knew about the by-elections, 78% were aware of voting locations and time. At 90%, the proportion was greatest in Cumberland (NS). The proportion was also greater amongst respondents aged 45 years and older (84% vs. 71%) and those who voted (91% vs. 70%). • ID requirement – The vast majority (95%) of those who knew about the by-elections knew about the ID requirements. The awareness level is greater amongst 18-34 year olds (99% vs. 94%).  Of those, only half claimed to have heard about the ID requirements during the recent by-elections. 4.1 AWARENESS OF THE BY-ELECTION AND VOTE

  23. 2009 BY-ELECTION POST-TEST 4.2 POST-TEST TELEPHONE INTERVIEWS ONLY

  24. 2009 BY-ELECTION POST-TEST POST-TEST – AIDED RECALL • Of the five elements of the media plan tested, the Voter Information Card obtained the best results in terms of aided recall. Of all those aware of the by-election, 78% recalled having received the VIC at home. • The aided recall level of the VIC is in line with previous post-test results which ranged between 57% to 86%. • The householder (38%), the newspaper (35%), and the radio (31%) received somewhat similar aided recall level. • The newspaper ad was especially strong in Montmagny (52%). • The radio ad(s) was especially strong in Cumberland (40%). • Historically, the aided recall for newspaper ads vary greatly from one by-election to the other (16% to 67%). The same observation applies to the householder results (24% to 61%) and to radio (14% to 68%). • The Internet ad was seen by 7% of all respondents (aware of the by-election). 4.2 POST-TEST

  25. 2009 BY-ELECTION POST-TEST POST-TEST – AIDED RECALL • Aided recall is greater amongst the following subgroups… • Voter Information Card • Female (83% vs. 74%) • Voters (89% vs. 72%) • Householder / Reminder Card • Voters (44% vs. 34%) • Newspaper • Montmagny residents (52%) • Voters (43% vs. 31%) • Radio • Cumberland residents (40%) • High school (34%) and college graduate (41%) vs. university graduates (21%) • Internet • No significant differences observed 4.2 POST-TEST

  26. 2009 BY-ELECTION POST-TEST 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION TELEPHONE INTERVIEWS ONLY

  27. 2009 BY-ELECTION POST-TEST KEY INFORMATION REGARDING THE VOTING PROCESS – UNAIDED SOURCES OF AWARENESS • On an unaided basis, the Voter Information Card was key to inform the population about the main information regarding the voting process (awareness of the date of the vote, advanced polls, location and hours, and ID requirements). • For the four aforementioned elements of information, the VIC sits atop the list of unaided awareness sources. The scores vary from 34% for the awareness of the advanced polls to 56% for voting locations and hours. The score of the VIC is three to four times greater then the following awareness sources. • After the mail (VIC, Householder), the newspaper (weeklies / dailies combined) was the second most important source of information about the voting process. Radio ads were well behind. • Elections Canada website was mentioned as a source of information for ID requirements. • Newspaper – As a whole, dailies were more often mentioned than weeklies for information regarding the date of the election and the advanced polls. For the other two elements of information (voting locations and hours and ID requirements, weeklies and dailies were equally mentioned. 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  28. 2009 BY-ELECTION POST-TEST DATE OF THE ELECTION – UNAIDED SOURCES OF AWARENESS Q: “Where did you get your information about the date of the election, that is, November 9th? Were there any other sources?” UP TO 5 ANSWERS 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  29. 2009 BY-ELECTION POST-TEST ADVANCED POLLS – UNAIDED SOURCES OF AWARENESS Q: “Where did you get your information about the dates for advanced voting? Were there any other sources?” UP TO 5 ANSWERS 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  30. 2009 BY-ELECTION POST-TEST VOTING LOCATIONS AND HOURS – UNAIDED SOURCES OF AWARENESS Q: “Where did you get your information about voting locations and hours? Were there other sources?” UP TO 5 ANSWERS 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  31. 2009 BY-ELECTION POST-TEST ID REQUIREMENTS – UNAIDED SOURCES OF AWARENESS Q: “During the by-election, where did you get your information about these requirements? Were there any other sources?” UP TO 5 ANSWERS 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  32. 2009 BY-ELECTION POST-TEST SOURCES OF AWARENESS OF KEY INFORMATION - AIDED • On an aided basis, the VIC was again the best vehicle to inform about the electoral process. More than 80% of those who remember receiving the VIC saw the date of the election and the voting locations and hours on it. • Around two-thirds of those who received the VIC saw the information regarding advanced polls and ID requirements, the highest score amongst the five elements of the ad campaign tested. Q: “Do you remember seeing or reading the following information on [the measured medium]… (VIC, Reminder Card…)?” 4.3 SOURCES OF AWARENESS OF ELEMENTS OF INFORMATION

  33. 2009 BY-ELECTION POST-TEST 4.4 DIFFERENCES WEEKLIES VS. DAILIES RADIO RURAL / URBAN TELEPHONE INTERVIEWS ONLY

  34. 2009 BY-ELECTION POST-TEST RURAL VS. URBAN • A few differences were observed between rural (Cumberland and Montmagny) and urban ridings (New Westminster and Hochelaga). • The following awareness levels were greater in rural ridings: • By-election (96% vs. 82%). • Advanced polls (87% vs. 77%). • Voting locations and hours (82% vs. 74%). • The aided recall rate of the following were greater in rural ridings: • Newspaper (42% vs. 27%) • Radio (36% vs. 24%) 4.4 DIFFERENCES

  35. 2009 BY-ELECTION POST-TEST RADIO – TWO FLIGHTS VS. ONE • CONTEXT - In the Cumberland riding two radio flights were purchased compared to one flight for the other three ridings. In all four ridings, one flight was aired only a few days before election day. • The aided recall rate of the radio ad is greatest in the riding where two flights were purchased (40% vs. 33% / 25% / 24%). • Overall, the performance of radio was stronger in rural ridings. • Aided recall was stronger in rural ridings. • Respondents who remember hearing the radio ad(s) recall more elements of information. 4.4 DIFFERENCES

  36. 2009 BY-ELECTION POST-TEST WEEKLIES VS. DAILIES • CONTEXT – To verify which, weeklies or dailies, performs better in terms of reach and transmitting elements of information regarding the vote, the decision was made to buy mostly weeklies in some ridings and mostly dailies in others. • Most of the print buys were weeklies in 2 ridings (New Westminster (BC) and Montmagny (Qc)) as opposed to dailies for the other two ridings Cumberland (NS) and Hochelaga (Qc). • The aided recall rate of the newspaper ad was greater where weeklies were purchased (41% vs. 28%). This is mostly due to the very high recall rate in Montmagny (52%), where five French weeklies were purchased to cover the riding. 4.4 DIFFERENCES

  37. 2009 BY-ELECTION POST-TEST POST-TEST – AIDED RECALL NEWSPAPERS • In Cumberland and Hochelaga, mostly dailies were purchased • Cumberland - Of those who recall seeing a newspaper ad, 71% mentioned one or several of the dailies purchased for the campaign. • Most of the buy was focused on dailies. Only a French weekly was purchased. • Hochelaga - Of those who recall seeing a newspaper ad, only 33% mentioned one or several of the dailies purchased. • In Montmagny and New Westminster, mostly weeklies were purchased • Montmagny - Of those who recall seeing a newspaper ad, 57% mentioned one or several of the weeklies purchased. • New Westminster - Of those who recall seeing a newspaper ad, only 14% mentioned one or several of the weeklies purchased. Note: Comparisons were made between Cumberland and Hochelaga AND between Montmagny and New Westminster. 4.4 DIFFERENCES

  38. 2009 BY-ELECTION POST-TEST SOURCES OF UNAIDED AWARENESS - NEWSPAPERS • Dailies • Cumberland - Of those who recall seeing a newspaper ad, 46% specifically mentioned the Halifax Chronicle Herald and 28% the Truro Daily News, both of which were bought. • Hochelaga - Of those who recall seeing a newspaper ad, 29% mentioned Le Journal de Montréal which is on the media plan. However, 38% mentioned La Presse which was not bought for the by-election. • Weeklies • Montmagny - Of those who recall seeing a newspaper ad, 57% mentioned one or several of the weeklies purchased. The five French weeklies purchased were mentioned by at least 8%. Little duplications were observed between publications (net aided recall: 57% / only 9% mentioned more than 1). • New Westminster - Of those who recall seeing a newspaper ad, 14% mentioned one of the two weeklies bought (the New Westminster Now / Royal City Record). Several (27%) were not able to specifically name the newspaper. Three dailies (which did not appear on the media plan) were mentioned: Vancouver Sun (21%), The Province (15%), and The Tri-City News (13%) 4.4 DIFFERENCES Base: Those who recall seeing a newspaper ad.

  39. 2009 BY-ELECTION POST-TEST B. COMPARISON BETWEEN TELEPHONE AND INTERNET SURVEYS

  40. 2009 BY-ELECTION POST-TEST METHODOLOGY – ALTERNATIVE • In addition to the telephone surveys, an additional Internet component was conducted in the New Westminster / Coquitlam riding. • The objective of the alternative methodology was to measure the impact of the methodology on the results, especially the aided recall level of the different elements of the media plan. • Number of respondents: 199 • Data collection: November 16-21, 2009 • Weighting: As was the case with the telephone results, Internet results were weighed based on age, gender, and voter turnout. TELEPHONE VS. INTERNET SURVEYS

  41. 2009 BY-ELECTION POST-TEST DIFFERENCES BETWEEN INTERNET AND TELEPHONE • Profile – Internet respondents were less educated and had a greater number of “Canadian” respondents. A greater proportion of them read dailies and weeklies. • Aided recall – Interestingly, the aided recall of four of five elements of the ad campaign were comparable between Internet and telephone respondents. Only the aided recall of the newspaper ad was lower amongst Internet respondents even though a greater proportion of them read weeklies (which was what was bought in New Westminster). • Sources of awareness – The biggest differences were observed in the sources of awareness of key information regarding the voting process. Given the similarities in terms of profile, the differences are possibly due to the differences in methodology: • All the potential media (VIC, newspaper – daily, Elections Canada website…) were listed to Internet respondents whereas the list was not read to respondents over the phone. • With Internet respondents, we showed the different pieces of the campaign. Although these pieces were masked, this methodology may have helped respondents remember the information presented on these pieces. TELEPHONE VS. INTERNET SURVEYS

  42. 2009 BY-ELECTION POST-TEST PROFILE – GENDER AND AGE TELEPHONE VS. INTERNET SURVEYS

  43. 2009 BY-ELECTION POST-TEST PROFILE – ETHNIC / CULTURAL BACKGROUND TELEPHONE VS. INTERNET SURVEYS

  44. 2009 BY-ELECTION POST-TEST PROFILE – MEDIA The proportion of those who read weeklies is greater amongst Internet respondents. TELEPHONE VS. INTERNET SURVEYS

  45. 2009 BY-ELECTION POST-TEST AWARENESS LEVELS OF KEY ELECTORAL INFORMATION • The awareness of the New Westminster by-election was greater amongst the Internet respondents. • Besides that difference, awareness scores are comparable between the two samples. TELEPHONE VS. INTERNET SURVEYS

  46. 2009 BY-ELECTION POST-TEST POST-TEST – AIDED RECALL • Aided recall between the two samples are comparable on four of the five elements of the ad campaign tested: VIC, Householder, Radio ad, and Internet banners. The aided recall of the newspaper ad was significantly lower amongst Internet respondents. • It is important to note that on the phone, the elements of the ad campaign were described whereas on the Internet, masked versions were shown. The radio ad was not played to either telephone or Internet respondents because once the ad was masked, there was not enough to play. TELEPHONE VS. INTERNET SURVEYS

  47. 2009 BY-ELECTION POST-TEST DATE OF THE ELECTION – SOURCES OF AWARENESS • In this case (and for the next three slides), the importance of the direct mail is even greater amongst Internet respondents, the Householder especially. Q: “Where did you get your information about the date of the election, that is, November 9th? Were there any other sources?” Telephone: Unaided multiple mentions: Internet: Aided mentions • Internet: • Internet (news content): 7% • Elections Canada website: 5% TELEPHONE VS. INTERNET SURVEYS

  48. 2009 BY-ELECTION POST-TEST ADVANCED POLLS –SOURCES OF AWARENESS Q: “Where did you get your information about the dates for advanced voting? Were there any other sources?” Telephone: Unaided multiple mentions: Internet: Aided mentions • Internet: • Elections Canada website: 3% TELEPHONE VS. INTERNET SURVEYS

  49. 2009 BY-ELECTION POST-TEST VOTING LOCATIONS AND HOURS –SOURCES OF AWARENESS Q: “Where did you get your information about voting locations and hours? Were there other sources?” Telephone: Unaided multiple mentions: Internet: Aided mentions • Internet: • Radio ads: 4% • Elections Canada website: 3% TELEPHONE VS. INTERNET SURVEYS

  50. 2009 BY-ELECTION POST-TEST ID REQUIREMENTS – SOURCES OF AWARENESS Q: “During the by-election, where did you get your information about these requirements? Were there any other sources?” Telephone: Unaided multiple mentions: Internet: Aided mentions • Internet: • Internet banners: 5% • Internet (news content): 5% TELEPHONE VS. INTERNET SURVEYS

More Related