Listening to Respondents for Better Results

1. Listening to Respondentsfor Better Results Alexander Hays Distributive Trades Division Statistics Canada ICES III - Montréal, Canada

2. Content • The Statistics Act • Surveys • Inter-provincial Wholesale Commodity Survey • Annual Retail Chain Survey • Response Rates • Beginning the Collaborative Process • Action Plan • Consultative Results • Actions from the Consultation • Respondent Reaction • Continuing Challenges • Conclusion ICES III - Montréal, Canada

3. The Statistics Act • Gives Statistics Canada the authority to collect the necessary information directly from respondents, and for business surveys, makes the reporting mandatory. Grants Statistics Canada the authority to collect, compile, analyse, and publish information. ICES III - Montréal, Canada

4. The Statistics Act • Considers refusal or the provision of false or misleading information a criminal offence subject to penalties. • Statistics Canada has hesitated in the past from taking legal action for non-compliance. ICES III - Montréal, Canada

5. Statistics Canada has favoured a collaborative approach working towards a viable solution satisfactory to both parties. ICES III - Montréal, Canada

6. Inter-provincial Wholesale Commodity Survey • Origin and destination by commodity survey. • Measures the value of inter-provincial trade by Canadian wholesalers. • Better understanding of provincial growth and measures of Gross Domestic Product (GDP) by province. • 1990, 1996, 2001. • Preset list of 103 commodities. ICES III - Montréal, Canada

7. Annual Retail Chain Survey • Operating and financial characteristics of domiciled retail corporate chain stores and department stores. • Better understanding of provincial economic growth and measures of GDP by province. • Retail Chain – 4 or more retail locations. • Location-based survey (individual chain stores) • Revenue and major expense items at the store location ICES III - Montréal, Canada

8. DISCONNECT Data Needs versus Data Availability ICES III - Montréal, Canada

9. Inter-provincial Wholesale Commodity Survey • Single: one province / territory, one NAICS industry • Complex: either one province / territory, multi NAICS or one NAICS, multi province / territory • Very Complex: multi province / territory and more than one NAICS industry ICES III - Montréal, Canada

10. Annual Retail Chain Survey ICES III - Montréal, Canada

11. Respondents willingness and ability to answer • understand the question • understand the concept • understand the terminology used • appropriateness of the question Question: Were the two surveys too onerous to complete? ICES III - Montréal, Canada

12. Beginning the Process –Inter-provincial Wholesale Commodity Survey • The 2001 survey was treated as a supplement to the Annual Wholesale Trade (Financial) Survey. • Did the supplement allow respondents the choice between completing 1 of the 2 aspects of the questionnaire under time or ease of data availability constraints? • Were the pre-set commodities not relevant to the wholesale operation in question? • Was origin and destination information not readily available from company books and records? ICES III - Montréal, Canada

13. Beginning the Process –Annual Retail Chain Survey • Studied ways to reduce and simplify the questionnaire to increase response rates. • Could model individual store level expenses and attain comparable estimates. • The 3 expense variables at the store location level were removed for the 2004 reference year. • Some of our clients expressed concerns. • Decided to study further: • Study: Impact on provincial – territorial gross margins. • Go to the respondent to find other opportunities to improvement the questionnaire and to understand reporting concerns. Question: How could we allay client concerns and improve response rates? ICES III - Montréal, Canada

14. Collaborative Approach Action Plan: • Work with Questionnaire Design and Resource Centre • Mock up sample questionnaires that meet the data requirements of Statistics Canada • Test questionnaires on selected respondents ICES III - Montréal, Canada

15. Action Plan Inter-provincial Wholesale Commodity Survey • 27 one-on-one interviews. • Wholesalers operating in a variety of NAICS classifications. • Wholesalers operating from either one of Canada’s 3 largest cities. Annual Retail Chain Survey • 23 one-on-one interviews. • Retailers operating in a variety of NAICS classifications. • Retailers with accounting functions operating out of Toronto or Montreal. ICES III - Montréal, Canada

16. Consultative Results –Inter-provincial Wholesale Commodity Survey • Origin of commodity more difficult than destination – wholesalers are sales driven. • Origin – billing or shipping address? • STC commodity list has no bearing on respondents product lines. • Once a reporting mistake is made – tend to be repeated year after year. • Reporting guides are not read. • One-on-one personalized help is often needed, e.g., Enterprise Portfolio Management Program. ICES III - Montréal, Canada

17. Consultative Results –Annual Retail Chain Survey • Due to the large number of store locations, a fully functional spreadsheet or relational database application would make it easier to complete the survey. • Respondents were happy that the 3 expense variables were removed. • Adding additional variables would require respondents to seek help from inside their organization, e.g., systems ICES III - Montréal, Canada

18. Actions from the Consultation Annual Retail Chain Survey • The 3 expense variables not re-added to questionnaire. • The electronic version of the questionnaire was made more user-friendly. Inter-provincial Wholesale Commodity Survey • Fully integrate the questionnaire into the Annual Wholesale Survey. • 3-tier collection strategy based on company size. • Mail-out / Mail-back • Telephone Assistance • On-site collection • Eliminate the commodity dimension • Conduct a pilot project due to unproven collection strategy. ICES III - Montréal, Canada

19. Respondent Reaction Annual Retail Chain Survey • The simple reduction of the 3 expense variables turned-around many non-compliant respondents. • Improved Response Rates Inter-provincial Wholesale Commodity Survey • Pilot project, although still on-going, is showing a 60% return with some level of origin & destination. • So far 59% of most complex companies responding to origin & destination. • Telephone assistance in completing the questionnaire is helping to convert non or partial response. ICES III - Montréal, Canada

20. Inter-provincial Wholesale Commodity Survey Obtaining information still requires a lot of time and effort. Acknowledgement of need for personalized assistance, but no enterprise has agreed to on-site collection by a Statistics Canada representative. Annual Retail Chain Survey Though response rates improved greatly, and the study on the Impact on Provincial Gross Margins showed little degradation in store level estimates, work continues in understanding the financial differences and the decision making process at the store level. Trade-off between Continuing Challenges ICES III - Montréal, Canada

21. Conclusion • Respondents: • Fewer questions = less burden • Questionnaires structured to respondent data sources. • Better understanding by Statistics Canada on respondents concerns. • Statistics Canada • Better understanding by the respondent on the need for the data. • Better understanding of record keeping and reporting capability. • Better response rates. • Information collection will always remain a contentious issue. • Respondent’s perspective – surveys are time consuming and can be conceptually impractical. • Listening to respondents can be a win-win situation. ICES III - Montréal, Canada

22. Listening to Respondentsfor Better Results Questions ICES III - Montréal, Canada