1 / 22

The Annual Structural Business Survey Developing and Testing an Electronic Form

The Annual Structural Business Survey Developing and Testing an Electronic Form. Ger Snijkers Evrim Onat Rachel Visschers Statistics Netherlands Division of Business Statistics. 1. Dutch Annual Structural Business Survey.  Annual survey of economic activity  Mandatory

shiri
Download Presentation

The Annual Structural Business Survey Developing and Testing an Electronic Form

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Annual Structural Business SurveyDeveloping and Testing an Electronic Form Ger Snijkers Evrim Onat Rachel Visschers Statistics Netherlands Division of Business Statistics ICES3, June 18-21, 2007, Montreal

  2. 1. Dutch Annual Structural Business Survey •  Annual survey of economic activity •  Mandatory •  75.000 business each year: •Sample of small firms, bigger firms each year •  Until now: paper questionnaire • • length may differ, 20 pages is typical •  Three parts: • • revenues and costs • • summary of business accounts: profits and losses • • industry specific specifications •  ICES3, June 18-21, 2007, Montreal

  3. 2. The paper questionnaireCharacteristics  A4 booklet • Right page: items • Left page: help texts - long and voluminous  Items are grouped in sections • Long sections • Completion process: complicated and hard • Large amount of detailed information• Broad range of business information: several departments, several respondents • No match with definitions used, but same label: CBS – business definitions • Lay-out: misinterpratations and errors ICES3, June 18-21, 2007, Montreal

  4. Vernieuwde huisstijl:2. Onderzoek naar de PS • Q oud ICES3, June 18-21, 2007, Montreal

  5. 3. The web questionnaireProject goal  Develop a web questionnaire • same contents • mixed-mode design: paper and web • support completion process: - motivate respondents to use this mode • into the field: March 2006  Start of project: June 2004  In the field: Spring 2006 ICES3, June 18-21, 2007, Montreal

  6. 3. The web questionnaire Developing and testing In five stages: 1. Testing the prototype (31-1-’05) • pre-tests to test usability: 3 waves 2. Revision of questionnaire (1-9-’05) • expert reviews 3. Testing of revised questionnaire (1-1-’06) • additional usability tests 4. Implementation of field pilot (1-3-’06) 5. Implementation of survey (1-3-’07) ICES3, June 18-21, 2007, Montreal

  7. 3.1. The web Q: The prototype ICES3, June 18-21, 2007, Montreal

  8. 3.1. The prototype: 3 test waves ICES3, June 18-21, 2007, Montreal

  9. 3.1. The prototypeResearch issues 1. How does the e-form work in practice? • Completing the questionnaire • Question-and-answer process • What features should be included to make it easy to use? • Respondent friendly: ‘Computer-assisted’ tools • User wishes 3. How should the web Q be designed in relation to the paper Q? • The same or a different design • ‘look-and-feel’ of paper and e-form ICES3, June 18-21, 2007, Montreal

  10. 3.1. The prototypeResearch issues 1. How does the Q work?  Laborious and complex process • Long, complex questionnaire (≥ 25 items) • Complex completion process: -several sessions, several informants - kick-and-rush behaviour • Imagine ... a respondent sitting behind his/her computer ...  Respondent got lost in the questionnaire ICES3, June 18-21, 2007, Montreal

  11. 3.1. The prototypeResearch issues 2. Features to make it easy to use? • What am I supposed to do (next)? • Easy to download, install, complete, send data back • It is one process: downloading – sending data back, • Clear instructions and explanations (but not read) • How is the questionnaire built up? • Show how the questionnaire is structured: overview • Help to find the way in the questionnaire • No hidden rules, no unexpected functionalities • Where am I? What did I do so far? • Provide overview of the completion process • Clear navigation, no scrolling • Printing function ICES3, June 18-21, 2007, Montreal

  12. 3.1. The prototypeResearch issues 2. Features to make it easy to use? Con’d • Printing • Passing on sections of Q to other departments • Checking the data before transmitting • Getting authorisation to release the data • Calculations • Well accepted, … even expected • Entry-search • ‘Google’-like search: “Where to put these data?” • Navigation and overview • Choose a setup people are familiar with: - setup of tax office, windows explorer ICES3, June 18-21, 2007, Montreal

  13. 3.1. The prototypeResearch issues 3. Design of paper and web Q? • The computer is different than paper • The web Q reacts to the respondent • Reading from the screen is different • Navigating and getting an overview works differently • Kick-and-rush behaviour, even stronger than on paper • The use of computer-assisted functionalities; the respondent expects the computer to react ICES3, June 18-21, 2007, Montreal

  14. 3.1. The prototype Conclusions of pre-test waves • Visual design  Clear and logical  Simple, transparent, consistent  No hidden and unexpected functionalities • Support the completion process  Other mode, other features, other visual design  Different than paper form, same ‘look-and-feel’ • Tailor to kick-and-rush behaviour  Small sections, small tasks  Short and clear explanations ICES3, June 18-21, 2007, Montreal

  15. 3.2. The revised questionnaire Based on: • Pre-test results • Expert reviews • Iterative process with - Professional designer - Questionnaire designers - Methodologists • A user friendly design was put first, not the IT tool • New prototypes designed in Power Point ICES3, June 18-21, 2007, Montreal

  16. ICES3, June 18-21, 2007, Montreal

  17. 3.3. The revised questionnaire Conclusions of add. pre-tests • 10 concurrent in-depth interviews • Usability and user friendliness has been improved  respondents enjoyed working with the questionnaire  they could handle the task  even though … the task had not changed  Web questionnaire design is communication design ICES3, June 18-21, 2007, Montreal

  18. 3.4. Field pilot Set-up:  March-July 2006  7200 businesses, 5 industries  Advance letter with • internet address: www.cbs.nl/productiestatistiek • user name and password • leaflet to introduce web questionnaire and explain why the survey is conducted • paper questionnaire not mentioned ICES3, June 18-21, 2007, Montreal

  19. Leaflet ICES3, June 18-21, 2007, Montreal

  20. 3.4. Field pilot Goals:  Implementation of web questionnaire • paper and web flows  Test the whole process • downloading – completing – sending-in data • investigate completion process in the field: - debriefing interviews with respondents - audit trails  Response rates  Data quality • data editing, mode effects ICES3, June 18-21, 2007, Montreal

  21. 3.5. Survey Now running  About 75.000 business received this questionnaire ICES3, June 18-21, 2007, Montreal

  22. Web questionnaire design is communication design ICES3, June 18-21, 2007, Montreal

More Related