E-Survey Workshop : Guidelines I and Practice

1. E-Survey Workshop : Guidelines I and Practice 曾淑芬 元智大學資訊社會學研究所

2. What is Electronic Survey? Email Survey Web-based Survey Why E-Survey? The rapid development of surveys on the WWW is leading some to argue that soon internet surveys will replace traditional methods of survey data collection. Major advantages have been mentioned: cost savings, speed, limited geographical constraint, ease of contacting respondents, more efficient data processing, ability for customization. Others are urging caution or even voicing skepticism about the future role Web surveys will play.

3. Promises and Challenges Promises • Strong reach / Penetration • Fast response speed • Low cost • Response flexibility • Control of anonymity • Minimized data-entered error • Minimized interviewer bias

4. Promises and Challenges Challenges • Generalizability and Response Rate: response rate to internet based surveys have been declining over the past 10 years • Accessibility: no national or global online directory of email addresses exist • Design Concerns: use design survey response features judiciously to maximize data quality and minimize error

5. Sources of Errors in Web-based Survey Coverage Error • The result of all units in a defined population not having a known nonzero probability of being included in the sample drawn to represent the population. A mismatch between the target population and the frame population.

6. Sources of Errors in Web-based Survey Coverage Error • Universal coverage of the Web remains quite limited. Coverage error represents the biggest threat to the representativeness of sample surveys conducted via the internet. • Some populations do not exhibit large coverage problems: employees of certain organizations, members of professional organizations, certain types of businesses, students at many universities and college, and groups with high levels of education

7. Sources of Errors in Web-based Survey Sampling error The result of surveying a sample of the population rather than the entire population. Not all members of the frame population are measured.

8. Sources of Errors in Web-based Survey Sampling error • While coverage error refers to people missing from the frame, sampling error arises during the process of selecting a sample from the frame population, necessitating a means of identifying people on the frame. • A misguided assumption behind many Web surveys that large samples necessarily mean more valid responses.

9. Sources of Errors in Web-based Survey Measurement error The result of inaccurate responses that stem from poor question wording, poor interviewing survey mode effects and/or some aspect of the respondent’s behavior.

10. Sources of Errors in Web-based Survey Measurement error • Measurement error is the deviation of the answers of respondents from their true values on the measure. • The appearance of a survey can vary from respondent to respondent because of different browser settings, user preferences, variations in hardware, and so on. • There is much work to be done to determine optimal designs for different groups of respondents and types of surveys.

11. Sources of Errors in Web-based Survey Non-response error The result of non-response from people in the sample, who, if they had responded, would have provided different answers to the survey questions than those who did respond to the survey.

12. Sources of Errors in Web-based Survey Non-response error • Non-response error arises through the fact that not all people included in the sample are willing or able to complete the survey. • Non-response error is a function of both the rates of non-response and of the differences between respondents and non-respondents on the variables of interest. • There is at present little experimental literature on what works and what does not, in terms of increasing response rates to web surveys.

13. Types of Web survey (Couper, 2001) • Probability-based Web-based surveys Intercept surveys List-based samples of high coverage populations Mixed-mode designs with choice of completion method Pre-recruited panels of internet users Probability samples of full population • Non-Probability-based Web-based survey Polls as entertainment Unrestricted self-selected surveys Volunteer opt-in panels

14. Seven Response Types for Web-based Surveys (Bosnjak and Tuten, 2001) • Unit non-responders • Complete responders • Answering drop-outs • Lurkers • Lurking drop-outs • Item non-responders • Item non-responding dropouts

15. Seven Response Types for Web-based Surveys (Bosnjak and Tuten, 2001) Source: Bosnjak and Tuten,2000 (http://ascusc.org/jcmc/vol6/issue3/boznjak.html)

16. Design of Web-bases Surveys (Dillman and Bowker, 2001) • Introduce the web questionnaire with a welcome screen (Non-response error) • Provide a PIN number (Sampling, Coverage) • First question should be interesting to most respondents, easily answered, and fully visible on the first screen of the questionnaire. (Non-response) • Present each question in a conventional format similar to that normally used on paper self-administered questionnaires. (Measurement, Non-response)

17. Design of Web-bases Surveys (Dillman and Bowker, 2001) (Cont.) • Restrain the use of color so that figure/ground consistency and read-ability are maintained, navigational flow is unimpeded, and measurement properties of questions are maintained. (Measurement) • Avoid differences in the visual appearance of questions that result from different screen configurations, operating systems, browsers, partial screen displays and wrap-around text. (Coverage, Measurement, Non-response)

18. Design of Web-bases Surveys (Dillman and Bowker, 2001) (Cont.) • Provide specific instructions on how to take each necessary computer action for responding to the questionnaire and other necessary instructions at the point where they are needed. (Non-response) • Use drop-down boxes sparingly, consider the mode implication, and identify each with a “click here” instruction. (Measurement) • Do not require respondents to provide an answer to each question before being allowed to answer any subsequent ones. (Non-response)

19. Design of Web-bases Surveys (Dillman and Bowker, 2001) (Cont.) • Provide skip directions in a way that encourages marking of answers and being able to click to the next applicable question. (Measurement) • Construct web questionnaires so they scroll from question to question unless order effects are a major concern, and/or telephone and web survey results are being combined. (Coverage, Measurement, Non-response) • When the number of answer choices exceeds the number that can be displayed in a single column on one screen, consider double-banking with an appropriate grouping device to link them together. (Measurement)

20. Design of Web-bases Surveys (Dillman and Bowker, 2001) (Cont.) • Use graphical symbols or words that convey a sense of where the respondent is in the completion process, but avoid ones that require significant increases in computer memory. (Coverage, Non-response) • Exercise restraint in the use of question structures that have known measurement problems on paper questionnaires, e.g. check-all-that-apply and open-ended questions. (Measurement, Non-response)

21. Other Web Survey Design Suggestion: Zanutto (2001) • Use a cover letter with the questionnaire • Make the survey simple, and have it take no longer than 20 minutes • Give an estimated time that it will take to complete the survey • Be sure the first question is interesting, easy to answer, and related to the topic of the survey • Be concerned about privacy issues for the respondents and the data that is collected. • Allow an alternate mode of completion if people are concerned about privacy, i.e. print and mail in the survey.

22. E-Survey Workshop : Part III 實作教學 元智大學資訊社會學研究所 黃心怡

23. 分析結果 確認研究主題 資料收集 設計問卷 發送E-mail邀請 問卷程式設計 問卷前測 開始你的Web-survey

24. 架設你的網路問卷平台

25. The Advantage of Web-Survey • Cost saving • Ease of contacting respondents • More efficient data processing • Ability for customization

26. How to designSTEP1:Introduction

27. STEP2:Layout (1)選擇問卷形式 傳統問卷格式 題組形式 每題一頁 (2)設定版面大小 較常使用的是800*600，建議在網頁中加註讓受訪者瀏覽時較好觀看

28. STEP3:Webpage Component • Frame(表單) • Radio(按鈕類) • Checkbox(勾選類) • Drop-Down(下拉式) • Text(文字欄) • Hyperlink(超連結)

29. STEP4:Save Result 文字檔 • 簡單HTML語法,藉由網頁製作軟體如: FrontPage、Dreamwaver等，受訪者在完成線上問卷後，可直接回傳結果至研究者的網路伺服器，以文字檔儲存 資料庫 • 當調查規模龐大且題目眾多時，資料庫有著良好的儲存與管理的功能；相對的，與資料庫連結的網路問卷也需要較複雜的程式技巧與工具，其與法如ASP、PHP、JAVA。

30. STEP5:Analyze Result • 網路問卷降低了過去人工coding的時間與金錢 • 線上調查的電子檔容易讓統計軟體讀取 (通常.txt, .csv為最普遍的可讀取格式)

31. STEP BY STEP: Form (表單)

32. STEP BY STEP : Form

33. STEP BY STEP : Pixels(瀏覽像素)

34. STEP BY STEP : Pixels

35. STEP BY STEP : Table(表格)

36. STEP BY STEP : Table

37. STEP BY STEP : Table

38. STEP BY STEP : Radio Button

39. STEP BY STEP : Radio Button

40. STEP BY STEP : Radio Button 名稱 = Q1 是 = 1 否 = 0 不選擇預設值

41. STEP BY STEP : Radio Button

42. STEP BY STEP : Check Box

43. STEP BY STEP : Check Box 名稱 = Q2_1 有勾選 = 1 不選擇預設值

44. STEP BY STEP : Check Box

45. STEP BY STEP : Dropdown

46. STEP BY STEP : Dropdown

47. STEP BY STEP : Dropdown 名稱 = Q3 新增選項 =台北市 勾選指定值=01 起始狀態=未選取

48. STEP BY STEP : Dropdown

49. STEP BY STEP : Dropdown 驗證功能: 受訪者送出問卷後，檢查其填寫狀態，避免漏答

50. STEP BY STEP : Text Box