1 / 33

Conceptual and Operational Issues in the Measurement of Internet Use *

Conceptual and Operational Issues in the Measurement of Internet Use *. Jonathan Zhu City University of Hong Kong enjhzhu@cityu.edu.hk * Funded by the UGC of HKSAR (CityU1152/00H). @. Background: the Diffusion of the Internet in Hong Kong, Beijing and Guangzhou. Source: J. H. Zhu (2003).

anila
Download Presentation

Conceptual and Operational Issues in the Measurement of Internet Use *

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Conceptual and Operational Issues in the Measurement of Internet Use* Jonathan Zhu City University of Hong Kong enjhzhu@cityu.edu.hk * Funded by the UGC of HKSAR (CityU1152/00H) @

  2. Background: the Diffusion of the Internet in Hong Kong, Beijing and Guangzhou Source: J. H. Zhu (2003)

  3. Internet Penetration Rate in East Asia

  4. Wired Internet Use vs. Wireless Internet Use

  5. Diffusion of Cable TV, the Internet, and Mobile Phone in Hong Kong

  6. Internet vs. Mobile Phone in Beijing and Guangzhou

  7. Issues in Measurement of Internet Use and Users • The size of “Internet users” in a society is a function of: • Definition of study population (SP) • Method of sample weighting (SW) • Requirement of minimal usage (MU) • The amount of “online time” by Internet users is a function of: • Definition of study population (SP) • Method of sampling weighting (SW) • Method of data collection (DC) • Treatment of extreme values (EV)

  8. Criteria for Evaluation of Measurement • Validity: how accurate or correct is the measure as compared with the “truth”? • Reliability: how precise or stable is the measure over time and/or across space? • Practicality: how efficient or economic is the measure in data collection and analysis?

  9. Data • Hong Kong Survey 2002: telephone interviews of 1,800 residents at 6 and above in Dec. 2002 by Jonathan Zhu and his team • AC Nielsen/Netratings 2002-03: online tracking of 1,500 Internet users from 811 households in Hong Kong in Oct. 2002 and Jan. 2003.

  10. Definitions of Study Population • WIP-Hong Kong: 18-74 • CNNIC: 6+ • Another popular definition: 18+ • HK Census 2002: • 6-17: 16.4% • 18-74: 80.0% • 75+: 3.6%

  11. Impact of Population Definitions on Internet User Size Data: Hong Kong 2002

  12. Requirements of Minimal Usage

  13. Impact of Minimal Requirements on Internet User Size Data: Hong Kong 2002

  14. Age Distribution of the Sample before and after Weighting Data: Hong Kong 2002

  15. Impact of Weighting Methods on Internet User Size Data: Hong Kong 2002

  16. Summary: Internet Users by Population, Usage Requirement & Weighting Method Data: Hong Kong 2002

  17. A Mathematical Model of “True” Internet Users (TIU) TIU = 55.3 – 1.4SP18-74 - 3.7SP18+ - 4.5MU – 5.4SW (Adjusted R2 = 99.6%, Standard Error = 0.3%) Where TIU is the “Unadjusted” Internet Users (%) for HK in 2002, which should be 1.4% less for a study population of 18-74, or 3.7% less for a study population of 18+, or 4.5% less if those use the Internet less than 1 hour per week are excluded, or 5.4% less if the sample is weighted based on population census.

  18. Impact of Population Definitions on Online Time (at Home) Data: Hong Kong 2002

  19. Impact of Weighting Methods on Online Time (at Home) Data: Hong Kong 2002

  20. Impact of Extreme Values on Online Time (at Home) Data: Hong Kong 2002

  21. Impact of Data Collection (DC) Methods on Online Time Data: HKS 2002 & Netratings 2002-03

  22. Summary: Online Time by SP, SW, DC, and EV Data: Hong Kong 2002

  23. A Mathematical Model of “True” Online Time (TOT) TOT = 532 + 16SP18-74 – 22SW – 49EV - 249DC (Adjusted R2 = 93.5%, Standard Error = 34.3) Where TOT is the “Unadjusted” Online Time (min.) for HK users in 2002, which should be 16 min. more for a study population of 18-74, 22 min. less if the user sample is weighted, 49 min. less if extreme values are removed, or 249 min. less if data are collected through online tracking method.

  24. Telephone interview data include: Online time at both home (68%) and elsewhere (32%); Non-HTTP based activities such as using POP3 Email (=136 min./week) and other protocols; Online tracking data include: Online time only at home; Only HTTP=based activities protocols). Caution: Different Definitions of “Online” Activities It is estimated that tracking data may measure only 51% of the total online time..

  25. Estimated Distribution of Online Time by Location and Protocol of Usage

  26. Conclusion: How Many Internet Users Are There? • The size of “Internet Users” is significantly affected by the definition of study population (SP), the requirement of minimal usage (MU) and the method of sample weighting (SW). • SP (e.g., general population vs. adults) may produce a difference of 1-4% and MU (e.g., no requirement vs. 1 hour per week) up to 5%. While there is no “correct” definition of SP or MU, it is important to report the definition and adopt, whenever possible, multiple definitions. • SW (weighted vs. unweighted) may contribute another 5% difference. Since Internet use is highly correlated with age and sex, it seems both necessary and effective to weight the sample to ensure the accuracy of the measurement.

  27. Conclusion: How Much Time Do They Spend Online? • The amount of online time is marginally affected by SP (p = 0.3) and SW (p = 0.2) probably due to the fact the base of analysis is already restricted to users. • Online time is significantly affected by the treatment of extreme values (EV), which may inflate online time by up to 10%. It is thus necessary to control for it (i.e., removing EVs). • Online time is most significantly affected by the method of data collection (DC, e.g., interviews vs. online tracking), which may result in a difference of 2-folder. Although online tracking is generally more accurate, it is far more expensive and impractical in many societies. It is thus important to keep in mind the magnitude of inflation in self-reported data.

  28. Ultimate Criteria for Evaluation • Validity: how accurate or correct is the measure as compared with the “truth”? • Reliability: how precise or stable is the measure over time and/or across space? • Practicality: how efficient or economic is the measure in data collection and analysis?

  29. Consistency in Measurement of Internet Users over Time and across Space* * Based onWIP definition.

  30. Stability in Measurement of Sex Ratio among Internet Users in Hong Kong

  31. Stability in Measurement of Online Locations in Hong Kong

  32. Consistency in Difference between Methods across Age Cohorts

  33. Final Verdicts • Measurement of Internet users and online time based on interviews data is largely reliable over time and across space. • The interview-based measurement is generally more practical than online tracking method. • The interview-based measurement is generally weaker in validity, as compared to online tracking method. However, it could be adjusted if the departure from the “truth” is known (e.g., based on comparison with online tracking data.

More Related