Internet Resources Discovery (IRD)

1. Internet Resources Discovery (IRD) Internet/WWW Technical Background Thanks to Miki Even-Haim and Yoram Dahan T.Sharon-A.Frank

2. Measuring the Web "When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of science." - Lord Kelvin T.Sharon-A.Frank

3. Internet/WWW Statistics • Internet Size & Growth • Population Sizes • Various Activities • Web Size & Growth • Web Pages and Formats T.Sharon-A.Frank

4. The Domain Survey The Domain Survey attempts to discover every host (i.e., uniquely reachable connected computers) on the Internet by doing a complete search of the Domain Name System. The latest results gathered during late Jan 2001 are listed, together with Mark Lottor’s work in this area over many years. For more information see RFC 1296; for more data see the archive site at the Internet Software Consortium, http://www.isc.org/ds/ Beginning with the January 1998 survey, Lottor began using a new method of doing the survey to avoid the increasing blocking of DNS zone transfers. This method of querying the DNS for known IP address is explained at http://www.isc.org/ds/new-survey.html. It is not backward compatible with the old results. The old and the new data is juxtaposedin these trends graphs with dotted lines. T.Sharon-A.Frank

5. Internet Domain Survey Host Count T.Sharon-A.Frank

6. Internet Hosts 1995-2001 New survey data Adjusted old survey data

7. Internet Hosts - Overall Trend Historical Projected

8. Trends in Internet Hosts • The figure of 109 million hosts represents a significant new benchmark for the number of Internet hosts. The current annual growth rate now stands at 51%, within the 46-67 % rates seen over the past 2 years. • It shows continued strong exponential growth, with the 100 million host barrier being crossed in late 2000. If the same growth rate is sustained, the Internet would cross the 1 billion host mark in mid 2005. • The Internet is now expanding at the rate of 63 new hosts and 11 new domains per minute worldwide. T.Sharon-A.Frank

9. Total domains registered • Total domains registered worldwide: 33,293,791 • International (COM): 23,121,005 • International (EDU): 6,708 • International (GOV): 1,269 • International (NET): 4,343,150 • International(ORG): 2,671,279 • United Kingdom (CO.UK): 3,150,380 T.Sharon-A.Frank

10. Where the Internet hosts are by domain (Jan 2000) T.Sharon-A.Frank

11. Where the Internet hosts are by domain (Jan 2001) T.Sharon-A.Frank

12. Hosts: Large Three-Letter Domains

13. Trends in Domains Growth • The largest domain is COM, jumping 7.8 million hosts since January 2000 to a new high of 36.3 million hosts. That represents a current annualized growth rate of 32%. As a percentage of the entire Internet, the COM host count stayed about the same at 33.2% of all Internet hosts. • The number of hosts in the NET domain - which is heavily used by ISPs for dialup customers - remained the fastest growing of all the large domains, expanding at an annual growth rate of 45% to 30.8 million hosts. T.Sharon-A.Frank

14. Internet/WWW Statistics • Internet Size & Growth • Population Sizes • Various Activities • Web Size & Growth • Web Pages and Formats T.Sharon-A.Frank

15. Netizens T.Sharon-A.Frank

16. Internet Users around the Globe T.Sharon-A.Frank Source: http://www.geocities.com/Eureka/Enterprises/6930/enstat.html

17. Internet Users Statistics T.Sharon-A.Frank Source: http://www.geocities.com/Eureka/Enterprises/6930/enstat.html

18. Top 20 Countries in Internet Usage T.Sharon-A.Frank

19. US Internet Users by Age T.Sharon-A.Frank

20. European Internet Users by Age T.Sharon-A.Frank

21. US Internet Users by Gender T.Sharon-A.Frank

22. European Internet Users by Gender T.Sharon-A.Frank

23. Internet/WWW Statistics • InternetSize & Growth • Population Sizes • Various Activities • Web Size & Growth • Web Pages and Formats T.Sharon-A.Frank

24. Language populations Source: Global reach http://glreach.com/globstats/index.php3?goto T.Sharon-A.Frank

25. US Internet Users by Income 1998 T.Sharon-A.Frank

26. Number of surfers vs. computer holders T.Sharon-A.Frank

27. Operations done by Americans on the Internet T.Sharon-A.Frank

28. Leading Search Engines (by entries) T.Sharon-A.Frank

29. Internet/WWW Statistics • Internet Size & Growth • Population Sizes • Various Activities • Web Size & Growth • Web Pages and Formats T.Sharon-A.Frank

30. Number of Web Sites T.Sharon-A.Frank

31. Number of Web Sites • 1997: 1,570,000 • 1998: 2,851,000 • 1999: 4,882,000 • 2000: 7,399,000 • 2001: 8,745,000 T.Sharon-A.Frank

32. Number of Unique Web Sites* • 1998: 2,636,000 • 1999: 4,662,000 • 2000: 7,128,000 • 2001: 8,443,000 * If a site is located at multiple IP addresses, the site is retained in the sample only if the numerically lowest IP address is in the sample. T.Sharon-A.Frank

33. Types of Unique Web Sites Public: Offers content that is freely accessible to the general public. Private: Offers restricted access to content: for example, via fee payment or prior authorization. Provisional: Is in a transitory or unfinished state (e.g., “under construction”). T.Sharon-A.Frank

34. Growth of Unique Web Sites T.Sharon-A.Frank

35. Web Pages Statistics (1) Note: all numbers below (source data: "Accessibility of Information on the Web“) refer to publicly indexable web pages; publicly indexable web pages exclude pages that are not normally considered for indexing by web search engines, such as pages with authorization requirements (including firewalls), pages excluded from indexing using the robots exclusion standard, dynamic pages, etc; • 12/97 : At least 320 million pages; • 02/98 : 2.8 million servers on the publicly indexable web; 289 average pages per server; 800 million publicly indexable web pages; 18.7 kilobytes is the mean size of a page; 3.9 kilobytes is the median size of a page; T.Sharon-A.Frank

36. Web Pages Statistics (2) • 02/99: • 7.3 kilobytes average size of textual content per page (after removing HTML tags, comments and extra white space); • 0.98 kilobytes median size of the textual content; • 15 terabytes of pages is the amount of data on the web; • 6 terabytes is the amount of text data; • 62.8 images per web server; • 15.2 Kbytes - average image size; • 5.5 Kbytes - median image size; • 180 million images on the publicly indexable web; • 3 terabytes - total amount of image data; T.Sharon-A.Frank

37. Web Pages Statistics (3) • As of 7/5/2000, the web has roughly: • 2,170,000,000 pages; • 40,800,000,000,000 bytes of text; • 489,000,000 images; • 8,160,000,000,000 bytes of image data; • In the last 24 hours, the web added: • 4,420,000 new pages; • 82,800,000,000 new bytes of text; • 994,000 new images; • 16,600,000,000 new bytes of image data; • 49,400,000 pages changed; • 11,100,000 images changed; • Average life span of the web page: 44 days; T.Sharon-A.Frank

38. Internet/WWW Statistics • Internet Size & Growth • Population Sizes • Various Activities • Web Size & Growth • Web Pages and Formats T.Sharon-A.Frank

39. Number of Web Pages T.Sharon-A.Frank

40. Growth of the Internet in Pages T.Sharon-A.Frank

41. What is the "average page" like? T.Sharon-A.Frank

42. Embedded Image Count T.Sharon-A.Frank

43. What data formats are being used? T.Sharon-A.Frank

44. More data formats being used T.Sharon-A.Frank

45. References • Internet Domain Survey • http://www.isc.org/ds/ • Online Computer Library Center • http://wcp.oclc.org/stats/size.html • UCLA Center for Communication Policy • http://www.ccp.ucla.edu/pages/InternetStudy.asp • Network Facts • http://www.netfactual.com/ • Internet Statistics • http://www.mit.edu/people/mkgray/net/ • Domain Statistics • http://www.domainstats.com T.Sharon-A.Frank