Internet Performance Measurements using Firefox Extensions Scot L. DeDeo Professor Craig Wills

1. Internet Performance Measurements using Firefox Extensions Scot L. DeDeo Professor Craig Wills

2. Motivation • There exists an API within the Firefox web browser that allows for interacting with its downloading and processing of websites • Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?

3. Outline • Introduction to Firefox Extensions • Description • Capabilities • Page Stats Extension • Overview • Approach and paths taken • Demo • Summary

4. Introduction • Firefox Extensions are add-ons that integrate directly into the Firefox Web browser. • Function through open source API that allows for manipulating both the content being received by the web browser as well as the graphical outlook.

5. Capabilities • Graphical add-ons (Use the XML User Interface Language – XUL) • Allows for changing the existing Firefox overlay, adding new buttons, menus, and windows. • E.g. Google toolbar allows for a toolbar to be inserted directly into Firefox. • Middleware modifications (Cross Platform Component Object Module – XPCOM Interfaces - Accessible through JavaScript or C++) • Allows for changing networking properties, modifying / blocking data requests / responses, etc. • E.g. Fasterfox allows you to tweak many network and rendering settings such as simultaneous connections, pipelining, cache, DNS cache, and initial paint delay. • E.g. AdBlock and AdBlock plus allow for use of a pre-formed list or to customize a list of sites to block downloads from.

6. Capabilities • Combination of Graphical add-ons and Middleware modifications can yield new applications that exist within Firefox. • E.g. Gmail Space – Allows users to take advantage of Gmail’s 2 GB quota by providing a graphical interface to easily transfer files. • Fixing bugs in other sites • Allows for changing sites html, style sheets, and inserting JavaScript. • E.g. Style sheets can be modified to change the background color and text of a site (change to black on white instead of yellow on white).

7. Website Request / Response Breakdown Spawns Container Page 1. Location Changes – Request for container (main) page goes out. 2. Response for container (main) page is received and processed. 3. Additional requests are spawned in response to the processed page: CSS, JavaScript, images, etc. 4. Responses come in and are processed and may / may not spawn additional requests. JavaScript CSS Spawns Jpeg JavaScript Browser location set to: http://www.scotd.com/index.php Spawns http://www.scotd.com/index.php?id=all http://www.scotd.com/scot.css http://www.scotd.com/background/valley2k.jpg

8. Outline • Introduction to Firefox Extensions • Description • Capabilities • Page Stats Extension • Overview • Approach and paths taken • Demo • Summary

9. Page Stats Extension • Measures the HTTP requests associated with the load of a webpage. • Page Measurements Include: # of Request Request submit time (msec) Finish Time (msec) HTTP Response Code Size (bytes) content type URL • Page Summary Includes: URL total page size (bytes) total time (msec) total # of requests # dead requests • Capabilities: • Control input and output with a graphical interface • Output to a log file • Run in a batch mode and visit a list of sites

10. Page Stats Extension • Usage: 1.Graphical Interface found under the tools menu in Firefox - “Site Measurements” • Ability to enable / disable measurement listeners, log file, and select a list of sites to visit. • Also allows for saving and clearing the text on the measurement window. • Provides a summary of all sites visited.

11. Page Stats Extension • Usage: 2. Batch mode – Invoked by a configuration file specified on the command line (firefox pagestatsconfig=“/home/sdedeo/config.txt”). Parameters: measurement: on log: /home/sdedeo/log.txt sites: /home/sdedeo/sites.txt exitOnFinish: true requestTimeout: 8000 pageTimeout: 45000 • Allows for easily measuring pages in bulk • Pages are still fully rendered

12. Approach and paths taken • Began in January 2006 based on work completed by Paul Timmins • Included initial work of setting up an extension with ideas on how to make measurements • Initial version based on an XPCOM component called nsiWebProgress which reports to the browser: • State, Progress, Location, Status, and Security changes

13. Approach and paths taken • First Approach Issues • nsiWebProgress progress notification only occurs on text based files (E.g. html, CSS, JavaScript). Images and other static files are omitted. • No way to measure the time the request is submitted • Occasionally the progress notification gives an unknown length of a file • If a site was visited more then once, caching effects skew measurements • Second Approach – nsiHTTPChannel • Intercepts requests browser sends – “http-on-modify-request” • Intercepts responses before processing – “http-on-examine-response” • Clears system cache between sites in order to maintain consistent measurements

14. Approach and paths taken • Second Approach (Continued) • nsiHTTPChannel provides no state notifications • Solved using nsiWebProgress state change to detect state stopped / finished • Probe to insure all requests that have gone out have come back (nsiWebProgress state stop returned before all images received) • Timeout Mechanism • Two Timeouts: • Max time between receiving responses – defaults to 8 seconds • Max time for a page – defaults to 45 seconds

15. Approach and paths taken • Sites with Issues: • http://www.cnn.com routinely sends requests for images in which no response comes. These are JavaScript invoked images on mouseovers • This problem is solved by the request timeout and the requests are labeled as ‘dead’ • http://my.netscape.com uses a JavaScript based redirect: ‘document.location = http://my.netscape.com/index2.psp’ which when used in batch mode was causing the extension to move onto the next site and process two sites simultaneously • This was solved by giving a one milliseconds delay before switching sites after a site’s results were processed • http://www.dallasnews.com sends an image banner which receives an update once every two seconds • This was solved with the max site timeout

16. Demo Launch Firefox

17. Outline • Introduction to Firefox Extensions • Description • Capabilities • Page Stats Extension • Overview • Approach and paths taken • Demo • Summary

18. Summary • Firefox extensions are a powerful tool that allow for: • Measuring and manipulating how a page is downloaded and processed • Adding additional features, both graphical and background • Building Extensions: • Firefox 1.5 enables much easier testing of extensions • XPCOM API is still in the beta stages and certain libraries must be used with caution since they change • Documentation available is still minimal, but the development community is growing at a fast pace

19. Summary • Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions? • The pagestats extension shows one way we can create quantitative measurements for a website • Pagestats can be used in conjunction with other extensions, such as adblock and fasterfox, to compute a measurement of effectiveness

20. Questions / Comments Scot L. DeDeo Professor Craig Wills Page Stats Extension: http://www.wpi.edu/~sdedeo/pagestats Related Sites: https://addons.mozilla.org/firefox/extensions/ http://www.xulplanet.com (XUL and XPCOM API Reference)