Enhancing the Web With End-User Programming Tak Yeon Lee, Ben Bederson

1. Enhancing the Web With End-User Programming Tak Yeon Lee, Ben Bederson

2. Outline • Problem • Inefficiencies on the Web • Challenge • Building extension is very hard • Our goal • Expressive and Easy Programming Env. • - Improves a wide range of tasks • - No programming skills required

3. Snapshot ANY WEBSITE

4. Inefficiencies on the Web 1. Missing relevant information “I want to see the pictures.”  Detail page “Is it a good value?”  Kelly’s Blue book “Is the model reliable?”  Auto forums “How far is this place?”  Google Map

5. 2. Removing unwanted information AD AD

6. 2. Removing unwanted information

7. 3. Repetitive Operations “I want to download all PDFs named as [first author]-[title]-[year].pdf ” 10.1.1.154.444.pdf Howe-TheRiseOfCrowdsourcing-2006.pdf Kittur-08-001.pdf Kittur-CrowdsourcingUserStudies-2008.pdf 60947(1).pdf Brabham-CrowdsourcingAsAModelFor-2008.pdf Alonso2008.pdf Alonso-CrowdsourcingForRelevanceEvaluation-2008.pdf

8. 4. Finding a needle in a haystack Grading Procedure 1. Finding mistakes 2. Add comments Takes 4-8 hours to grade 70 students 3. Calculate the total points 4. Submit to grade server

9. Summary of Potential Mistakes Grading with an extension MyDouble.zero 2. Click to add predefined comment 1. Click to scroll to the code line of the mistake “Previously it took several hours, but now I can finish in 40 minutes.” - Lecturer “Three times faster than before, at least!” -Grading TA MyDouble.zero 3. Setting partial credit MyDouble.zero 4. Report total score to another server

10. Browser Extensions add new features and personalize experience. Over 50,000 on Chrome Web Store Print All E-mails in a folder AdBlock >15 million users Photo Zoom >2 million users • Building an extension requires • Programming knowledge(HTML, CSS, and Ajax) • Time and effort of development • The grading example took >40 hours

11. OUR GOAL IS TO BUILD Expressiveand Easy programming env. • The toolkit should support tasks like... • Automating repetitive jobs • Integrating data from multiple sources • Custom filtering • Information Summary We defined a domain language model having Pick, Change,Add operations as generic functionalities of the extensions.

12. Pick ChangeAdd model Pick extract information from web pages or elements. <A> tags The rise of crowdsourcing.pdf http://sistemas-humano.com... Crowdsourcing user studies... http://dl.acm.org/citation.cfm?id=1357127 Crowdsourcing as a model ... http://con.sagepub.com/... URLs

13. Pick ChangeAdd model Change manipulates dataset. [b] [a] • Arithmetic operations • +, -, /, %, ... • sum, avg, count, ... • String manipulation • concatenate, split, find, replace, ... • Filter • Sort [a] * [b]

14. Pick ChangeAdd model Change manipulates dataset. [b] [a] • Arithmetic operations • +, -, /, %, ... • sum, avg, count, ... • String manipulation • join, split, find, replace, ... • Filter • Sort Join [a] with [b]

15. Pick ChangeAdd model Change manipulates dataset. [b] [a] • Arithmetic operations • +, -, /, %, ... • sum, avg, count, ... • String manipulation • join, split, find, replace, ... • Filter • Sort Filter [a] having [b]

16. Pick ChangeAdd model Change manipulates dataset. [a] • Arithmetic operations • +, -, /, %, ... • sum, avg, count, ... • String manipulation • join, split, find, replace, ... • Filter • Sort Sort [a]

17. Pick ChangeAdd model Add modify the original web pages. • Create new elements • image, text, button, ... • Modify element’s style • Show / Hide, changing color, fonts, ... • Triggering events • click, ... • Type text

18. Pick ChangeAdd model e.g. Removing AD banners [1]. Pick ad banner elements [2]. Hide[1] AD AD

19. Pick ChangeAdd model e.g. Downloading files with custom name pattern [1]. Pick title text The rise of crowdsourcing.pdf [2]. Merge [1] with name pattern “.pdf” Crowdsourcing user studies with Mechanical Turk.pdf [3]. Pick download links Crowdsourcing as a model for.pdf [4]. Add [2] to ‘download’ attribute of [3] Crowdsourcing for relevance evaluation.pdf

20. OUR GOAL IS TO BUILD Expressive and Easy programming env. • Requires no programming knowledge • Programming-by-Example techniques • Telling computer what I want, without worrying how to do.

21. Programming-by-Example technique Traditional Programming Programming-by-Example ? 1, 2, 3 INPUT int sum=0; for(int v in input) { sum=sum+v; } return sum; ! PROGRAM ? 6 OUTPUT

22. Programming-by-Example technique Programs with multiple inputs e.g. Adding suffix to strings [1] [2] Input Merge [1] with [2] Output

23. Programming-by-Example technique Multi-step programming e.g. Counting items of [1] that exist in [2] [1] [2] Filter [1] exist in [2] [3] Count [3]

24. Programming-by-Example technique Multi-step programming e.g. Counting items of [1] that exist in [2] [1] [2] Filter [1] exist in [2] [3] Count [3]

25. Demo time

26. Conclusion • A new opportunity for end-users to personalise the web by creating their own extensions. • Programming-by-Example techniques • with Pick, Change, Add model • Future work includes • Usability Tune up • Evaluation of the system’s learnability • Please email us if you have an idea for extensions • Tak Yeon Lee <tylee@umd.edu>