A Qualitative Study of Animation Programming in the Wild

1. A Qualitative Study of Animation Programming in the Wild Aniket Dahotre, Yan Zhang, Christopher Scaffidi ESEM 2010

2. Roles for animation programming • A vehicle for getting kids excited about programming • An environment for teaching programming skills • A medium for communicating and entertaining • A platform for research aimed at raising usability of programming tools • Examples of animation programming tools: • Logo, KidSim, AgentSheets, Alice, Hands, Scratch Intro Technical  Social  Remixing  Closing

3. Scratch as a particular programming tool • Turing-complete language • Events, loops, conditionals, sprites, sound… • Drag-and-drop programming + Online community for sharing, trying, discussing & remixing animation projects Intro Technical  Social  Remixing  Closing

4. Scratch online repository Intro Technical  Social  Remixing  Closing

5. Statements and anecdotesfrom related work • Scratch site is “the YouTube of interactive media”. • Users are “a new generation of creative, systematic thinkers comfortable using programming to express ... ideas” • Supports “creative appropriation… the utilization of someone else’s creative work in the making of a new one” • “The site’s collection of projects is wildly diverse, including video games, interactive newsletters, science simulations, virtual tours, birthday cards, animated dance contests, and interactive tutorials, all programmed in Scratch” • One project obtained over 100 user comments. • Several programmers formed collaborative partnerships. • One particular Tetris game was remixed dozens of times. Intro Technical  Social  Remixing  Closing

6. Questions for our study • To what extent is Scratch succeeding as a basis for developing programming skills in the wild? • Technical programming skills • Social skills • Remixing/reuse skills Intro Technical  Social  Remixing  Closing

7. Research methods • Screen-scraped 100 randomly-selected animations • Including their code, usage statistics, and user comments • Developed coding schemes for several research questions • One student examined subset of animations, proposed coding scheme • This student and another independently applied scheme to half • Negotiated modifications to coding scheme • Then checked each other’s work on the other half of the animations • < 10% disagreement; negotiated to resolve all disagreements • Coalesce codes to increase clarity of presentation as needed • In a few cases, related work provided relevant coding schemes • Which we applied without modification • A few research questions could be answered directly with quantitative (non-coded) data Intro Technical  Social  Remixing  Closing

8. Functional roles of animations To teach skillor knowledge To entertainingly challenge To communicate fictional plot 56%: No clear functional role Intro  Technical Social  Remixing  Closing

9. Use of programming constructs:repository vs. afterschool “Clubhouse” Intro  Technical Social  Remixing  Closing

10. Apparent design patterns AbstractHandler onHit(type1) onHit(type2) onHit(type3) Controller methodWithLoop() Sprite CollisionHandler onHit(type2) CollisionHandler onHit(type1) onHit(type2) Sprite Sprite onMessage() Sprite of type 1 Sprite of type 2 Intro  Technical Social  Remixing  Closing

11. Conclusion regardingtechnical skill development • Relatively successful platform for development of technical programming skills • Comparable primitive use relative to Clubhouse users • Some demonstration of patterns (perhaps subconscious) • Comparable in complexity to spreadsheets and other programs created by end-user programmers (see paper for details) Intro  Technical Social  Remixing  Closing

12. Kinds of user comments exchanged Approval, no suggestion Other commentsfrom users, otherthan projectcreator Comments fromproject creator Approval + idea Disapproval Intro  Technical  Social Remixing  Closing

13. Active collaboration on projects • By “collaboration,” we mean • joint design or implementation • by a team of multiple people • consciously working on a common intellectual goal • Reviewed user comments • E.g., to find comments like “I used your suggestion – thanks!” • E.g., or like “Thanks, my friend helped me with that.” • Also examined source code to look for any action on suggestions • Looked for comments revealing code edits by multiple people • We found no such indications of any collaboration at all. Intro  Technical  Social Remixing  Closing

14. Conclusion regardingsocial skill development • Uneven success as platform for development of social programming skills • Half of projects received comments • Most comments were complimentary and offered suggestions • But unable to find evidence of actual collaboration Intro  Technical  Social Remixing  Closing

15. Frequency of downloading and remixing • Related work: 15% of projects were created by remixing • But what fraction of projects are used to create remixes? • Of our 100 projects… • 50% were downloaded at least once • 10% were remixed at least once • 5% were remixed at least once by other users Intro  Technical  Social  Remixing Closing

16. Changes made to programs during remixing • 20 of our 100 projects were created by remixing. Of these… Intro  Technical  Social  Remixing Closing

17. Conclusion regardingremixing skill development • Uneven success as platform for development of remixing skills • Most remixes simply involved multimedia tweaks • Few attempts at script modification during remixing • Nearly half of script modifications led to major bugs • Even the biggest modifications were still fairly small (see paper) • Frequency of remixing unimpressive vs other systems (see paper) Intro  Technical  Social  Remixing Closing

18. Results and opportunities for future work • Technical skill development: relatively successful • Perhaps still a need for helping animators to do higher-level design • Social skill development: uneven success • Definitely a need for helping animators to collaborate • Remixing skill development: uneven success • Definitely a need for helping animators to remix code Intro  Technical  Social  Remixing  Closing

19. Additional empirical questions for future work • Do animators consciously understand design patterns? • What usually happens after code downloads? • Is some hard-to-detect collaboration somehow occurring? • What kinds of interaction happen in the online forums (outside the context of particular projects)? • How well do these programming skills transfer to other programming languages and tools? Intro  Technical  Social  Remixing  Closing

20. Thank You… • For the opportunity to present. • For your questions, thoughts, and constructive feedback. Intro  Technical  Social  Remixing  Closing

21. Frequency of interactions between users Intro  Technical  Social Remixing  Closing

22. Size of repository animationsby animations’ functional role