Media computation as a context for learning computing

1. Media computation as a context for learning computing Mark Guzdial College of Computing/GVU Georgia Institute of Technology

2. Story • Perlis’ challenge: Computer science is more important than calculus • Evidence that we’re not there yet • And what the solution might look like • Our attempt: Introduction to Media Computation • How this might be used/useful elsewhere

3. Computer science is more important than Calculus • In 1961, Alan Perlis argued that computer science is more important in a liberal education than calculus • Explicitly, he argued that all students should learn to program. • Calculus is about rates, and that’s important to many. • Computer science is about process, which is important to everyone

4. How close are we to being able to teach everyone CS? • Not very • CS1 is one of the most despised courses for non-majors • At many departments, CS retention rates are lower than the rest of campus • At Georgia Tech: 65% for 1995 cohort, vs. 73% for Engineeering • Drop-out rates near 50% at many institutions • Female enrollment in CS has been dropping nationally

5. Why? • Several recent studies and books claim that CS instruction tends to dissuade anyone but white males • “Tedious,” “taught without application relevance,”“boring,” “lacking creativity,” “asocial”

6. The potential of computing • How can we realize the potential impact of computing if future professionals in other disciplines despise computer science? • To realize that potential, we need practitioners in other disciplines to understand computing. • At least some of those practitioners need to understand it as a creative, exciting venture—as we do!

7. The best uses for computing technologies will come from others • Thomas Edison vs. D.W. Griffith • If we want computing technologies to become useful, they have to get out of our hands. • It can’t be just through applications • That presumes that the current technologists know how everyone else wants to do things and should do things • Suggestion: D.W. Griffith knew things that Edison didn’t.

8. The Challenges • We need to motivate CS, potential CS, and non-CS students to care about computing • We need to make it social, creative, relevant, exciting, and not tedious • Which is how many of us already see Computing, but that’s not getting communicated

9. Intro Computing at Georgia Tech • CS1321 “Introduction to Computing” • Required of every student at Georgia Tech • Uses Scheme and How to Design Programs • Followed up by CS1322 “Introduction to Object-Oriented Programming” in Java • COE 1361 “Computing for Engineers” • Engineering problem-solving in MATLAB • Still in pilot stage: 75-90 students a term • 800 new Engineering Freshmen a year enter Tech

10. Our Attempt: Introduction to Media Computation • A course for non-CS and non-Engineering majors • International Affairs, Literature, Public Policy, Architecture, Management, Biology, etc. • 120 students this semester,planning 400-600 in the Fall • 2/3 female in this semester’s CS1315 • Focus: Learning programming within the context of media manipulation and creation

11. Motivating the Computing • As professionals, these students will often the use the computer as a communications medium. • All media are going digital,and digital media are manipulated with software. • Knowing how to program, then, is a communications skill.

12. Programming as a Communications Skill • Knowing how to program means to understand one’s tools. • Maybe means can transfer tool skills more easily • Students tell us that they’re excited to learn how PhotoShop works. • And it means that, if you have to, you may be able to grow your own

13. Programming as Communicating Process • A program is a succinct, executable process description • That makes valuable for explaining process • We use examples from Biology (e.g., explaining DNA transcription) and Management (e.g., simulations) to make this point

14. Python as the programming language • Huge and contentious issue • Use in commercial contexts legitimatizes the choice • Industrial Light & Magic, Google, Nextel, etc. • Minimal syntax • Looks like other programming languages • Potential for knowledge transfer • Actually using Jython (http://www.jython.org) for Java class libraries

15. How the class was developed • Created in response to “recent unpleasantness” • On-line surveys, meetings with students • Developed with an advisory board with representatives from across campus • Architecture, Math, Electrical and Computer Engineering, Literature, Center for Enhancement of Teaching and Learning • Trialed in faculty workshop in mid-December

16. Course Objectives • Students will be able to read, understand, and modify programs that achieve useful communication tasks • Not programming from a blank piece of paper • Students will learn what computer science is about, especially data representations, algorithms, encodings, forms of programming. • Students will learn useful computing skills, including graphing and database concepts

17. Data structures • Some data structure discussions come naturally • Sounds are in arrays • Pictures are matrices of pixels, each of which has red, green, and blue components • Files are stored in trees • Much of our discussion of data structures comes in the form of encodings comparisons • RGB vs. CMYK for encoding color • 22.1 Khz vs. 44.2 Khz for audio sampling rates

18. def clearRed(picture): for pixel in getPixels(picture): setRed(pixel,0) def greyscale(picture): for p in getPixels(picture): redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p, makeColor(luminance,luminance,luminance)) def negative(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255-blue) setColor(px,negColor)

19. def chromakey(source,bg): for x in range(1,getWidth(source)): for y in range(1,getHeight(source)): p = getPixel(source,x,y) # My definition of blue: If the redness + greenness < blueness if (getRed(p) + getGreen(p) < getBlue(p)): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,x,y))) return source

20. Use a loop!Our first picture recipe original def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5) Used like this: >>> file="/Users/guzdial/mediasources/barbara.jpg" >>> picture=makePicture(file) >>> show(picture) >>> decreaseRed(picture) >>> repaint(picture)

21. Recipe to Increase the Volume def increaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 2) Using it: >>> f="/Users/guzdial/mediasources/gettysburg10.wav" >>> s=makeSound(f) >>> increaseVolume(s) >>> play(s) >>> writeSoundTo(s,"/Users/guzdial/mediasources/louder-g10.wav")

22. A Sunset-generating function • How do we turn this beach scene into a sunset? • What happens at sunset? • Tried increasing the red, but that failed. • New Theory: As the sun sets, less blue and green is visible, which makes things look more red.

23. A Sunset-generation Function def makeSunset(picture): for p in getPixels(picture): value=getBlue(p) setBlue(p,value*0.7) value=getGreen(p) setGreen(p,value*0.7)

24. Introducing IF:Making Barb a redhead def turnRed(): brown = makeColor(57,16,8) file = r"C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\barbara.jpg" picture=makePicture(file) for px in getPixels(picture): color = getColor(px) if distance(color,brown)<50.0: redness=getRed(px)*1.5 setRed(px,redness) show(picture) return(picture) Original:

25. Generalizing Algorithms • We talk about algorithm complexity later in the course, after the media is done. • We talk about different approaches to the same problem, where the criteria might be aesthetics or correctness, instead of speed or size • For example, generating greyscale • During the media, we point out similar themes in different functions. • We refer to them as “sub-recipes”

26. Scaling the picture down def copyBarbsFaceSmaller(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)/2)): sourceY = 25 for targetY in range(100,100+((200-25)/2)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 2 sourceX = sourceX + 2 show(barb) show(canvas) return canvas

27. Scaling the picture up def copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor(getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 0.5 sourceX = sourceX + 0.5 show(barb) show(canvas) return canvas

28. Recipe for halving the frequency of a sound This is how a sampling synthesizer works! def half(filename): source = makeSound(filename) target = makeSound(filename) sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex, getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex + 0.5 play(target) return target Here’s the piece that does it

29. Compare these two def copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor( getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 0.5 sourceX = sourceX + 0.5 show(barb) show(canvas) return canvas def half(filename): source = makeSound(filename) target = makeSound(filename) sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex, getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex + 0.5 play(target) return target

30. Both of them are sampling • Both of them have three parts: • A start where objects are set up • A loop where samples or pixels are copied from one place to another • To decrease the frequency or the size, we take each sample/pixel twice • In both cases, we do that by incrementing the index by 0.5 and taking the integer of the index • Finishing up and returning the result

31. Using your personal pictures

32. And messin’ with them

33. Data-first Computing • Real users come to a user with data that they care about, then they (unwillingly) learn the computer to manipulate their data as they need. • CS1315 works the same. • We use pictures of students in class demonstrations. • Students do use their own pictures as starting points for manipulations. • They started doing this in the second week • How often do students use their second week of CS1 on their own data? • How does that change the students’ relationship to the material?

34. Rough overview of Syllabus • Defining and executing functions • Pictures • Psychophysics, data structures, defining functions, for loops, if conditionals • Sounds • Psychophysics, data structures, defining functions, for loops, if conditionals • Text • Converting between media, generating HTML, “flattening” media and saving to a database • Movies • Then, Computer Science

35. Computer science as a solution to their problems • “Writing programs is hard! Are there ways to make it easier? Or at least shorter?” • Object-oriented programming • Functional programming and recursion • “Movie-manipulating programs take a long time to execute. Why?” • Algorithmic complexity • “Why is PhotoShop so much faster?” • Compiling vs. interpreting • Machine language and how the computer works

36. Assignments encourage collaboration • Homework are all collaborative • Quizzes are preceded by nearly-identical, collaborative pre-quizzes • Two “take-home exams” (programming assignments) are non-collaborative • “Lablets” on application software are collaborative

37. Assignments encourage creativity • For several homeworks, the task is to manipulate media in some way, but we don’t care what media • For example, creating a collage or building an animation • Encouraging homework results to be posted to CoWeb in galleries • We’ve purchased webcams to loan to students to create their own media

38. First Homework assignment (Due last Friday) Homework 1: Write a program named hw1 to accept a picture as input, and change its pixels as follows: • Set the green component to 125% of its current value • Decrease the blue by 25% • Decrease the red by 75%

39. Solutions shared in the CoWeb

40. Grade distribution Much better than anticipated.

41. Tools to support media computation MediaTools for exploring media JES: Jython environment for students

42. Assessing the effort • We don’t know if this will work • Will students learn? • Will they learn similarly to the other CS1 classes? • Will the class change their attitude toward CS? • Hopefully, something better than what we’ve been seeing • Will the differences appear among the women, specifically? • If there are problems in student performance, where are they?

43. Assessment measures • Comparing all of our CS1 classes in terms of learning and motivation, broken out by gender and major • Surveys: Initial, midterm, and final • Observational study of student performance to understand problems and strategies • Watched two groups working on HW1 • Interview study of impact on women

44. How can this be useful to you? • We’d like to find partners in developing and trialing the course as a non-majors programming course. • We have software, draft of book, and slides in development. • We’d like to develop something that works more than just at Tech. • We have Java classes to offer to majors courses that implement most of the media interface. • Media focus may be motivating to our majors, and may help with retention issues.

45. public greyScale(String filename) throws java.io.IOException // parms: string defining the file path { JavaPicture src = new JavaPicture(); src.setFilename(filename); src.loadImage(); // System.out.println("Height = " + src.getHeight() ); // System.out.println("Width = " + src.getWidth() ); for (int x=1; x < src.getHeight();x++) { for (int y=1; y < src.getWidth();y++) { JavaPixel p = src.getPixel(y,x); int nC = (p.getRed()+p.getGreen()+p.getBlue() )/ 3; p.setBlue(nC); p.setRed(nC); p.setGreen(nC); } } src.saveImage("C:\\grey.jpg"); }

46. Summary • Perlis’ challenge suggests that CS is more important than Calculus • But need to update our pedagogy to make it happen • Media Computation may be a useful context to motivate student performance • Our class is aimed at addressing the challenges we’ve identified, and we’re trying it this semester • The approach right now is aimed at non-majors, but certainly could be used with majors, too.

47. Acknowledgements • Course materials development: Jason Ergle, Claire Bailey, David Raines, Joshua Sklare, Adam Wilson, Andrea Forte, Mark Richman, Matt Wallace, Alisa Bandlow. • Assessment: Andrea Forte, Rachel Fithian, Lauren Rich • Thanks to Bob McMath and the Al West Fund, to GVU and CoC, and the National Science Foundation

48. For further information • Course CoWeb: http://coweb.cc.gatech.edu/cs1315 • Where we planned the course:http://coweb.cc.gatech.edu/mediaComp-plan • guzdial@cc.gatech.edu