A Media Computation Cookbook

1. A Media Computation Cookbook Manipulating Images and Sounds for Use in Alice Part 1: Image Manipulations Part 2: Advanced Image Manipulations, e.g., changing colors in an area Part 3: Chromakey for digital video effects Part 4: Manipulated Alice images Part 5: Sound manipulations Part 6: Manipulated sounds in Alice

2. Questions being answered here today • How do we make it easier to build new things? • How do we make it easy to write a program to access specific pictures? • How do we change the size of a picture? • How do grab part of a picture? • How do you combine parts of pictures? • How do we use programming to replace re-typing code? • How do we replace the background of a picture with something else?

3. How do we make it easier to build new things? • Don’t start from scratch! • Use the old file and add more to it. • When you start JES, open up the file that you used last time. • If you end your file name with “.py” you can figure out which are Python files. • Type more code into that file. • If you want, you can download my myfunctions.py from http://coweb.cc.gatech.edu/mediaComp-teach/43

4. How do we make it easy to write a program to access certain pictures? • setMediaPath() • Type this in the Command Area (black area at bottom of JES) and find the folder where you store your pictures. • (Could be the Desktop, probably better to use a particular folder. I call mine “mediasources”.) • makePicture(getMediaPath(“IMG_0810.JPG”)) • Will make a picture from that file in the “media path” folder.

5. How do we change the size of a picture? • To copy a picture, get the color values from pixels in one picture, and set those color values in the pixels in the other picture. def copyPicture(picture): returnPicture = makeEmptyPicture(getWidth(picture),getHeight(picture)) for pixel in getPixels(picture): color = getColor(pixel) setColor(getPixel(returnPicture,getX(pixel),getY(pixel)),returnPicture) return returnPicture

6. To scale the picture • smaller = scale(picture,0.5) • How does it work? • To get it to be only ½ the size (in both horizontal and vertical) directions, we need to lose some pixels. • Easiest way: Skip a few. Every other pixel gets copied. • Can generalize this for any size scaling.

7. Code to scale def scale(picture,factor): newHeight = int(factor*getHeight(picture))+1 newWidth = int(factor*getWidth(picture))+1 returnPic = makeEmptyPicture(int(newWidth),int(newHeight)) sx = 0 for tx in range(0,newWidth): sy = 0 for ty in range(0,newHeight): if (int(sx) < getWidth(picture)) and (int(sy) < getHeight(picture)): sp = getPixel(picture,int(sx),int(sy)) tp = getPixel(returnPic,tx,ty) setColor(tp,getColor(sp)) sy = sy + (1/factor) sx = sx + (1/factor) return returnPic

8. How do we grab part of a picture? guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) • copyPartPicture(picture,startX,startY,endX,endY) • Gives you a new picture inside the box defined by upper-left corner and lower-right corner.

9. Code to grab part of a picture def copyPartPicture(picture,x1,y1,x2,y2): returnPicture = makeEmptyPicture(x2-x1,y2-y1) targetx = 0 for srcx in range(x1,x2): targety = 0 for srcy in range(y1,y2): srcpixel = getPixel(picture,srcx,srcy) targetPixel = getPixel(returnPicture,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1 return returnPicture

10. How do we combine parts of a picture? grayscale(betsy) clearRed(james) pastePicture(betsy,scale(james,0.35),0,0) • pastePicture(canvas,partial,startX,startY) • Pasting “partial” picture onto the “canvas” starting from (startX,startY)

11. Code to paste a picture def pastePicture(canvas,source,startx,starty): targetx = startx for x in range(0,getWidth(source)): targety = starty for y in range(0,getHeight(source)): srcpixel = getPixel(source,x,y) if (targetx < getWidth(canvas)) and (targety < getHeight(canvas)): targetPixel = getPixel(canvas,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1

12. How do use programming to replace re-typing code? • Building a collage of pictures: def makeACollage(): betsy = makePicture(getMediaPath("IMG_0802.JPG")) hunter = makePicture(getMediaPath("IMG_0808.JPG")) guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) grayscale(betsy) clearRed(james) pastePicture(betsy,scale(james,0.35),0,0) maxBlue(guys) pastePicture(betsy,scale(guys,0.35),450,340) negative(hunter) pastePicture(betsy,scale(hunter,0.35),0,340) writePictureTo(betsy,"C:/collage.jpg")

13. Result:

14. How do we replace the background of a picture with something else? • Take a picture with a background that is easy to test and isn’t in the foreground. • Two examples. Which one do you think will work?

15. Code to chromakey for any background color def chromakey(picture,background,color,threshold=100.0): for pixel in getPixels(picture): pixelColor = getColor(pixel) if distance(color,pixelColor) < threshold: newColor = getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

16. Trying to make the yellow background work >>> chromakey(yellowBarb, moon, makeColor(167,159,110)) >>> writePictureTo(yellowBarb, "C:/yellowBarb-th100.jpg") >>> chromakey( yellowBarb, moon, makeColor(167,159,110),50) >>> writePictureTo( yellowBarb, "C:/yellowBarb-th50.jpg") Yellow is too close to Barb’s skin tone to work.

17. Using the general chromakey form for a blue background >>> chromakey(blueJenny,moon,makeColor(36,62,95))

18. Doing a specific version of Chromakey • Coming up with a test for being “truly blue” def chromakeyBlue(picture,background): for pixel in getPixels(picture): pixelColor = getColor(pixel) if getRed(pixel)<getBlue(pixel) and getGreen(pixel)<getBlue(pixel): newColor = getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

19. Nicely chromakeyed

20. Try it! • Take your own picture and put it on a new background. • Create a collage with various pieces of the pictures we have.