1 / 58

CC Lab: Proce55ing with image, sound and video

CC Lab: Proce55ing with image, sound and video. By the way, apparently we should now use “Processing”, not “Proce55ing”. (Eventually we’ll go in and correct it in these PPTs.). The goals of this week module are to:.

edita
Download Presentation

CC Lab: Proce55ing with image, sound and video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CCLab:Proce55ingwith image, sound and video

  2. By the way, apparently we should now use “Processing”, not “Proce55ing”. (Eventually we’ll go in and correct it in these PPTs.)

  3. The goals of this week module are to: Learn how to do some basic image, video and sound processing with proce55ing.

  4. There’s a lot of info in todays presentation.(We may work on this over two weeks. We’ll see how it goes.)

  5. Lets start with loading animage

  6. - Images must be in the "data" directory to load correctly. • Processing currently works with GIF, JPEG, and Targa images. • So, in order to put the images in the “Data” directory, go to Add File… from the Sketch Menu.

  7. Code PImage b; // Images must be in the "data" directory to load correctly b = loadImage(”dt.jpg"); image(b, 0, 0);

  8. What’s happening in this code?PImage b PImage is the function name and b is the declared variable name for image.// followed by comment.b = loadImage(“dt.jpg”) calls the image from the data directory to be display in the sketchImage(b, 0, 0) displays the defined b image, followed by x and y co-ordinates of the upper left hand corner of the image.

  9. Change the co-ordinates of image(b,0,0) to image(b,50,50)and see what happens

  10. There are a bunch of image proce55ing functions. We will go over them in this presentation. Also look at the proce55ing reference to check the syntax while practicinghttp://www.proce55ing.org/reference/index.html

  11. Image mode PImage b; b = loadImage(”dt.jpg"); imageMode(CORNER); image(b, 35, 35, 50, 50); Modifies the location from which images draw. image(b,35, 35, 50, 50) where 35,35 are the x and y co-ordinates of upper left hand side of the image and 50, 50 are the width and height of the image.

  12. Image mode PImage b; b = loadImage(”dt.jpg"); imageMode(CORNERS); there’s a difference between CORNER and CORNERS image(b, 35, 35, 50, 50); Modifies the location from which images draw. image(b,35, 35, 50, 50) where 35,35 are the x and y co-ordinates of upper left hand side of the image and 50, 50 are the x and y co-ordinates of lower right hand side corner of the image.

  13. Background PImage b; b = loadImage(”dt.jpg"); background(b); So, you can also set the image as a background and then draw shapes and then top.

  14. Tint and Alpha PImage b; b = loadImage(”dt.jpg"); // Tint blue tint(0, 153, 204); image(b, 0, 0); So, if you remember your RGB channels, you will get this straight. Tint(0,153,204); so our Red color is 0, Green is 153 and Blue is 204. So, R, G and B takes 8 bits each, a maximum value of 255, now notice blue is 204 which is closest to 255 and so is taking on the tint.

  15. Tint and Alpha PImage b; b = loadImage("dt.jpg"); image(b, 0, 0); // Tint blue and set transparency tint(0, 153, 204, 126); image(b, 50, 0); So, the 32-bit graphics are completed by the 4-channels, the last one being Alpha, RGBA. Notice the last number in tint value is 126, which brings the opacity close to 50%. Just to show the variation the image is loaded again with x-cord.on 50 pixels.

  16. Lets move on to Pixel manipulations.

  17. Filters PImage b; b = loadImage(“1.jpg”); Image (b,0,0); //Hit play to preview the image // insert the filter filter(THRESHOLD); //hit play now. By Default processing applies .5 threshold Value. You can apply any value between 0 and1, integer or float. For example Filter (THRESHOLD, 0.4); before after

  18. Lets try some more Filters filter(GRAY); filter(INVERT); filter(POSTERIZE, 4); filter(BLUR, 6);

  19. Get PImage myImage = loadImage("1.jpg"); image(myImage, 0, 0); PImage cp = get(10,20,20,20); image(cp, 30, 30); Reads the color of any pixel or grabs a section of an image. In this example, PImage cp = get (10,20,20,20) goes to 10x and 20y cords of the image and grab 20 px width and 20 px height from that co-ordinate. image(cp,30,30) maps the grabbed pixels on 30x and 30y.

  20. Get PImage myImage = loadImage(”1.jpg"); image(myImage, 0, 0); color cp = get(30, 20); fill(cp); rect(30, 20, 55, 55); Reads the color of any pixel or grabs a section of an image. In this example, color cp = get(30, 20) picks up the color of pixel located on 30x and 20y of the image. fill(cp) Fills the shape with the picked up color from that particular pixel.

  21. Set PImage myImage = loadImage("1.jpg"); image(myImage, 0, 0); color white = color(255); set(30, 20, white); set(85, 20, white); set(85, 75, white); set(30, 75, white); Changes the color of any pixel or writes an image directly into the display window. In this example, color white = color (255) selects the white color and set (30, 20, white) plants white color on the specific pixel co-ordinate.

  22. Copy PImage img = loadImage("eames.jpg"); image(img, 0, 0); copy(15, 25, 10, 10, 35, 25, 50, 50); Copies a region of pixels from the display window to another area of the display window. In this example, copy(15,25, 10,10 are x, y cords, width and height of the source, 35, 25, 50, 50 are x, y cords, width and height of the destination. Notice the source is 10 x 10 pixels and destination is 50 x 50 pixels, which results in 5 times scaling of pixels pasted.

  23. Blend background(loadImage("22.jpg")); PImage img = loadImage("1.jpg"); image(img, 0, 0); blend(img, 0, 0, 33, 100, 67, 0, 33, 100, ADD); Blends two color values and copies a single pixel or region of pixels from one image into another. We can use two images, to see one blending into other. blend(img, 0, 0, 33, 100,where img is the source image, x and y cords, width and height of the source image; 67,0,33,100 are the x and y cords, width and height of blending area.

  24. Blend Finally the Mode, ADD, SUBTRACT, LIGHTEST, DARKEST sets the superimposition types. blend(img, 0, 0, 33, 100, 67, 0, 33, 100, SUBTRACT); blend(img, 0, 0, 33, 100, 67, 0, 33, 100, MULTIPLY); blend(img, 0, 0, 33, 100, 67, 0, 33, 100, LIGHTEST); blend(img, 0, 0, 33, 100, 67, 0, 33, 100, DARKEST);

  25. So you have a cool Proce55ing program… What else can you do?

  26. How about adding a little sound.

  27. Now, we used to use “Psound”: Description: Class for storing and playing sounds of the .wav format.    Methods: play() - Plays the sound once loop() - Plays the sound continuously noLoop() - Stops the sound from looping pause() - Pauses the sound playback stop() - Stops the sound playback volume() - Sets the volume of the sound playback duration() - Returns the total length of the sound in seconds time() - Returns the current position of sound playback in seconds

  28. Unfortunately, “Psound” isn’t included in the latest version of Proce55ing…

  29. So, we are looking into a couple of alternatives:the Sonia and Ess librariesYou can go to:http://processing.org/reference/libraries/

  30. Things we want to do with sound:

  31. Things we want to do with sound: • Play control (play, stop, FF, reverse) • Sampling • Sound Looping

  32. For today, let’s browse through the possibilities and check out what the different sound libraries are capable of.Form 4-5 groups and go do a bit of research…

  33. Loading and Capturing Video

  34. The Processing Video Library Video can be captured from: USB cameras. IEEE 1394 (Firewire) cameras. Video cards with composite/s-video. Video can be loaded from: Any QuickTime file on your computer or on the Internet

  35. Software Requirements for Video Both Mac and PC’s need this software installed to work with video: Proce55ing version 0093 (0095 wont work due to a few bugs) Apple QuickTime

  36. PC Webcams If your PC Webcam does not use QuickTime... (Such as Logitech webcams.) you need to install WinVDIG With WinVDIG QuickTime applications can capture video on Windows. It is free to use. http://www.vdig.com/WinVDIG/

  37. If you run Windows... And if you get this error in Proce55ing: “This version of QTJava has expired” trying to load videos… You need to upgrade your Apple QuickTime 7 software. Apple forgot to remove the expiration date from the QuickTime 7 beta code base when it went into to production. http://www.apple.com/quicktime

  38. Movie Description: Datatype for storing and playing movies in Apple's QuickTime format.    Methods: read() - Reads the current frame from the movie available() - Returns true when a new frame is available play() - Plays the movie once pause() - Pauses the movie playback stop() - Stops the movie playback loop() - Plays the movie continuously noLoop() - Stops the movie from looping jump() -Jumps to a specific location in the movie duration() - Returns the total length of the movie time() - Returns the current playback position speed() - Sets a multiplier for how fast/slow a movie should play framerate() - Sets how often new frames are read from the movie

  39. Movies must be located in the sketch's data directory or an accessible place on the network to load without an error.

  40. Load a Movie //Load_Quicktime.pde import processing.video.*; Movie myMovie; void setup() { size(160, 120); background(255,0,0); myMovie = new Movie(this, "honda_clip1.mov"); myMovie.loop(); } void draw() { image(myMovie, 40, 30, 80, 60); } // Called every time a new frame is available to read void movieEvent(Movie m) { myMovie.read(); }

  41. Load a Movie Code Details import processing.video.*;  Import the Video Library. Movie myMovie;  Movie datatype variable. myMovie = new Movie(this, "honda_clip1.mov");  Load movie and pass it to your variable. Constructors  Movie(parent, filename) Movie(parent, filename, fps) Movie(parent, url) Movie(parent, url, fps)    Parameters  parent PApplet: typically use "this" filename String: name of the movie fps int: frames per second url URL: location of the file on the network myMovie.loop();  Loop the movie. image(myMovie, 40, 30, 80, 60);  Place movie on screen image(img, xpos, ypos, width, height); void movieEvent(Movie m) {  Called every time a new frame is available to read myMovie.read(); }

  42. Loading Two Movies //Quicktime_Two.pde import processing.video.*; Movie myMovie, yourMovie; void setup() { size(200, 200); myMovie = new Movie(this, "station.mov"); yourMovie = new Movie(this, "street.mov"); myMovie.loop(); yourMovie.loop(); } void draw() { image(myMovie, 0, 0, 200, 200); image(yourMovie, 50, 50, 75, 75); } void movieEvent(Movie m) { if(m == myMovie) { myMovie.read(); } else if(m == yourMovie) { yourMovie.read(); } } void mousePressed() { myMovie.stop(); yourMovie.play(); } void mouseReleased() { myMovie.play(); yourMovie.stop(); }

  43. Two Movie Code Details • Load each movie into unique variables.  Loop both of the movies. • Place both movies on the screen. • Called when a new movie frame is available. • When mousePressed/mouseReleased play/stop the movies. myMovie = new Movie(this, "station.mov"); yourMovie = new Movie(this, "street.mov"); myMovie.loop(); yourMovie.loop(); image(myMovie, 0, 0, 200, 200); image(yourMovie, 50, 50, 75, 75); void movieEvent(Movie m) { if(m == myMovie) { myMovie.read(); } else if(m == yourMovie) { yourMovie.read(); } } void mousePressed() { myMovie.stop(); yourMovie.play(); } void mouseReleased() { myMovie.play(); yourMovie.stop(); }

  44. Pixel Fun //Quicktime_Pixel.pde import processing.video.*; int numPixels; int blockSize = 5; Movie myMovie, yourMovie; color myMovieColors[]; void setup() { size(480, 360); noStroke(); background(0); myMovie = new Movie(this, "station.mov"); yourMovie = new Movie(this, "nuke.mov"); myMovie.loop(); yourMovie.loop(); numPixels = width / blockSize; myMovieColors = new color[numPixels * numPixels]; } void draw() { for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { fill(myMovieColors[j*numPixels + i]); rect(i*blockSize, j*blockSize, blockSize-1, blockSize-1); } } tint(255, 150); image(yourMovie, mouseX, mouseY); } void movieEvent(Movie m) { if(m == myMovie) { myMovie.read(); for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { myMovieColors[j*numPixels + i] = myMovie.get(i, j); } } } else if(m == yourMovie) { yourMovie.read(); } }

  45. Pixel Code Details Again we are using two movies, two variables and loops like the last example. int numPixels;  Variable for number of pixels. int blockSize = 5;  Variable for the block size. color myMovieColors[];  Datatype color Array. numPixels = width / blockSize;  Width of the movie divided by blockSize to get number of pixels. myMovieColors = new color[numPixels * numPixels];  Fill the array with new colors.

  46. Pixel Code Details Continued • For loop creates rectangles (virtual pixels) and fills them from the myMovieColors array. This is the values from myMovie. • tint() – Sets the tint of the image to be displayed • Place yourMovie on the screen positioned by the mouse void draw() { for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { fill(myMovieColors[j*numPixels + i]); rect(i*blockSize, j*blockSize, blockSize-1, blockSize-1); } } } tint(255, 150); image(yourMovie, mouseX, mouseY);

  47. Pixel Code Details Continued void movieEvent(Movie m) { if(m == myMovie) { myMovie.read(); for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { myMovieColors[j*numPixels + i] = myMovie.get(i, j); } } } else if(m == yourMovie) { yourMovie.read(); } } The if statement of the mouseEvent() reads in the frame of myMovie and gathers the pixel values for myMovieColors array. The else if reads in the current frame of yourMovie.

  48. One Last Examplewith a URL. //Quiktime_Jump.pde import processing.video.*; Movie myMovie; void setup() { size(200, 200); framerate(30); myMovie = new Movie(this, "http://itp.nyu.edu/~dbo3/icm/choke.mov"); } void draw() { image(myMovie, 0, 0, 200, 200); } void mouseMoved() { myMovie.jump(mouseX*(myMovie.duration()/width)); myMovie.read(); }

  49. Capturing Video

  50. Capture Description: Datatype for storing and manipulating video frames from an attached capture device such as a camera. Methods read() - Reads the current image available() - Returns true when a new image is available list() - Lists the devices current attached and on framerate() - Sets how often new frames are read stop() - Stops capturing frames Constructors:  Capture(parent, width, height) Capture(parent, width, height, fps) Capture(parent, name, width, height) Capture(parent, name, width, height, fps)    Parameters:  parent PApplet: typically use "this“ width int: width of the frame height int: height of the frame fps int: number of frames to read per second name String: name of the camera

More Related