1 / 10

Media Processing 101: Top-Down vs. Bottom-Up Approaches

Learn media processing logistics, top-down and bottom-up approaches, code extraction, design decisions, and more for CS414 Spring '09. Dive into ffmpeg, APIs, examples, and design considerations.

bertat
Download Presentation

Media Processing 101: Top-Down vs. Bottom-Up Approaches

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 414, MP1 Help Session, Spring 09 Presented by Hoang Nguyen

  2. Overview • Some logistics • Top-down approach • Start with a well-known tool (ffmpeg) • Identify relevant APIs and examples • Extract and modify code for the project • More examples to get started, but less control • Bottom-up approach • Start with basic building block(s) • Create your own custom code almost from scratch • Fewer examples, but more control • Big picture (design decisions) • Reminders

  3. Logistics • Group Setup • Group Number (1-7) • Group Directory • MP hints updated (walkthrough) • http://www.cs.uiuc.edu/class/sp09/cs414/help/cs414Hints.htm • Java Media Framework is NOT available due to 64-bit incompatibility • Install ffmpeg on your home/group directory + common errors • ALSA configuration on csil-0216 machines http://alsa.opensrc.org/index.php?title=FAQ026 • TA OH: 3-4pm Thu/Fri, SC 0207

  4. Top-down • ffmpeg/SDL • Read libavcodec and libavformat docs • Follow tutorial “How to write Video Player in Less than 1000 lines’ http://dranger.com/ffmpeg/ • Need to use video formats and data structures defined by tools • Need to figure out relevant APIs and examples!!!

  5. Top-down (cont.) • Tutorial01.c: Basic • Tutorial02.c: output to the screen as fast as possible • Need to figure out how to display at the correct frame rate • Tutorial05.c: Rate control • packet->dts is always available for MJPEG (I-frames only) • “ffmpeg –i input.X” also gives frame rate information • SDL_Delay() • Keep in mind: Processing delay needs to be taken into account for a correct frame rate display • Tutorial07.c: av_seek_frame() • av_seek_frame() may be useful for fast forward/rewind • Tutorial03.c: Playing audio

  6. Bottom-up MPEG/AVI Frame 1 Frame 2 Frame 3 Frame 4 … Convert JPEG 1 JPEG 2 JPEG 3 JPEG 4 … Image sequence Store Size of JPEG 1 For (I) Buf = Read_JPEG(I) Display_JPEG(Buf) Delay(…) Pos. of JPEG 1 JPEG 1 Pos. of JPEG 2 Size of JPEG 2 …. JPEG 2 JPEG 1 …. JPEG 2

  7. Bottom-up (cont.) • Design and implement your own simple motion JPEG format • Video header (e.g., number of frames) followed by sequence of JPEGs with JPEG headers (e.g., frame size) • Create JPEGs from video format X ffmpeg -i $1 -t 10 -f image2 -vcodec mjpeg img%d.jpg • Combine all the JPEGs into your own motion JPEG with appropriate headers for overall video and individual frames

  8. Bottom-up (cont.) • Start with basic building blocks • E.g. assume function display_jpeg() that can display single JPEG image from buffer • Create thread that reads video data from disk to a buffer • Later MPs will require thread to read video data from the network • Create thread that displays video from the buffer • E.g.: http://www.csee.umbc.edu/~pmundur/courses/CMSC691C/Client.html • Producer/Consumer

  9. Producer-Consumer Thread Thread Producer Consumer produce()/put() consume()/get() frames Buffer Disk/Net Display/Net Tutorial05.c: PacketQueue, packet_queue_put(), packet_queue_get() Java: http://www.java2s.com/Code/Java/Threads/ProducerConsumerTest.htm C/pthread: http://procrastinationrising.com/pthread-producer-consumer-example-in-c/

  10. Big picture • Fast forward & rewind • Access individual frames • Create index file for video • Media streaming over network (MP3) • Players should read from buffers and another thread should write to the buffer (either from disk or network) • Synchronization (MP4) • Timestamp audio and video frames

More Related