1 / 56

Dalí A Multimedia Software Library

Dalí A Multimedia Software Library. Wei Tsang Ooi Brian Smith, Hsi Haye Chan, Matthew Chiu, Sugata Mukhopadhyay, Dan Rubinovitz, Jiesang Song, Steve Weiss. Motivations. Black Box. C Source. Current Software Solution. MPEG to RealVideo. OpenFile(f) GetNextFrame(). Black Box. Real

maili
Download Presentation

Dalí A Multimedia Software Library

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DalíA Multimedia Software Library Wei Tsang Ooi Brian Smith, Hsi Haye Chan, Matthew Chiu, Sugata Mukhopadhyay, Dan Rubinovitz, Jiesang Song, Steve Weiss.

  2. Motivations

  3. BlackBox C Source Current Software Solution

  4. MPEG to RealVideo OpenFile(f) GetNextFrame() BlackBox Real Video Encoder RGB MPEG Real

  5. MPEG to RealVideo OpenFile(f) GetNextFrame() YUV to RGB RGB to YUV RGB YUV YUV Decoder Encoder compressed data MPEG Real

  6. MPEG to RealVideo YUV to RGB RGB to YUV YUV Decoder Encoder compressed data MPEG Real

  7. Decode I-Frame only OpenFile(f) GetNextFrame() BlackBox ? RGB MPEG

  8. Decode I-Frame only OpenFile(f) GetNextFrame() ? YUV to RGB RGB YUV Decoder compressed data MPEG

  9. Decode I-Frame only ? YUV to RGB RGB Decoder I-frames I Frame filter compressed data MPEG

  10. Others Motivations.. • Read from network ? • Tune I/O ? • Run on handheld ? BlackBox

  11. BlackBox C Source Current Software Solution

  12. Dalí is in between.. BlackBox Dalí C Source

  13. Example

  14. Example : • Process a subset of the gray scale version of each I-frame

  15. sequence hdr GOP hdr picture picture pic hdr pic data MPEG structures

  16. What we want to do parse sequence header

  17. What we want to do parse sequence header while not eof do- find & parse picture header

  18. What we want to do parse sequence header while not eof do- find & parse picture header- if it is an I-frame then - parse the picture into DCT domain - IDCT into spatial domain

  19. What we want to do parse sequence header while not eof do- find & parse picture header- if it is an I-frame then - parse the picture into DCT domain - IDCT into spatial domain - take the gray scale version of the frame and process it

  20. Abstractions

  21. Abstractions • For input/output • For representing elements in MPEG • For storing images in DCT domain • For storing images in spatial domain

  22. BitParser BitParser BitStream & BitParser BitStream File

  23. Parse Encode Find Skip Dump MPEG Headers MpegSeqHdr MpegGopHdr MpegPicHdr Picture Data

  24. Parse Encode Find Skip Dump Find MpegSeqHdr MpegGopHdr MpegPicHdr Picture Data

  25. Parse Encode Find Skip Dump Skip MpegSeqHdr MpegGopHdr MpegPicHdr Picture Data

  26. Parse Encode Find Skip Dump Dump MpegSeqHdr MpegGopHdr MpegPicHdr Picture Data MpegGopHdr

  27. Parse Encode Find Skip Dump Parse MpegSeqHdr MpegGopHdr MpegPicHdr Picture Data width: 320height: 240pic rate: 30 fps etc

  28. Parse Encode Find Skip Dump Encode MpegSeqHdr width: 320height: 240pic rate: 30 fps etc

  29. ScImage ScBlock :- { short dc; char index[63]; int ac[63];} represents 8x8 pixels

  30. ByteImage unsigned char 0..255 represents a pixel

  31. The Code

  32. Example MpegSeqHdr *seqHdr = MpegSeqHdrNew (); MpegPicHdr *picHdr = MpegPicHdrNew (); BitStream *bs = BitStreamNew (65536); BitStream

  33. Example BitParser *bp = BitParserNew (); BitParserAttach (bp, bs); BitStream BitParser

  34. MpegSeqHdr MpegGopHdr MpegPicHdr Example BitStreamReadFromFile (bs, f, 0); BitStream File BitParser

  35. Example MpegSeqHdrFind (bp); MpegSeqHdr MpegGopHdr MpegPicHdr BitParser

  36. Example MpegSeqHdrParse (bp, seqHdr); w = (seqHdr->width + 7)/8; h = (seqHdr->height + 7)/8; MpegSeqHdr MpegGopHdr MpegPicHdr BitParser

  37. Example ScImage *y = ScNew (w, h); ScImage *u = ScNew (w/2, h/2); ScImage *v = ScNew (w/2, h/2); y u v

  38. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); status = MpegPicHdrFind (bp); }

  39. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { } status = MpegPicHdrFind (bp); }

  40. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : } status = MpegPicHdrFind (bp); }

  41. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : } BitStreamFill(bp, f); status = MpegPicHdrFind (bp); }

  42. Refill the BitStream BitParser

  43. Refill the BitStream BitParser

  44. Refill the BitStream File BitParser

  45. Memory Mapped BitStream bs = BitStreamMMapReadNew(“tennis.mpg”); File

  46. Operate on a subset

  47. * Physical ByteImage width: 8 height: 6x: 0y: 0 parentWidth: 8 isVirtual: 0firstByte

  48. * Virtual ByteImage width: 5 height: 4x: 1y: 1 parentWidth: 8 isVirtual: 1firstByte

  49. Example ScImage *virt = ScClip (y,6,2,20,20); 20 blocks 6

  50. Example ScImage *virt = ScClip (y,6,2,20,20); ByteImage *byte = ByteNew (160, 160); ScToByte (virt, byte); 20 blocks 6

More Related