1 / 56

Dali Virtual Machine

Explore Dali Virtual Machine for efficient multimedia data processing, video editing, and content conversion. Learn about its design goals, bytecode examples, and strategies for performance optimization.

tsonya
Download Presentation

Dali Virtual Machine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Dali Virtual Machine Wei Tsang Ooi (With Brian, Sugata, Tibor, Steve, Haye, Matthew, Dmitriy)

  2. Motivations • We know how to encode, transmit and store multimedia data today • People start looking into ways to process multimedia data

  3. Video Editing • Concat, cross-fade, overlay, cut and paste

  4. Gateway • Convert from one format to another • Change compression rate transcoder output video input video format B (format A)

  5. DBMS “ Find all movies in the database that contain this image “

  6. Set-top Boxes • Subtitling and close captioning • Mixing of incoming movies

  7. Lecture Browser • Match video to slide • Switch between video streams

  8. Current Solutions • Black box C code (mpeg_play) • Hard to reuse/adapt/break apart • Unpredictable performance • Standard library (PPM, IJG) • Least common denominator (RGB for all) • Video frames --> RGB --> gray

  9. History • Rivl - high level scripting language for multimedia processing • Still suffers from “black box” problem

  10. Experience • Real cost of processing • Inherently complex operations • Projective transform • Layered operations • Video decoding • Simple operations but lots of data • Copy

  11. Dali Architecture User Expert User Dali Code DVM Code Dali Virtual Machine Dali Compiler MMX TriMedia C

  12. Today’s Talk DVM Code Dali Virtual Machine

  13. DVM Code : Design Goal • Small number of composable abstractions • Simple, predictable opcodes • High performance • Target for compiler • Easy to extend

  14. Abstractions • Byte image • Bitstream • MPEG • JPEG • Bit masks • PCM

  15. Abstractions • Byte image • Bitstream • MPEG • JPEG • Bit masks • PCM

  16. Byte Image • Two dimensional array of byte • Can be either physical or virtual physical image virtual image

  17. Byte Image Can Represent • Gray scale image • RGB image • YUV image • Alpha channel

  18. Example Code : Fractal

  19. image smaller newh neww smaller = byte_new (neww, newh); byte_shrink_2x2(image,smaller);

  20. target dx image 2dy target= byte_clip(image,0,0,dx,2*dy);byte_set(target,0);target = byte_clip(image,3*dx,0,dx,2*dy);byte_set(target,0);

  21. target dx image smaller newh neww target = byte_clip(image, dx, 0, neww, newh);byte_copy(smaller, target);

  22. dx image 2dy smaller newh neww target = byte_clip(image, 2*dx, 2*dy, neww, newh);byte_copy(smaller, target);target = byte_clip(image, 0, 2*dy, neww, newh);byte_copy(smaller, target);

  23. DVM Code Dx = 0.25 * byte_width(image);dy = 0.25 * byte_height(image);neww = 0.5 * byte_width(image);newh = 0.5 * byte_height(image);smaller = byte_new (neww,newh);for (i = 0; i < n; i++) { byte_shrink_2x2(image,smaller); Target = byte_clip(image,0,0,dx,2*dy); byte_set(target,0); target = byte_clip(image,3*dx,0,dx,2*dy); byte_set(target,0); target = byte_clip(image, dx, 0, neww, newh); byte_copy(smaller, target); target = byte_clip(image, 2*dx, 2*dy, neww, newh); byte_copy(smaller, target); target = byte_clip(image, 0, 2*dy, neww, newh); byte_copy(smaller, target);}

  24. General Dali Strategies • Specific instruction • byte_shrink_2x2, byte_shrink_4x4, etc • Explicit memory allocation • byte_new • Reduce data • byte_clip • Composable abstraction • byte image

  25. Performance run fractal with n = 4 on 800x600 gray scale image about 21 frames/sec for 320x240 video

  26. Abstractions • Byte image • Bitstream • MPEG • JPEG • Bit masks • PCM

  27. MPEG • Getting important and pervasive • Complex format • Complex code (mpeg_play) • Most decoders provides GetNextFrame() interface • Random access, direct transfer difficult

  28. Abstraction for MPEG Video seq hdr gop hdr gop hdr seq end . . . gop gop

  29. Abstraction for MPEG Video seq hdr gop hdr gop hdr seq end . . . gop gop pic hdr pic hdr . . . pic pic

  30. DVM Code Examples • Skip to the n-th framefor (i = 0; i < n; i++) { mpeg_pic_hdr_find(bs); mpeg_pic_hdr_skip(bs); }

  31. DVM Code Examples • Skip to the n-th framefor (i = 0; i < n; i++) { mpeg_pic_hdr_find(bs); mpeg_pic_hdr_skip(bs); }

  32. DVM Code Examples • Skip to the n-th framefor (i = 0; i < n; i++) { mpeg_pic_hdr_find(bs); mpeg_pic_hdr_skip(bs); }

  33. DVM Code Examples • Skip to the n-th framefor (i = 0; i < n; i++) { mpeg_pic_hdr_find(bs); mpeg_pic_hdr_skip(bs); }

  34. Concat 2 Video Sequences inbs1 inbs2 hdr gop hdr gop outbs hdr gop hdr gop

  35. DVM Code • Concat two sequences While not end_of(inbs1) { gop_hdr_dump(inbs1, outbs); gop_dump(inbs1, outbs); } While not end_of(inbs2) { gop_hdr_dump(inbs2, outbs); gop_dump(inbs2, outbs); }

  36. Performance Analysis • Decode MPEG sequences to PPM • No output • Run on 3 different sequences

  37. Performance

  38. Performance

  39. MPEG Strategies • Expose structure • Gop header, gop instead of just frames • Break up layer • Parse, decode, color space conversions

  40. Show Time ! • Decode a MPEG sequence • For each frame scale it to “stamp” size • Create a contact sheet

  41. Abstractions • Byte image • Bitstream • MPEG • JPEG • Bit masks • PCM

  42. What Is Bitstream ? • Skip to the n-th framefor (i = 0; i < n; i++) { mpeg_pic_hdr_find(bs); mpeg_pic_hdr_skip(bs); }

  43. Bitstream chunk of memory parser 1011001

  44. Mode of Operations • File i/o static memory parser fread/fwrite

  45. Mode of Operations • Network static memory parser socket

  46. Mode of Operations • Memory map entire file parser

  47. mpeg system stream inbs audio video audio video audio video

  48. mpeg system stream inbs audio video audio video audio video filter videobs

  49. mpeg system stream inbs audio video audio video audio video filter videobs parser

  50. Bitstream Strategies • Predictable performance • Explicit I/O

More Related