1 / 29

CrystalMobile: multimedia framework for mobile platforms

CrystalMobile: multimedia framework for mobile platforms. www.crystalmobile.com. Kim A. Bondarenko Crystal Reality LLC, CSA. kim@crystalreality.com. Computer Technology Department Saint-Petersburg State University of Information Technology, Mechanics and Optics,. Russia. Introduction.

jana-joseph
Download Presentation

CrystalMobile: multimedia framework for mobile platforms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CrystalMobile: multimedia framework for mobile platforms www.crystalmobile.com Kim A. BondarenkoCrystal Reality LLC, CSA kim@crystalreality.com Computer Technology DepartmentSaint-Petersburg State University ofInformation Technology, Mechanics and Optics, Russia

  2. Introduction • Application of video technologies on mobile platforms is very wide. - entertainment and adult industry - information communications - collecting of private videos - etc.

  3. Problems • Mobile platforms have not enough memory space for full-quality videos and GPRS channels have no required bandwidth capacity. • CPUs are too slow for application of complex algorithms.

  4. Why to compress video? • Uncompressed video occupies huge amounts of data. For example, for storing of one minute quality video with 320x240 resolution with 15 frames per second it is needed 320x240x3x15x60 = 200 MBytes of data. • Powerful CPU platforms allowed the implementation of complex algorithms giving 100-500 compression ratio.

  5. Common ideas of video compression • Superfluous information. • Neglect of the insignificant details for human-eye.

  6. Superfluous details that are considered in CM Video • The low space correlation of picture. In any image, near lying pixels have little difference between them. • The low time correlation of picture. In any video clip there are a lot of close frames with little difference between them.

  7. Low space correlation Normal Picture Half resolution

  8. Low time correlation Normal Picture Difference between two following frames

  9. The details that are insignificant for human-eye • The human eye almost does not analyze the color, but very good differentiate brightness of the image. • Insignificant noisy movements of image parts are mostly invisible for human-eye. • The disturbances on noisy image part are weakly differentiated.

  10. Chroma vs color 1/8 of color 1/8 of chroma

  11. Insignificant movements Normal picture The picture with random distortion

  12. The disturbances on noise Normal Picture Disturbances on noise (red)

  13. YUV Colorspace • During the translation into YUV-color every pixel (RGB vector) is multiplied by 3x3 matrix. • As the result, the Y – brightness channel and two U and V color compositions are determined. • CME uses confidant transformations with integers – it gives significant performance improvements on most platforms. • Meanwhile, the resolution of U and V channels is divided by two along each axis. As human eye does not differentiate colors as good as brightness, it’s possible to reduce the detalization of color planes twice.

  14. Low time correlation of the video • B, I and P frames. • The definition of context change and the division of video streams into blocks (‘sandwiches’). The encoding is done by sandwiches of 1-3 frames.

  15. B, I and P frames

  16. Frame processing. Macroblocks. • Every frame in sandwich is divided into 16x16 blocks (macroblock). • Every macroblock consists of 4 brightness blocks of 8x8 and two color blocks 8x8, since the color resolution of the frame is divided by two. • Most processing methods operate on macroblocks.

  17. Motion compensation • Motion compensation is intended for prediction of the motion on the picture. It uses low time correlation between close frames. • CME Video provides motion compensation of 16x16 blocks with bi-linear interpolation.

  18. Motion compensation Frame from the stream Motion vectors

  19. I-frames • Encoder analyses whole frame to find images cannot be predicted from the history.These frames should be stored as independent data blocks, I-frames. • Positioning is precise to I-frames, so I-frames periodicity should be at least one I-frame for 30 seconds.

  20. Wavelets (DWT) • The images are passed the 2D wavelet transformation. Every frame is passed by bicubic/bilinear wavelet transformation. The main advantage of the format is about wavelet transformation is applied to separated images in the picture. • The smoothing is done by bicubic/bilinear interpolation, there are no effects of DCT. The pictures on the next slide show the difference between the losses using DCT (left) and bicubic wavelet composition(right) using the same bitrate.

  21. DCT vs Wavelets DCT Wavelets

  22. Quantization • The sandwich is passed through the process of quantization – every point is rapidly divided by the definite number from vector of quantization. Every element of vector of quantization is correspondent to some frequency of wavelet composition. On this step, the most loss is occurring

  23. Storing the coefficients • Each frame is passed to zero-quantors extraction by applying quadro-tree processing. After quantization most number of small coefficients is zeroed, that is why quadro-tree processing is very efficient on this step – it’s the main part of compression • After group encoding, 16 data blocks of each frame are compressed using Huffman method.

  24. The current status of CME Engine • Video codec of CME Engine is done. High-performance audio codec is under development. • The current implementation uses fixed-point ANSI C without any proprietary libraries.

  25. Symbian 6.1 platform • CME Engine perfectly runs on Symbian 6.1 platform (Nokia Series 60 phones). There is a room for tuning video parameters, but overall playback is good. • Release version of player software for Symbian platform is ready and was offered to public on 01.10.2003.

  26. Player is working on Nokia3650 • Encoding on PC • Realtime playback on Nokia Series60 family • High quality of the video

  27. Results & comparisons • CM Video has better quality than H.263 and MPEG1 with the same bit rate. • CM Video technical parameters: - 176x144 10fps on Nokia3650/7650 - 1 Mb per minute bitstream for good quality.

  28. Standard video formats & players • MPEG1. Unusable for mobiles. • H.263 Video Recorder for Nokia from Hantro Oy and Emuzed. • Real Video. Real Player for Nokia from Real Networks.

  29. Crystal Reality LLC www.crystalreality.com • Founded in March 2003 • 900.000 downloads of Crystal Player Professional • Very strong user community in Europe, Russia and USA • Crystal Mobile Engine was developed during July 2003 – Sept 2003 • 3 fulltime and 2 part time developers are now employed • Develops mobile technologies for the future

More Related