Real time H.264 decoder implementation in robot control

1. Real time H.264 decoder implementation in robot control SaurabhGhorpade

2. H.264 decoder [1] • algorithms for evaluating the inverse transforms and packet analysis • high speed real time implementation in C/C++ for robot control • feature recognition to be implemented

3. Existing functionality • Java implementation- successful in terms of functionality but slow speed. • Reasons[2]: • Class loading at startup • Unnecessary array validations • Excessive use of heap • Garbage collection

4. C for real time control • No Java Virtual Machine so direct interaction with hardware • Better memory management using pointers • Compiler optimizations, macros can be used. • Easy and fast control system, navigation and socket.

5. Environment setup • Robot communication protocol [3], router. • Operating system: Linux • C Compiler.

6. Implementation • Multithreaded socket programming • To send commands to robot • To decode the video packet

7. Image decoding • Image divided into groups of blocks (GOB) and each GOB is further divided into macroblocks • Each macroblock contains 16x16 image in Y CbCrformat, type 4:2:0

8. Steps • Entropy decoding • Inverse zigzag, followed by inverse quantization and inverse transformation. • Forming the picture by the picture format given in fig 6. • Extracting the motion vector information from the packet followed by motion compensation. • Finally getting the video.

9. (Fig 1) Client server [3]

10. (Fig 2) packet [3]

11. (Fig 3) Modified JPEG [3]

12. H.264 Encoder (a) and decoder (b) [15]

13. (Fig 4) Motion vector estimation [3]

14. (Fig 5) 16x16 intra prediction modes [3]

15. (Fig 6) Picture format [3]

16. (Fig 7) Hardware [3]

17. Hardware description [3] • A.R. Drone: Quadrotor. Batteries: • The AR.Drone uses a charged 1Ah, 11.1V LiPo batteries to fly Motion sensors • ultrasound telemeter • camera aiming downwards Video streaming: The frontal camera is a CMOS sensor with a 90 degrees angle lens.

18. References • [1] F. Pescador, M.J.Garrido, C.Sanz, E.Juarez, M.C.Rodriguez and D.Samper, “A real-time H.264 MP decoder based on a DM642 DSP”,14th IEEE International Conference on Electronics, Circuits and Systems, Madrid, Spain, ICECS 2007, Vol. 11, pp.1248 – 1251, Dec. 2007. • [2] Java is slower than C: http://www.jelovic.com/articles/why_java_is_slow.htm • [3] A.R. Drone Developer’s guide: https://projects.ardrone.org/login?back_url=http%253A%252F%252Fprojects.ardrone.org%252Fattachments%252Fdownload%252F365%252FARDrone_SDK_1_7_Developer_Guide.pdf • [4]A.R.Drone • http://ardrone.parrot.com/parrot-ar-drone/usa/ • [5] Y-Cb-Cr format: http://en.wikipedia.org/wiki/YCbCr • [6] H.264 Reference: http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC • [7] W.T.Staehler and A.A.Susin, “Real-time 4x4 intraframe prediction architecture for a H.264 decoder.” UFRGS, Alegre Telecommunications Symposium, 2006 International, pp. 416 – 421, Sept. 2006  

19. References • [8]Wifi wiki : http://en.wikipedia.org/wiki/Wi-Fi • [9] make wiki : http://en.wikipedia.org/wiki/Make_(software) • [10] Introduction to make (linux make for building C/C++ sources) http://linuxdevcenter.com/pub/a/linux/2002/01/31/make_intro.html • [11] JPEG wiki: http://en.wikipedia.org/wiki/JPEG • [12] Shih-Tse Wei, Chia-Wei Tien, Bin-Da Liu and Jar-Ferr Yang , “Adaptive Truncation Algorithm for Hadamard-Transformed H.264/AVC Lossless Video Coding”. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, pp. 538 - 549, May 2011. • [13] Runlength coding wiki: http://en.wikipedia.org/wiki/Run-length_encoding • [14] Huffman coding wiki: http://en.wikipedia.org/wiki/Huffman_coding • [15] Soon-kak Kwon*, A. Tamhankarand K.R. Rao, “ Emerging H.264/AVC video coding standard”, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006. • http://www-ee.uta.edu/dip/Courses/EE5351/ee5351.htm-H.264 review

20. Thank you! • Any questions?