1 / 49

91st MPEG meeting Kyoto, Japan

91st MPEG meeting Kyoto, Japan. Krzysztof Wegner Poznań University of Technology Chair of Multimedia Telecommunication and Microelectronics. march 2010. 91st MPEG meeting. 3D Video Ad Hoc Group High-Performance Video Coding (HVC) Discution on Royality-free Standarization

mala
Download Presentation

91st MPEG meeting Kyoto, Japan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 91st MPEG meetingKyoto, Japan Krzysztof Wegner Poznań University of Technology Chair of Multimedia Telecommunication and Microelectronics march 2010

  2. 91st MPEG meeting • 3D Video Ad Hoc Group • High-Performance Video Coding (HVC) • Discution on Royality-free Standarization • Establish of JCT with ITU-T • Proposition of establish work group on standarisation of mobile visual search

  3. 3DV Ad. Hoc raport – m17086 • Exploration Experiments (N10720) (18) • Depth maps for new sequences (4) • Technical Contributions (8) • Preprocessing methods • Depth Estimation for Moving Camera • PSPNR Tool 2.1 • Distortion Model of Virtual Views • Other (4) • Experimental Framework for 3D Video (draft)

  4. New Multiview Test Sequences for MPEG-FTV • Poznań University of Technology • Poznan_Street • Poznan_CarPark • Poznan_Hall – Poznan_Hall1 and Poznan_Hall2 • Gwangju Institute of Science and Technology (GIST) • Cafe • Nagoya University • Restaurant • Kendo • Balloons

  5. Multiview Camera Test Sequences for MPEG-FTV – m16922 • Nagoya University – 3 sequences • 8 cameras –1024x768@29.4114 – 300 frames • Restaurant Kendo Balloons • Rectified and color corected

  6. 3D Video Test Sequence and Camera Parameters – m16949 • Gwangju Institute of Science and Technology • 1 sequence – 5 cameras – 5 depth camera • Color Image - 1920x1080@30 • Depth Image – 176x144@30 – only 3 camera • Rectified and color corrected – contrast issue

  7. Poznań Multiview Video Test Sequences and Camera Parameters – m17050 • Poznań University of Technology - 3 sequences • 9 cameras – 1920x1080@25 – 600 frames Poznan_Hall Poznan_Street Poznan_CarPark • Poznan_Hall was split onto 2 sequences because of columns

  8. Exploration Experiments (EE1) Depth map generation • Poznań Sequences • M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” ,Poznań University of Technology - Poland • M17304 – O. Wang, M. Yang, X. Ji, Q. Dai,„3D Video Coding EE1 Results for “Poznan_Hall” Test Sequence”Tsinghua University – China • GIST sequence • M17223 – C. Lee, M. Kang, Y. Ho,„Results of EE1 on ‘Cafe’ Sequence”Gwangju Institute of Science and Technology (GIST)

  9. Exploration Experiments (EE1) • Nagoya Sequences • M17207 - M. Tanimoto, T. Fujii, M. P. Tehrani, M. Wildeboer, „3DV/FTV EE1 report on Kendo and Balloons sequences”Nagoya University – Japan • M17259 – F. Bruls, „3DV EE Results”Philips • M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology (NIST) -

  10. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” • Manual data for some frames was created Poznan_Street

  11. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” • Manual data for some frames was created Poznan_CarPark

  12. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” Poznan_Street

  13. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” Poznan_CarPark

  14. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” Poznan_Hall1

  15. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences” Poznan_Hall2

  16. M17176 - O. Stankiewicz, K. Wegner, M. Domański, „First version of depth maps for Poznan 3D/FTV test sequences”

  17. M17304 – O. Wang, M. Yang, X. Ji, Q. Dai,„3D Video Coding EE1 Results for “Poznan_Hall” Test Sequence”Tsinghua University – China • Results only for Poznan_Hall • with use of DERS 4.0

  18. M17304 – O. Wang, M. Yang, X. Ji, Q. Dai,„3D Video Coding EE1 Results for “Poznan_Hall” Test Sequence”Tsinghua University – China • Results only for Poznan_Hall • with use of DERS 4.0 and refinement

  19. M17304 – O. Wang, M. Yang, X. Ji, Q. Dai,„3D Video Coding EE1 Results for “Poznan_Hall” Test Sequence”Tsinghua University – China • Results only for Poznan_Hall

  20. M17223 – C. Lee, M. Kang, Y. Ho,„Results of EE1 on ‘Cafe’ Sequence”Gwangju Institute of Science and Technology (GIST) • Depth esitmated with use od DERS 5.0

  21. M17223 – C. Lee, M. Kang, Y. Ho,„Results of EE1 on ‘Cafe’ Sequence”Gwangju Institute of Science and Technology (GIST) • Objective quality assesment

  22. M17223 – C. Lee, M. Kang, Y. Ho,„Results of EE1 on ‘Cafe’ Sequence”Gwangju Institute of Science and Technology (GIST) • Subjective quality assesment

  23. M17207 - M. Tanimoto, T. Fujii, M. P. Tehrani, M. Wildeboer, „3DV/FTV EE1 report on Kendo and Balloons sequences”Nagoya University – Japan • Presented only at the meeting

  24. M17259 – F. Bruls, „3DV EE Results”Philips • Presented only at the meeting

  25. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Fully automatic depth estimation with use of DERS 5.0

  26. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Fully automatic depth estimation with use of DERS 5.0

  27. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Fog artefact

  28. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Fully automatic depth estomation with use of DERS 5.0

  29. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Fully automatic depth estomation with use of DERS 5.0

  30. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Balloon artefact

  31. M17295 – T. Senoh, K. Yamamoto, R. Oi, T. Mishina, T. Kurita„Report of 3DV Exploration Experiment1”National Institute of Information and Communications Technology • Objective quality assesment

  32. Subjective quality of data set • Subjective tests were conducted during 91st MPEG meeting • Sequences marked as Unacceptable need improvement of depth data before proceeding to EE4

  33. 3D Video - Timeline • Finalize anchor coding for existing sequences (April 2010) • Reevaluate entire test set during 92nd MPEG meeting • Final depth maps + preliminary anchor coding for new sequences (until April 2010) • Internal Draft CfP (April 2010) • Final Anchor coding for new sequences (until July 2010) • Preliminary and Final CfP (July/Oct. 2010)

  34. M17225 – E. Lee, Y. Kang, J. Jung, Y. Ho „3-D Video Generation using Multi-depth Camera System” - GIST • GIST camera system • 5 cameras (Basler Pylon GigE) • 5 depth camera (Swiss Ranger SR4000 )

  35. M17225 – E. Lee, Y. Kang, J. Jung, Y. Ho „3-D Video Generation using Multi-depth Camera System” - GIST • Radial Distorton Correction • Large amount of lens radial distortion in depth images

  36. M17225 – E. Lee, Y. Kang, J. Jung, Y. Ho „3-D Video Generation using Multi-depth Camera System” - GIST • Initial Depth Map generation • Registrated depth images were used to estimate depth maps • Segments with assign initial depth value

  37. M17225 – E. Lee, Y. Kang, J. Jung, Y. Ho „3-D Video Generation using Multi-depth Camera System” - GIST • PSNR comparison bettwen DERS and GIST algorithm • DERS 34.89 dB • GIST 35.18 dB

  38. M17208 - M. Tanimoto, T. Fujii, M. Panahpour, M. Wildeoer,„Depth Estimation for Moving Camera Test Sequences” – Nagoya • Method design for moving cameras • Standard cost based on color similarity • Additional cost based on motion Obtain matching cost by block matching Obtain motion mapsfrom motion estimation Create binary motion edge map MVedge from the motion gradient. Update Graph Cuts data term Solve by Graph Cuts

  39. M17208 - M. Tanimoto, T. Fujii, M. Panahpour, M. Wildeoer,„Depth Estimation for Moving Camera Test Sequences” – Nagoya • Based on motion estimation (4x4 blocks) motion map is created Image Motion maphorizontal component

  40. M17208 - M. Tanimoto, T. Fujii, M. Panahpour, M. Wildeoer,„Depth Estimation for Moving Camera Test Sequences” – Nagoya • Edges in motion map is calculated • Temporal consistency cost is modified as folows Motion maphorizontal component Edges in motion maphorizontal component

  41. M17208 - M. Tanimoto, T. Fujii, M. Panahpour, M. Wildeoer,„Depth Estimation for Moving Camera Test Sequences” – Nagoya • Results modified DERS 5.0 • Slightly better Algorithm Off Algorithm On

  42. M17208 - M. Tanimoto, T. Fujii, M. Panahpour, M. Wildeoer,„Depth Estimation for Moving Camera Test Sequences” – Nagoya • Results modified DERS 5.0 • Slightly better – hardly visiable Algorithm Off Algorithm On

  43. Other activities • Preparation of draft Experimental Framework for 3D Video • System configuration • Signal (przepływy) • Depthestimationalgorithm • Viewsynthesisalgorithm • Test condition

  44. Other activities • Experimental Framework for 3D Video Stereoscopic Display Auto-stereoscopic Display

  45. Collaboration with ITU-T • All activities of JVT were discontinued at London meeting • New Terms of Reference of the Joint Collaborative Team on Video Coding Standard Development (N11112) was established • Proposed actions • Recommend establishment of JCT based on N11112, and nomination of chair from ISO/IEC • Recommend to issue CfP jointly • Recommend to start the joint activity in the interim by holding a joint meeting in Dresden during 2010/04/15 – 2010/04/23 in order to • evaluate the responses to the CfP • initiate steps towards selection of technology

  46. High-Performance Video Coding (HVC) • Joint Call for Proposals with VCEG (N11113) • Long and boring • Five test classes: • A: 2560x1600 cropped from 4Kx2K, 2 sequences • B: 1920x1080p 24/50-60 fps, 5 sequences • C: 832x480 WVGA, 4 sequences • D: 416x240 WQVGA, 4 sequences • E: 1280x720p 50-60 fps, 3 sequences

  47. High-Performance Video Coding (HVC) • Two constraint categories: • Random access (for classes A-D) • Low delay (for classes B-E) • Only classed B,C,D and E will be used in subjective tests • ClasseA can be evaluated based on Rate/PSNR only • Candidates • 33 pre-registrations received • Test fee: 8900 CHF + 7.6% VAT

  48. High-Performance Video Coding (HVC) • Timeline: • 2010/01/22: Formal registration period opens. • 2010/02/15: Formal registration period ends. • 2010/02/22: Coded test material shall be available at the test site. By this date, the payment of the testing fee is expected to be finalized. • 2010/03/02: Subjective assessment starts. • 2010/04/12-13: Registration and submission of documents describing the proposals . • 2010/04/15: Cross-checking of bitstreams and binary decoders . • 2010/04/16: Subjective test results available within standardization body. • 2010/04/16-23: Evaluation of proposals at MPEG meeting

  49. Thanks for attention • Questions?

More Related