1 / 48

Video and Image Processing At Purdue

Video and Image Processing At Purdue. Edward J. Delp Video and Image Processing Laboratory ( VIPER) School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA email: ace@ecn.purdue.edu http://www.ece.purdue.edu/~ace. Acknowledgements. Students -

kerryn
Download Presentation

Video and Image Processing At Purdue

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video and Image Processing At Purdue Edward J. Delp Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA email: ace@ecn.purdue.edu http://www.ece.purdue.edu/~ace

  2. Acknowledgements • Students - • Eduardo Asbun • Dan Hintz • Paul Salama • Ke Shen • Martha Saenz • Eugene Lin • Ray Wolfgang • Greg Cook • Sheng Liu

  3. Intel T4E Project • Purdue awarded $6.2 million in August 1997 for equipment • this is one of many strong relationships between Intel and Purdue • Has had a very significant impact on how we do research! THANKS! http://www.cs.purdue.edu/homes/jtk/intel/

  4. Image and Video Processing at Purdue Purdue has a rich history 60 year history in image and video processing.

  5. VIPER Research Projects • Scalable Video and Color Image Compression • still image compression (CEZW) • high and low bit rate video compression (SAMCoW) • wireless video • Error Concealment • Content Addressable Video Databases (ViBE) • Scene Change Detection and Identification • Pseudo-Semantic Scene Labeling • Multimedia Security: Digital Watermarking

  6. VIPER Research Projects • Multicast Video • Analysis of Mammograms • Embedded Image and Video Processing

  7. Other Purdue Projects • Electronic Imaging - Jan Allebach and Charles Bouman • half-tone printing • compound document compression • image databases • Remote Sensing - David Langrebe • Medical Imaging - Charles Bouman, Peter Doerschuk, Thomas Talavage, Edward Delp • computed imagng • functional MRI • x-ray crystallography • breast imaging

  8. Density 1 Density 2 Density 3 Density 4 Analysis of Mammograms

  9. Detection Results A 12.4mm lesion detected at the second coarsest resolution Automatic Detection Ground Truth

  10. Detection Results A 6.6mm lesion detected at the finest resolution Automatic Detection Ground Truth

  11. ViBE: A New Paradigm for Video Database Browsing and Search • ViBE has four components • scene change detection and identification • hierarchical shot representation • pseudo-semantic shot labeling • active browsing based on relevance feedback • ViBE provides an extensible framework that will scale as the video data grows in size and applications increase in complexity

  12. Video Analysis: Overview Closed-caption information Proc. Audio data Proc. Shot Transition Detection and Identification Compressed video sequence Image data (DC frames) Data Extraction Proc. MPEG-related data (MVs, AC coeffs, etc.) Shot attributes Transition locations and types Shot Labeling Shot trees Intrashot Clustering Proc. Captions

  13. Navigation via the Similarity Pyramid Zoom in Zoom out Zoom in Zoom out

  14. Browser Interface Control Panel Similarity Pyramid Relevance Set

  15. Video Over IP: Unicast

  16. Video Over IP: Multicast

  17. Video Over IP • Currently multicasting 3 streams • Multicast experiments with Europe • Multicast HDTV over Internet2 • Issues: • what is the backward information? • which video compression technique? • how is network control connected to the server/encoder?

  18. Why is Digital Watermarking Important? • Scenario • an owner places digital images on a network server and wants to detect the redistribution of altered versions • Goals • verify the owner of a digital image • detect forgeries of an original image • identify illegal copies of the image • prevent unauthorized distribution

  19. Why is Watermarking Important?

  20. Why is Watermarking Important?

  21. Why Watermarking is Important?

  22. Why is Watermarking Important?

  23. VW2D Watermarked Image

  24. Image Adaptive Watermarks (DCT)

  25. Scalable Image and Video Compression • Problem: desire to have a compression technique that allows decompression to be linked to the application • databases, wireless transmission, Internet imaging • will support both high and low data rate modes • Other desired properties: • error concealment • will support the protection of intellectual property rights (watermarking)

  26. Rate Scalable Image and Video Coding • Applications • Internet streaming • Image and video database search - browsing • Video servers • Teleconferencing and Telemedicine • Wireless Networks

  27. Scalability • Picture Coding Symposium(April 1999) - panel on “The Future of Video Compression,” importance of scalability: • rate scalability (Internet and wireless) • temporal scalability (Internet and wireless) • spatial scalability (databases - MPEG-7) • content scalability (MPEG-4) (Computational Scalability - implementation issues)

  28. Scalability “Author and Compress once - decompress on any platform feed by any data pipe”

  29. Scalability: Compression Standards • Scalability in JPEG • progressive mode • JPEG 2000 • Scalability in MPEG-2 • scalability is layered • Scalability in MPEG-4 • layered • “content” • fine grain scalability (fgs)

  30. Color Embedded Zero-Tree Wavelet (CEZW) • Developed new technique known as Color Embedded Zero-Tree Wavelet (CEZW) • Modified EZW with trees connecting all color components • can be extended to other color spaces

  31. Spatial Orientation Trees EZW SPIHT

  32. New Spatial Orientation Tree (CEZW)

  33. Color Image Compression Original CEZW JPEG SPIHT

  34. Coding Artifacts CEZW Original JPEG SPIHT

  35. Comparison JPEG 0.25 bits/pixel CEZW 0.25 bits/pixel

  36. Color Compression - Experiments • Objectives: • Evaluate scalable color image compression techniques • Color Transformations • Spatial Orientation Trees and Coding Schemes • Embedded Coding • Embedded Zerotree Wavelet: Shapiro (Dec’93) • Set Partitioning in Hierarchical Trees: Said & Pearlman (Jun’96) • Color Embedded Zerotree Wavelets: Shen & Delp (Oct ‘97) M. Saenz, P. Salama, K. Shen and E. J. Delp, "An Evaluation of Color Embedded Wavelet Image Compression Techniques," VCIP 1999

  37. Metrics

  38. SAMCoW • New scalable video compression technique - Scalable Adaptive Motion COompensated Wavelet compression • Features of SAMCoW: • use wavelets on entire frame and for prediction error frames • uses adaptive motion compensation to reduce error propagation • CEZW is used for the wavelet coder on both the intra-coded frames and prediction error frames

  39. Generalized Hybrid Codec

  40. Adaptive Motion Compensation

  41. SAMCoW Enhancements • B frames (ICIP98) • unrestricted motion vectors (ICIP98) • half-pixel motion searches (ICIP98) • detailed study of PEF (ICIP99 and VLBW99) • denoising techniques • bit allocation and rate control (ICIP99)

  42. Error Concealment • In data networks, channel errors or congestion can cause cell or packet loss • When compressed video is transmitted, cell loss causes macroblocks and motion vectors to be removed from compressed data streams • Goal of error concealment: Exploit redundant information in a sequence to recover missing data

  43. Error Concealment Original frame Damaged frame

  44. Approaches for Error Concealment • Two approaches for error concealment: • Active concealment: Use of error control coding techniques and retransmission • unequal error protection • Passive concealment: The video stream is post- processed to reconstruct missing data • Passive concealment is necessary: • where active concealment cannot be used due to compliance with video transmission standards • when active concealment fails

  45. Error Concealment

  46. Error Concealment

  47. Future Work • Video Streaming (wired and wireless) • Color Compression experiments (JPEG2000) • Video databases ViBE • Video watermarking • Internet2 and multicasting scalable video • Error concealment in embedded codecs

  48. How I Spent My Summer

More Related