1 / 65

Video Analysis Tool Box for Digital Video Forensics

Video Analysis Tool Box for Digital Video Forensics . By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised by Dr. Chathura De Silva PhD (NUS-Singapore), MEng (NTU-Singapore), BSc Eng.( Hons ) (Moratuwa) Senior Lecturer

ataret
Download Presentation

Video Analysis Tool Box for Digital Video Forensics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Analysis Tool Box for Digital Video Forensics By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised by Dr. Chathura De Silva PhD (NUS-Singapore), MEng (NTU-Singapore), BSc Eng.(Hons) (Moratuwa) Senior Lecturer Department of Computer Science and Engineering, University of Moratuwa, 02-02-2014

  2. Agenda • Project Intro • Implementation • Video Stream Analyzer • Video Stabilizer • Compressed domain video analysis • Demo • Video Stream Analyzer • Video Stabilizer

  3. Problem Statement • Can we trust digital videos? • Are they real, computer generated or tampered • Extract some wanted Information from video • Difficult due to unclearness of video

  4. Can we trust digital videos? Figure 1‑1 : A still from controversial video aired on Channel 4 • Source – YouTube http://www.youtube.com/watch?v=rjVxyG_7mTk

  5. Can we trust digital videos? Figure 1‑2 : A still from eagle catching kid • Source – http://www.cbc.ca/news/canada/montreal/child-snatching-eagle-video-created-as-student-project-1.1268125

  6. Unclear Videos

  7. Solution • Video Stream analysis tool • To ensure the authenticity and integrity • Video Enhancement tool • To improve the visual quality

  8. Market Analysis • Video Stream Analyzing Tools • ElecardStreamEye • Tektronix MPEG Software Tools • MPEG-2 Transport Stream packet analyser • TSReader • MTS4EA Elementary Stream Analyzer

  9. Screenshots from ElecardStreamEye

  10. Some Features Of Video Stream Analyzing Tools • Navigation and display of media stream picture-by-picture (I, P, B). • Display of the current frame. • Display of the time, type, size and number of a current frame in a stream, decoding order and offset from the file beginning. • Display of the bit rate (declared in the sequence header) and a calculated bit rate. • Display of detailed information about macroblocks in MPEG-1 (ISO/IEC 11172-2), MPEG 2 (ISO/IEC 13818-2), MPEG-4 (ISO/IEC 14496-2) and AVC/H.264 (ISO/IEC 14496-10) video streams. • Information about motion vectors • Frame-accurate positioning. • Display of the stream and gathering of statistics relating to the entire file.

  11. What is missing in existing tools • All high end tools are commercial ones, Expensive • Almost all tools are designed for DVB purposes • Lack of detailed analyzing features for video forensic • Lack of textual representation

  12. Video Forecsic /Enhancement Software • Cognitech • Ocean Systems dTective • Salient Stills VideoFOCUS • StarWitness • Avid Technology, Inc. • Intergraph Video Analyst • TREC, Inc. • Forevid • MotionDSPIkena • Amped FIVE • Kinesense

  13. Features Of Video Enhancement Tools • Video Stabilization • Denoising • Deblur Filters • Detection Filters • Enhancement • Histogram Editor • Segmentation • Tracking • Transform • Zoom • Velocity • Reconstruction

  14. Implementation of Video Stream Analyzer

  15. Literature Review • Market Analysis • Most of tools are for DVB, not for forensic purposes • Lack of textual output • Possible resources [31- 35] • Video Player libraries / source codes • Decoder Libraries/Source Codes • MPEG2Event[33]is a possible candidate

  16. Video Stream Analyzer • Implementation • Based on MPEG2Event[33] library • Mpeg demuxer • ffmpeg for stream information • A simple thread sync model to get motion vectors • Quartztype library for video player implementation • C# inbuilt xml libraries for xml manipulation

  17. What is MPEG stream MPEG Bit-Stream Structure (Source [1] )

  18. GOP Structure and Display order GOP Structure and Transmission Order. (Source: [2])

  19. Video Stream Analyzer https://code.google.com/p/mymscproject/source/browse/trunk/Experiments/Mpeg2Event/Downloads/VideoAnalyzerSetup.msi?r=174

  20. Video Stream Analyzer • Features • Free and open source • Textual output (xml) of analysis reports • For further analysis • All most all the features in slide 10 • Motion vector display • Frame navigation • Bit stream index/ Display index view • User data display • Motion vector summary – xml output • Full video summary – xml output

  21. Video Stream Analyzer -outputs

  22. Motion Vectors Summary • <Pictures> • <MPicture> • <PicType>I</PicType> • <MBlocks> • <MVector> • <Row>0</Row> • <Col>0</Col> • <ForwardX>0</ForwardX> • <ForwardY>0</ForwardY> • <BackwardX>0</BackwardX> • <BackwardY>0</BackwardY> • <IsForwardPresent>false</IsForwardPresent> • <IsbackwardPresent>false</IsbackwardPresent> • <IsSkipped>true</IsSkipped> • <PicType>I</PicType> • </MVector> • <MVector> • <Row>0</Row> • <Col>1</Col> • <ForwardX>0</ForwardX> • <ForwardY>0</ForwardY> • <BackwardX>0</BackwardX> • <BackwardY>0</BackwardY> • <IsForwardPresent>false</IsForwardPresent> • <IsbackwardPresent>false</IsbackwardPresent> • <IsSkipped>true</IsSkipped> • <PicType>I</PicType> • </MVector> • <MVector> • <Row>0</Row> • <Col>2</Col> • <ForwardX>0</ForwardX> • <ForwardY>0</ForwardY> • <BackwardX>0</BackwardX> • <BackwardY>0</BackwardY> • <IsForwardPresent>false</IsForwardPresent> • <IsbackwardPresent>false</IsbackwardPresent> • <IsSkipped>true</IsSkipped> • <PicType>I</PicType> • </MVector>

  23. MacroBlocks Summary

  24. Implementation of Video Enhancement tool

  25. Video Enhancement Tool • Objective • Enhance the visual quality • Video Stabilization • Noise removal • Color Correction

  26. Video Stabilization • Removing annoying shaky motion from videos • helpful in identifying people, number plates, etc. from low-quality video cameras • Three aspects • Inter frame motion estimation • Motion smoothing and compensation • Filling up the missing image areas. • Main references [3], [4] , [5]

  27. Video Stabilization Inter Frame Motion Estimation Result of video stabilization. Top row: Original input sequence, middle row: stabilized sequence which still has missing image areas, and bottom row: stabilized and completed sequence. The grid is overlaid for better visualization. (Source [3])

  28. Handling camera motion a) Natural scene, b) when camera is moved, c) when image is shifted to compensate the motion.

  29. Motion Model • = frame at time t • = frame at time t -1 • = affine transform = T * How to find T ? • There papers ([3], [4], [5]) mentionedabove use diffèrent mechanisms to find the transform matrix parametrs

  30. Motion Estimation • Computing inter frame motion • Use of object recognition • Scale Invariant Feature Transform(SIFT) features • Minimizing quadratic error function with a proposed model [5]

  31. Stabilization

  32. Motion Smoothing • A stabilized motion path is obtained by removing undesired motion fluctuation. • Assumed that the intentional motion in the video is usually slow and smooth • Uses Gaussian kernel in most literatures • Applies Gaussian kernel to neighboring N frames • Gaussian kernel + curve fitting methods

  33. Motion Smoothing • Let Nt = {j|t-k<=j<=t+k} be the neighboring frames • And It is the frame at the origin • Calculate the position of each neighboring frame Is, relative to frame It using transform matrixes defined above ( lets say Tst) • Find the correcting transformation S from the original frame It to the motion-compensated frame I’t according to • Where G is a Gaussian kernel of size k

  34. Motion Smoothing The global transformation chain T defined over the original video frames Ii, and the transformation from the original path to the smoothed path S. (Source [3])

  35. Filling up missing image areas • Not addressed here • Due to time limitations and complexity • Some techniques used in research literature • Motion Inpainting [3] • the local motion data in the known image areas is propagated into the missing image areas. • The propagation starts at pixels on the boundary of the missing image area. Using motion values of neighboring known pixels • Use of dynamic programming [3],[6]

  36. Noise Removal (Denoising Filters) • Denoising Filter Categories [7] • Nonmotion compensated spatiotemporal • Motion compensated spatiotemporal • Nonmotion compensated temporal • Motion compensated temporal filters • What is first? motion compensation or denoising? • Each have their procs and cons • Depends on the video • References • [7], [8], [9], [10], [11]

  37. Implementation of Video Enhancement Tool

  38. Implementation of Video Enhancement Tool • Video Stabilization • Based on the algorithm described in [5] • + Improvements to reduce error propagation • Makes a additional pass (slow) • + Improvements to support color video • Noise Removal and color Correction • Based on openCV filters

  39. A Compressed Domain Approach for Video Tamper Detection

  40. Common Modes of Tampering • Classification • Spatial, Temporal or Both(Spatio-temporal) • Temporal • Frame insertion, deletion, modification and reordering • Spatial • Similar, but operates on objects within a frame • Based on MPEG's layer structure ([12]) • GOP level, frame level and single- or multi-block level • GOP jittering, inter-GOP frame and intra-GOP frame jittering, or their combinations

  41. Tamper detection techniques References – [12], [13], [14], [15], [16], [17]

  42. Compressed Domain approaches • Many researches can be found on detecting alterations by image processing means. • But very few initiatives in Compressed Domain • Due to the lack of techniques to grab the frame data from compressed video • Possibilities • Analyze stream structure headers w.r.t. its contents • Analyze userdata • Analyze inconsistencies in frame sizes, block sizes • Analyze motion vectors

  43. Our Approach • Based on motion vectors • Assumption • For an unaltered video the motions present in the video need to be smooth and therefore should not contain sudden changes • Tampering disturbs the motion flow in a video • Extract the motion information from video and compare them with the above assumption. • Motion information is derived from the motion vectors [17], [18]

  44. Concept of Motion Vector • MPEG achieves it its high compression rate by the use of motion estimation and compensation • Instead of encoding each block in the current frame, matching block is search in the past frame and if suitable block is found, the difference between the two macroblocks is encoded and transmitted • The displacement between the two macroblocksis represented using motion vector

  45. Concept Of Motion Vector cont.. • Suppose that the macroblock “x” is the macroblock we wish to encode and macroblock “y” is the counterpart in the past frame. • A search is done around “y” (search area) to find the best match for “x”. • The displacement between the two macroblocks gives the motion vector associated with “x”.

  46. Motion Vectors vsMacroBlocks • As a part of MPEG encoding process motion vectors are calculated and inserted to the bitstream. • In mpeg bitstream motion vectors are associated with macroblocks. But not every macroblock will contain a motion vector • There are four basic types of macroblocktypes • Intra • Skipped • Forward predicted • Backward predicted

  47. Forward and Backward predictions Summary of motion vectors and macroblocks

  48. Deriving Motion Field from Motion Vectors • Motion vectors represent the motions ,But not directly (therefore we need to derive) • Steps • Reorder the bitstream into display order • A simple algorithm • Generate Motion felid from motion vectors • [18] defines set of rules. • Simplified mechanism and rule set is described in the[19]. • Filter out the outliers • Median filter (3*3 as in [18])

  49. Simplified rule set for deriving motion field • Macroblocks with no motion vector have the same movement as in the previous image. • This is required since there is a small percentage of intra-coded blocks in B- and P-frames that would otherwise have unknown motion. Also, I-frames have no motion at all so this rule will provide a smooth motion field for them. The justification for this is step is that consecutive images are strongly correlated therefore there is a high probability that the motion field has not changed much. • When a macroblock has two motion vectors, the one pointing back is reversed and added to the one pointing forward. • This step accomplishes both noise reduction (due to the averaging effect) and normalizes vector magnitude (so they span three images in B-frames which is what happens in P-frames). • Motion vector magnitude is normalized. • If a macroblock in a B-frame only has one motion vector then it has to be multiplied by a factor so its magnitude corresponds to a vector spanning three images. • Skipped macroblocks in I-frames have no movement, while in P-frames, they have movement similar to the previous block. • This is a rule obtained directly from the MPEG-2 standard.

More Related