91.204.201 Computing IV

1. 91.204.201 Computing IV Chapter Four: Highgui Module. High Level GUI And Media Xinwen Fu

2. References • Reading assignment: Chapter 4 • Application Development in Visual Studio • An online OpenCV Quick Guide with nice examples By Dr. Xinwen Fu

3. Outline • 4.1 Adding a Trackbar to our applications! • 4.2 Video Input with OpenCV and similarity measurement • 4.3 Creating a video with OpenCV By Dr. Xinwen Fu

4. Add a Trackbar in an OpenCV window by using createTrackbar • In the previous tutorials (about linear blending and the brightness and contrast adjustments) you might have • noted that we needed to give some input to our programs, such as and beta. We accomplished that by entering this data using the Terminal • OpenCVprovides some GUI utilities (highgui.h) • An example of this is a Trackbar By Dr. Xinwen Fu

5. Example Code • Let’s modify the program made in the tutorial Adding (blending) two images using OpenCV. • We will let the user enter the value by using the Trackbar. By Dr. Xinwen Fu

6. Outline • 4.1 Adding a Trackbar to our applications! • 4.2 Video Input with OpenCV and similarity measurement • 4.3 Creating a video with OpenCV By Dr. Xinwen Fu

7. Goal • Today it is common to have a digital video recording system at your disposal. • Therefore, you will eventually come to the situation that you no longer process a batch of images, but video streams. These may be of two kinds • real-time image feed (in the case of a webcam) • or prerecorded and hard disk drive stored files. • Luckily OpenCV threats these two in the same manner, with the same C++ class. • We will learn • How to open and read video streams • Two ways for checking image similarity: PSNR and SSIM By Dr. Xinwen Fu

8. Example Code • A program that reads in two video files and performs a similarity check between them. • This is something you could use to check just how well a new video compressing algorithms works. • Let there be a reference (original) video like this small Megamind clip and a compressed version of it. By Dr. Xinwen Fu

9. By Dr. Xinwen Fu

10. Outline • 4.1 Adding a Trackbar to our applications! • 4.2 Video Input with OpenCV and similarity measurement • 4.3 Creating a video with OpenCV By Dr. Xinwen Fu

11. Goal • Whenever you work with video feeds you may eventually want to save your image processing result in a form of a new video file. • For simple video outputs you can use the OpenCV built-in VideoWriter class, designed for this. • How to create a video file with OpenCV • What type of video files you can create with OpenCV • How to extract a given color channel from a video By Dr. Xinwen Fu

12. What is a video file? • Every video file in itself is a container. • The type of the container is expressed in the files extension (for example avi, mov or mkv). • This contains multiple elements like: video feeds, audio feeds or other tracks (such as subtitles). • How these feeds are stored is determined by the codec used for each one of them. • The audio tracks commonly used codecs are mp3 or aac. • For the video files the list is somehow longer and includes names such as XVID, DIVX, H264 or LAGS (Lagarith Lossless Codec). • The full list of codecs you may use on a system depends on just what one you have installed. By Dr. Xinwen Fu

13. Video File as Container By Dr. Xinwen Fu

14. OpenCV’s Capability • OpenCV is mainly a computer vision library, not a video stream, codec and write one. • OPenCV developers tried to keep this part as simple as possible. • Due to this OpenCV for video containers supports only the avi extension, its first version. • A direct limitation of this is that you cannot save a video file larger than 2 GB. Furthermore you can only create and expand a single video track inside the container. • No audio or other track editing support here. Nevertheless, any video codec present on your system might work. By Dr. Xinwen Fu

15. Add Audio and Other Feeds • If you encounter some of these limitations you will need to look into more specialized video writing libraries such as FFMpeg or codecs as HuffYUV, CorePNG and LCL. • As an alternative, create the video track with OpenCV and expand it with sound tracks or convert it to other formats by using video manipulation programs such as VirtualDubor AviSynth. By Dr. Xinwen Fu

16. Create a Video with OpenCV • To create a video file you just need to create an instance of the VideoWriter class. • You can specify its properties either via parameters in the constructor or later on via the open function: • The name of the output that contains the container type in its extension. At the moment only avi is supported. We construct this from the input file, add to this the name of the channel to use, and finish it off with the container extension. • The frame per second for the output video. Here we keep the input videos frame per second by using the get function. • The size of the frames for the output video. Here we keep the input videos frame size per second by using the get function. • The final argument is an optional one. By default is true and says that the output will be a colorful one (so for write you will send three channel images). To create a gray scale video pass a false parameter here. By Dr. Xinwen Fu

17. Example Code • As a simple demonstration I’ll just extract one of the RGB color channels of an input video file into a new video. You can control the flow of the application from its console line arguments: • The first argument points to the video file to work on • The second argument may be one of the characters: R G B. This will specify which of the channels to extract. • The last argument is the character Y (Yes) or N (No).If this is no, the codec used for the input video file will be the same as for the output. Otherwise, a window will pop up and allow you to select yourself the codec to use. • A valid command line would look like: • video-write.exe video/Megamind.avi R Y By Dr. Xinwen Fu