VideoTag

1. VideoTag Video Display and Recording Aaron Shepler

2. Motivation • Enable the user to view real-time video from the camera • Allow the user to annotate on top of the video image • Facilitate simple video recording

3. Previous / Related Work • Point Grey Research has written a C++ wrapper for reading/writing to an AVI • Microsoft is working on an InkCanvas control (to be released in Vista) that would potentially allow for something similar – inking on multimedia

4. Challenges • How do I use the camera in C# when the interface is in C? • The .NET framework allows for easy integration between different managed types of code: i.e. Managed C++ and Managed C# • Integration between unmanaged code and managed code is not very well documented

5. Challenges • How do I place the image on the screen once I have data using C#? • Lower level operations, such as direct access to screen are not available

6. Challenges • How do I manage the recording of the video • C# does not have built-in AVI recording functionality • Function calls from C would also have to be wrapped

7. Camera Approach / Solution • Two possible approaches that allow for use of camera in C# • Link in flycapture camera methods directly • Large blocks of unsafe { } code due to use of pointers • Link in a library that simplifies the interface • Few lines of unsafe { } code

8. Camera Approach / Solution • Mistake… • First tried to link in flycapture camera methods directly • Unsafe { } blocks of code were everywhere due to the use of pointers • Became a C-based program • Learned my lesson the hard way

9. Camera Approach / Solution • Encapsulated lower level camera functionality in a library… • Created a small library of methods • Interface was well defined • Less time spent matching compatible types

10. Camera Approach / Solution • CameraLib • Library with basic camera functionality • void initialize(); • Initializes the first camera on the FireWire bus, creates context, etc • void* grabImage(unsigned char*); • Retrieves an image based on the camera context • void destroy(); • Destroys the camera context • Pure C library for use in managed code

11. Video Approach / Solution • In order to place the image onto the screen, the InkPicture control was used • Allows the background image to be changed while maintaining the ink on top. • Create thread in C# • continuous capturing / refreshing the image

12. Recording Approach • Record to Avi using C# wrapper class • Same refresh thread adds the current bitmap to the next frame in the AVI • Pro: • Easy to access bitmaps out of the AVI later • Con: • Large file is unhandy • Avi wrapper written orginally by René Nyffenegger (modified by John Corinna), located at: • http://www.codeproject.com/csharp/steganodotnet4.asp

13. Demonstration

14. Discussion • How can we reduce the slight flicker with the video/ink? • Double buffering? • User Drawn Control?

15. Discussion • What methods of encoding the video data would be faster and more robust than AVI? • Various compression models? • Existing codecs?