150 likes | 381 Views
Building a Webcam Infrastructure for Linux. Martin Rubli Master Thesis November 3, 2006 Swiss Federal Institute of Technology, Lausanne Logitech, Fremont. Content. Webcam situation today Goals Current problems Streaming video (layer schema)
E N D
Building a Webcam Infrastructure for Linux Martin Rubli Master Thesis November 3, 2006 Swiss Federal Institute of Technology, Lausanne Logitech, Fremont
Content • Webcam situation today • Goals • Current problems • Streaming video (layer schema) • Building blocks of the new solution • Framework: component overview • Webcam library • Webcam panel library • Video application • Demo
Webcam situation today • Video4Linux 2: Linux multimedia subsystem • Good for TV tuners • Poor webcam support (focused on analog devices) • Drivers • Many drivers, few work well • Little reliable compatibility information • Good UVC driver the future is bright! • Few good webcam applications • Most applications expect analog devices • Migration from Video4Linux 1 is slow • Good user-space multimedia frameworks! • GStreamer, NMM, Phonon • Developers don’t use them often enough
Goals • Enhance the user experience! • Solid UVC driver in the kernel • Cool features (face tracking, …) • A solution that works for everyone … • … but best for Logitech customers. • Fix existing problems • Plan ahead (learn from V4L2) • Community acceptance • Open source as much as possible • But democracy is slow: avoid debate, push software out • Linux for testing (internal and user feedback)
Current problems • Video4Linux 2 (V4L2) • No device enumeration • No frame format enumeration • No format conversion • Inflexible controls • Linux UVC driver • No support for UVC extensions • Solution: • Fix existing components where possible • Use parallel solutions where necessary
Streaming video (layer schema) Before After Application Decode and display video Application Control video Decode/convertDisplay video Abstract sysfs/V4L2 controls V4L2 Abstract formats and controls GStreamer & Webcam libraries V4L2 sysfs UVC extension access Extended controls UVC driver Stream video (camera format) UVC driver Stream video (camera format) UVC camera UVC camera
Building blocks of the new solution • GStreamer Takes care of video data • Pipeline-based (display, record, …) • Filters (format conversion, effects, …) • Simple for applications to use • Webcam libraries Takes care of control and meta data • Communicate with the UVC driver to bypass V4L2 limitations • Simple for applications to use
Framework: component overview • Receive video data • Abstract controls • Video features • Relay controls • Add meta data • Provide video data to V4L2 • Provide raw control access via sysfs
Webcam library • Goal: Simplify the life of application developers • Low-level abstraction • Camera controls (V4L2 and sysfs) • Camera events (if supported by driver) • Helper functions • Device enumeration (missing in V4L2) • Device information (more details than V4L2) • Frame format enumeration (easier than V4L2)
Webcam panel library • Goal: Simplify the life of application developers even more • Single library for: • Access to camera controls (libwebcam) • Access to feature controls (liblumvp) • Adds meta data • Device name repository • Virtual controls • Remove disabled controls • Split up complex controls (e.g. AE mode bitmask) • Filter controls • Meta data to support GUI (r/o, r/w, action, …)
Video application • Only two interfaces required • GStreamer for video • libwebcampanel for everything else • No need to work with V4L2 directly • Video features come (almost) for free • Need a Logitech webcam • Application needs to add the filter element • Current features • Face tracking (multiple face/motion tracker) • Mirror • More in the future …
Licensing • Video processing library is closed-source • GStreamer is LGPL which specifically allows linking by closed-source software • All other components are open-source • Applications are GPL • Libraries are LGPL
Distribution • Compatibility information • What devices are supported? Which features? • List of drivers • Which drivers work? And with which cameras? • FAQ • Hardware and software issues • Developer FAQs • Downloads • Webcam framework • User exchange • Forum where users can ask questions (and get responses)