Download
1 / 15
150 likes | 266 Views
VideoForge. An image processing system. Contents. Current state of the art Introduction Tiers Base Core Modules User interface Interactive mode Scripting universal interface primary implementation COM scripting object Implementation templates Controllers Algorithms.
E N D
VideoForge An image processing system
Contents • Current state of the art • Introduction • Tiers • Base • Core • Modules • User interface • Interactive mode • Scripting • universal interface • primary implementation COM scripting object • Implementation templates • Controllers • Algorithms
Current state of the art • Libraries • Loki, Boost • Direct Show™, ACML, IPL(IPP), OpenCV, Microsoft Vision SDK, ImageMagick, … • Applications • GraphEdit (part of the DirectShow) • Photoshop • ImageJ • ImageForge • Adobe Premiere, …
Introduction • System for video and live-stream image processing • Platform independent • ANSI C++ • Easy-to-do computer vision • Three tier application • Base (Serialization, Communication, Logging) • Core (Module management, Memory management) • Modules (Devices, Algorithms,)
Base • Serialization • Binary vs. textual storage • Automatic vs. manual storage • Type lists, storing of arrays, classes … • Variable reflection • offers the possibility of calling the variable “by name” • m_plugin.FindVar<int>(“varName”); • m_plugin.GetVarNames(…);
User interface • Communication • Scripting possibility • User interface (Interactive mode) • Types of communication • Actions – non-parametric methods • Properties – like variables, accessor methods • Methods – methods with parameters • Events – (e.g. onClose, …)
Core • Module and plug-in management • Enumeration • Registration • Memory management • Memory (de-)allocation • Correct memory handling for erroneous modules • Actions (from plug-ins and core itself) • User interface handling
Modules and plug-ins • Module – file • Contains plug-in implementations • compiled-in or represented by DLL • Plug-ins • Sources • Feeds data into chain • Algorithms • Transforms data, somehow • Special plug-ins • usually compiled : persistence, etc.
Image & Frame • similar to IPL IMAGE structure • contains: • resolution • ROI (rectangle of interest) • MediaType • color mode (rgb, YUV, etc.) • bpp (=color depth) • Frame • time information (start, stop time, fps) • previous & next frame reference (=pointer)
Building the graph • Pipeline building • Interactive mode vs. non-interactive mode • Device (Controller = Device enumerator) • Algorithms (Algorithm chain = Composite algs)
System windows • Windows • show graphical information • part of a particular plugin • Bitmap windows • shows contents of an image buffer • Vector windows • usually used to show some information in the image • applied as a second layer onto the bitmap window • Window events • OnClose, …
Templates (1/3) • Application template • UI application • Console application • Module template • default implementation • rare modification • implements a list of plug-ins
Templates (2/3) • Controller template • Implement source enumeration • Implement data feed • Possibly implement UI • Algorithm template • Implement data transformation • Possibly implement UI • User must implement 1 method
Conclusion • We proposed system for implementation of image processing and computer vision tasks • as platform-independent as possible (ANSI C++) • consists of (lightweight) core and plugins • offers a user interactive, scriptable user interface • offers various templates for fast implementation • The system is a part of various grants • Grant UK • APVV Grant
Future work • Finalize implementation of the core with the basic modules (12/05) • basic image processing operations • Implementation of basic modules of 0(1/06) • working set optimization • Implementation of special modules (06/06) • Feature tracking • Cooperation with avatars • Statistics – classifier training
