VXL: Vision Something Libraries - Open Source Computer Vision Libraries

1. What is VXL ? • Vision Something Libraries • A collection of Computer Vision libraries • Open Source, grass roots effort, 53 developers • Supported by good community, fast response • C++ with pragmatic use of templates • 600k lines of code (over-estimate) • 1.0 beta release, CVS • Widely ported • Research to Customer delivery

2. Where is VXL? • Home: vxl.sourceforge.net • Dashboard (continuous and nightly test builds): www.cs.rpi.edu/research/vision/vxl/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html • GE Private: openge.crd.ge.com/projects/vxl • Documentation (Doxygen, structured comments): www.isbe.man.ac.uk/public_vxl_doc

3. 1975 1980 1985 1990 1995 2000 History of CV Environments at GE (Mundy) Geometer demo fails TargetJr starts Morse Project Oxford Leuven VIP Fortran (Flex) Generic Graphics Xray Metrology Image Calc (SRI) Lisp Geometer (GE) Lisp XCV TargetJr Used in DARPA Research Focus Project Light Gauge TargetJr (C++) Segmentation, 3d Modeling, Video, Feature Extraction PDP-9 PDP-11 PC Platforms Unix Platforms Lisp Machine Dynamic DB Operator Paradigm Image Understanding Environment (IUE) Object Modeling Paradigm VACE VXL simple, light!

4. Why Common Environments? (Mundy) • Accumulation of effort • Basic libraries endure • Useful algorithms endure • Application delivery • Direct from research to application • Matlab not viable • Sharing of effort • Code/algorithm exchange between projects

5. Other Toolkits • TargetJr: Computer Vision • VTK: Visualization • ITK: Image Segmentation and Registration

6. Motivation for VXL • Replace TargetJr and IUE • bulky, inconsistent interfaces, slow due to design • Clean, lightweight • Consistent interface to classes • Coding standards • Portability

7. VXL Portability • Platforms: Windows, Solaris, IRIX, Linux, FreeBSD, OSF, Cygwin, PlayStation2 • Compilers: Visual C++, gcc, MipsPro, aCC • VCL: Interface to “standard” library • No RTTI, exceptions, nested classes, member templates … • CMake: Cross-platform Make, www.cmake.org • Dashboard: Dart, public.kitware.com/Dart • Changing code base with fast response to problems

8. vcsl coordinate systems vpl POSIX compatibility vcl STL compatibility vidl video vil image vgl geometry vsl streaming vul utilities vgui GUI vbl basics vnl numerics png netlib core vxl contrib v3p GE Private VXL core hierarchy vcl_cout vcl_string vcl_list <vcl_deque.h> … Core Level 1 Each Independent vpl_rmdir vpl_usleep … points, lines planes conic sections bounding boxes polygons region scan iterators homogeneous memory management file I/O convolution pyramids warping vectors, matrices SVD, QR, eigen conjugate gradient Levenberg-Marquart FFT smart pointers arrays sparse arrays command line directories regex binary object I/O Core Level 2 Call Anything AVI, mpeg2 frame iterator Cartesian, cylindrical geographic, geodetic manipulation tableaus simple and portable OR use of native GUI

9. vxl contrib tbl brl rpl mul oxl gel oul VXL contrib hierarchy Video processing, segmentation Brown U. Dense image matcher, polygon triangulation, connected spatial objects, digital geometry, KLT GE Research 3D images, PDFs, classification Manchester U. Dense image matcher, polygon triangulation Otago U. Epipolar geometry, segmentation Oxford U. Robust estimation Rensselaer Image processing KUL?

10. Who Uses VXL? • Brown University • GE Research • Katholieke Universiteit Leuven, Belgium • Manchester University, UK • Oxford University, UK • Rensselaer Polytechnic Institute • University of Otago, New Zealand

11. level 2 level 1 level 0 VXL Library Level Concept (Perera) regl retinal registration retl Retina Project libraries publ internal public rtl retinal tracing rdbmel DBM-Estimator rpl Rensselaer libraries rrel robust estimation rsdl spatial data structures racl active contours core vxl core v3p 3rd party vgui user interface vnl algo netlib vbl basics vul utilities vgl geometry vnl numerics vil imaging tiff png vcl C++ compatibility jpeg

12. What is in VXL Core Level 1? • VCL: compatibility layer library • C/C++ Standard library interface • V3P: 3rd party libraries • jpeg, png, tiff, netlib, mpeg2 • VBL: Basics library • Smart pointers, arrays, sparse arrays • VGL: Geometry • Points, lines, planes, conic sections, bounding boxes, polygons, region scan iterators, homogeneous • VIL: Imaging • Image memory management, file I/O, convolution, pyramids, warping • VNL: Numerics • Vectors, matrices, basic ops, SVD, QR, eigen systems • Amoeba, Brent, conjugate gradient, Levenberg Marquart, LBFGS (v3p) • Polynomials, FFT (v3p)

13. What is in VXL Core Level 2? • VIDL: video sequence library • AVI, mpeg2, frame iterator • VCSL: coordinate systems • Cartesian, cylindrical, geographic, geodetic • Manipulation operations • VGUI: multi-platform, OpenGL based, GUI • Tableaus • Simple and portable, or make use of native GUI • Display image in 8 lines

14. What is in VXL GE Contrib? • VSRL: Dense image matcher • GTRL: Polygon triangulation • VSOL: Connected Spatial Objects • GST: Simplified 2D Topology representations • VDGL: Digital Geometry • VTOL: 1, 2 and 3D Topology representations • GKLL: Kanade-Lucas_Tomasi feature tracker • VMAL: Multi-View Line matching • GEVD: Van-duc region finder

15. How does GE Use VXL (GE Private)? • Optical metrology for turbine blades • Fold detection, reconstruction • Multiple view image enhancement and superresolution • Registration, numerics • Airway tracking in CT • B-Splines, flexible template matching • Surveillance • Particle filters, articulated object modeling • Pipeline inspection • Multiple hypothesis tracking, Kalman filters • GE Private, project based

16. Segmentation in VXL • Recently ported from TargetJr to VXL (Mundy, Brown) • Van-duc edge detector • Region segmentation based on Van-duc edge detector

17. Questions that should be answered in these slides • What is in Target that is not in VXL? • Is there anything in the C Standard Library or STL that is not in VCL? • Need to improve the listing of what is available in contrib. Next time ask the reflector.