Botany Image Retrieval

1. Botany Image Retrieval Haibin Ling University of Maryland, College Park

2. Content Problems Related works Our Experiments Summary: difficulties and future works

3. Query Query Image Prototype database Problems • Reduce botany image retrieval to leaf image retrieval • One-One classification • relatively easy • One-Many classification • More difficult • Segmentation and/or detection Query Query Image An example from specimen database

4. Works at other groups • Related works and publications • Linkoping Univ & Swedish Museum of Natural History: Classification of Leaves from Swedish Trees. • A master thesis by Oskar J O Söderkvist • Oregon State Univ: Image Retrieval from Plant Database • A master thesis by Ashit Gandhi (supervised by Lead by Thomas Dietterich) • National Institute for Agricultural Botany (UK) : Chrysanthemum Leaf Classification • A conference paper by Abbasi, Mokhtarian and Kittler • Summary • All the works are contour based • Their result are at beginning stages

5. Classification of Leaves from Swedish Trees (Linkoping Univ.) • 15 tree classes, 50 test images per class • Simple ANN + nine features (area, eccentricity, moment, etc.) • Average correct ratio is 82% • Some classes have rather low correct ratios: ulmus carpinifolia 52% sorbus hybrida 36% • from www.isy.liu.se/cvl/Projects/Sweleaf/samplepage.html

6. Content-Based Image Retrieval from Plant Database (OSU) • Six species, two kinds of data: isolated leaves & herbarium specimen • Method: match contours by dynamic programming • Good for isolated leaves, average correct ratio: 96.8% • Not so good for herbarium leaves, either as query image (correct ratio 59%) or as templates (61%). Image from http://web.engr.oregonstate.edu/~tgd/leaves/dataset.htm

7. Chrysanthemum Leaf Classification - National Institute for Agricultural Botany (UK) • Farzin Mokhtarian et al. • 400 leaves from 40 varieties. • Method: curvature scale space contours + eccentricity, circularity and aspect ratio • Result: the correct ratio (among the top 5 choices) is over 95%

8. Contour extraction Experiments at UMD (1) • D. Jacobs, H. Ling, I.J. Chu, started on March, 2003 • http://www.cs.umd.edu/~hbling/Research/Botany/botany.htm • Image preprocessing: • Contour extraction and simplification • Background subtraction • Contour based classification for isolated leaves • Model based leaf detection

9. Experiments at UMD (2) -- Isolated Leaves • Contour based classification for isolated leaves (nearest neighbor), using the Swedish leave data • Fourier descriptor, average correct ratio 90% • Shape context, average correct ratio 88% • Benefit: • Invariant to scaling, translation and rotation • Problems: • Both are “global” descriptors, difficult to handle occlusion or overlay • The leaves need to be segmented first.

10. leaf detection model specimen current result, no elimination yet Experiments at UMD (3) -- Model-Based Leaf Detection • Goal: extract leaves from specimen images, which may contain more than one leaf • Hough transform (weighted by matching cost) • Shortest path (grouping edge segments)

11. Left: 7 branches. Center: 9 branches, missing a stem. Right: dry specimen, no color and bad shape and texture. Summary - Difficulties • Difficulties • Detection/Segmentation • Deformed leaf shapes • Fresh vs. dry leaves (loss of color information, distortion, etc.)

12. Summary - Future works • Possible future experiments includes: • Composition system method and syntactic pattern recognition • Appearance based model • Texture analysis • Eigenshape analysis

13. Suggestions and Thanks Thanks!