Towards automatic coin classification

1. Towards automatic coin classification L.J.P. van der Maaten E.O. Postma

2. Introduction • RICH project (Reading Images for the Cultural Heritage) • Initiated by NWO-CATCH (grant 640.002.401) • Institutions involved: • MICC-IKAT (Maastricht University) • ROB (Dutch State Service for Archaeology) • People involved: • E.O. Postma, A.G. Lange, H.H. Paijmans, L.J.P. van der Maaten, P.J. Boon

3. Introduction • Automatic coin classification based on visual features • May allow sorting heterogeneous coin collections, both modern and historical • For modern coins, applications for charity organizations, financial institutions, and change offices (MUSCLE CIS benchmark) • For historical coins, applications in the cultural heritage domain

4. Introduction • Currently, some coin historical collections are being disclosed on the Internet, e.g., in NUMIS1 • NUMIS website shows information (and sometimes photographs) on collected coins • However, the use of such websites for non-experts is limited 1 A project of the Dutch Money Museum

5. Introduction • Non-experts who find a coin would like to know what sort of coin it is • i.e. coin classification based on visual features • Non-experts would benefit from a system for automatic coin classification • Also beneficial to experts to speed up and objectify the classification process

6. Introduction • A modern and a historical coin photograph

7. Introduction • This presentation • Presents a number of features that can be used for the classification of modern coins • Shows promising results for these features • Investigates the performance of the same features on a medieval coin dataset • Tries to provide some insight in why the features fail on the medieval coin data

8. Features • Contour features • Edge distance distributions • Edge angle distributions • Edge angle-distance distributions • Texture features • Gabor histograms • Daubechies D4 wavelet features

9. Contour features • Measure statistical distributions of edge pixels • Edge pixels computed using Sobel filter convolution (with non-maxima suppression and dynamic thresholding) • Coin borders are removed

10. Edge distance distributions • Estimate the distribution of the distances of edge pixels to the center of the coin • Rotation invariant feature • Can be measured on coarse-to-fine-scales

11. Edge angle distributions • Measure distribution of angles of edge pixels w.r.t. the baseline • Not rotation invariant by definition (however, the magnitude of the Fourier transform is) • Can be measured on number of fine scales

12. Edge angle-distance distr. • Incorporate both angular and distance information in the coin stamp • We measure EADD using 2, 4, 8, and 16 distance bins and 180 angular bins

13. Gabor histograms • Convolution of coin image with Gabor filters of various scales and rotations • Compute image histograms of the resulting convolution images • Apply PCA for dimensionality reduction (200 dimensional)

14. Daubechies D4 wavelet • Perform wavelet decomposition using Daubechies D4 wavelet • Computed wavelet coefficients are used as features (2-, 3-, and 4-level; ahvd) • Do this for 16 rotated versions of the coins in the training set (for rotation invariance) • Apply PCA for dimensionality reduction (results in 200-dimensional feature vector)

15. Experiments • Performed on the MUSCLE CIS benchmark coin dataset1 • The dataset contains 692 coin classes with 2,270 coin faces • Training set of 20,000 coins • Test set of 5,000 coins • Incorporate area measurements 1 Newer experiments than the ones described in the paper

16. Experiments

17. Experiments • Classification performances (5-NN classifier)

18. Experiments • Subsequently, we performed experiments on the Merovingen dataset1 • Contains 4,569 early-medieval coins • Class distribution skewed • Experiments using 10-fold cross validation 1 Dataset property of Dutch Money Museum

19. Experiments • Skewed class distributions

20. Experiments • Classification performances(naïve Bayes classifier)

21. Discussion • Although results on modern coin data are promising • Results on Merovingen coin dataset disappointing

22. Discussion • Reasons for results on medieval coins: • Contour features highly rely on the correct estimation of the center of the stamp • Texture features more suitable for coins with detailed artwork in stamps • Errors and inconsistencies in these kind of datasets

23. Discussion • Coin classified as Frankish • Coin classified as Frisian

24. Discussion • Reasons for results on medieval coins : • Medieval coins have larger within-class variances due to quick deterioration of medieval coin stamps • Medieval coins have smaller between-class variances (stamps often contain similar pictures, such as a cross or the head of an authority)

25. Discussion • Reasons for results on medieval coins : • Experts indicate that classifications are based on small details • I.e. expert classifications are based on a large number of small (undocumented) rules • Experts (consciously or not) take extrinsic information into account (such as finding location)

26. Discussion • How should a system for automatic classification of medieval coins work? • Text is highly discriminating, however, cannot be read by state-of-the-art in character recognition • We foresee the development of a semi-automatic adaptive system in which the expert indicates distinguishing features of the coin • Over time, the system should be able to learn the undocumented rules

27. Conclusions • Contour and texture features perform well in the classification of modern coins • The results of these features on early-medieval coins are disappointing • There are various reasons why the features fail in the classification of medieval coins • Future work: semi-automatic approach

28. Questions ?