Face Recognition Benchmarks, Caveats, Comparisons and Directions Gurumurthi V. Ramanan AU-KBC RESEARCH CENTRE MIT, ANN

10. Face Recognition-Benchmarks Verification Rates

11. Face Recognition-Benchmarks Degradation in Verification Rates due to lighting conditions

12. Face Recognition-Benchmarks Identification Rates

13. Face Recognition-Benchmarks Identification Rates due to age

14. Face Recognition-Benchmarks Degradation in Identification Rates due to time

15. Misperceptions about Commercial Products: 100% match to any image at any angle Instantly recognises any person Database capabilities running to millions Reality (Research challenges): Affected by lighting, angle,size of face, quality of captured and known image Technology demonstrations done under lab / studio conditions

16. Research done across Face Recognition (FR) algorithms indicate that they are susceptible to factors such as Race, age, gender and time. These factors are not well quantified even in the research literature. A recent study on the FERET database indicated Wearing glasses makes people more recognisable contradicting previous study Relative to the majority white faces, the Asian faces and African � American faces are significantly easier to recognise. (Total � 1072 subjects: White - 720, Asian - 143, African-American -121, other races- 88). Different FR algorithms exhibit different behavior when confronted with factors such as gender.

17. In the US, State Departments of Motor Vehicles (DMV) are using software developed by Visionics and Polaroid, to prevent criminals from obtaining multiple licenses under different names. Computerised identity verification is in use by 37 of the 50 American States. The Californian DMV database contains millions of images. The violence surrounding the Euro 2000 football games in the Netherlands was analyzed through CCTV cameras. FR systems recognized many individuals entering and leaving the country resulting in arrests by the Dutch authorities. http://news6.thdo.bbc.co.uk/1/hi/euro2000/teams/england/796242.stm.

18. Royal Canadian Mounted Police (RCMP) is using a face-scanning camera in the cell area of Pearson Airport to match people who are known criminals or terrorists. Anser also uses Visionics� FaceIt system as part of their project with the National Center for Missing and Exploited Children to locate missing children on the Internet. In the July 2000 presidential elections in Mexico, Visionics� FaceIt facial recognition tools were used to build a database of 8 million voters in efforts to eliminate voter fraud. Testing and internal market research is also big business within the biometric industry. The results of many of these pilots are confidential and some are sold as IP.

19. In Super Bowl in Tampa Bay, Florida, FR system was used during the game and the week prior to it. The police claimed that they uncovered 19 people with criminal records in the crowd of over 100,000 at the Super Bowl. But American Civil Liberties Union (ACLU) claimed that �a study of the police logs showed that the system never correctly matched a face in its criminal database or resulted in any arrests�. Civil liberties groups fear a �function creep� could occur i.e., biometrics could be introduced for one reason and then turned around and used for another different one.

20. In 1893, the Home Ministry Office, UK, accepted that no two individuals have the same fingerprints leading to the development of Automatic Fingerprint Identification Systems (AFIS) in the 1960s. The current AFIS system at FBI consists of a large database of approximately 46 million �ten prints� and conducts, on an average, approximately 50,000 searches per day. Lesson: It takes time for wide acceptance of a new biometric. First modern systematic use of fingerprints seems to be in India in order to prevent the rich from paying the poor to serve in the prison in their place.

21. Some aspersions on fingerprints : stigma of criminality fingerprint quality dependent on subject population and collection environment. a 2004 fingerprint algorithms contest (NIST) revealed that fingerprint matching algorithms have false non-match error rate of 2%. This means a 100,000 transactions/day (typical in a high throughput environment) would result in 2,000 false rejects/day. lack of standards hampers inter-operability between proprietary but inexpensive systems.

22. Verification rates of the top three FR systems vs. single fingerprint matcher (Source: FRVT 2002 Evaluation report)

23. Comparative performance: At false accept rates around 0.0l, verification performance is comparable. At false accept rates below 0.01, fingerprint performance is better. At rates above 0.01, the best face recognition systems perform better. For False accepts around 0.01, that face recognition performance is now comparable to large-scale fingerprint systems available in 1998. In 2005 FRVT, FR algorithms are expected to catch up.

24. Iris has low error rates but these are not verifiable due to lack of publicly available databases. This hampers research as well as verifiability. Fragility in recent pilot studies - relatively high failure to enroll rates An Iris pilot at a school reported 22% of unsuccessful transactions in a total of 9412 transactions over three months of which 16% was due to camera capture errors and 6% due to access attempts by unknown users. More research needs to be done before any meaningful comparisons can be made.

25. Face Recognition under outdoor lighting conditions Face Recognition for Surveillance with a watch-list size of at least 100 faces 3D FR algorithms FR algorithms for victim identification FR algorithms for applications such as missing children FR algorithms that measure resemblance A deeper understanding of the manifold of faces Fusion of biometrics

26. For national id and using the body as a passport, these questions are the most challenging: How to acquire repeatable and distinctive patterns from a broad population? How to accurately and efficiently represent and recognise biometric patterns? (Anil Jain)