Large-Scale, Real-World Face Recognition in Movie Trailers

1. Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright

2. Plan of Attack • Extract Facial Tracks from videos • Extract the features from the facial tracks. • Build framework to load and test data. • Begin with baseline testing (Sparse, min, mean, etc) • Algorithm development…

3. Dataset • 2010 Movie Trailers • 305 videos • Avg. of 625 faces per trailer • Avg. of 23 face trackers per trailer • PubFig • 58,787 photos • 200 people • Avg. of 295 photos per person.

4. Dataset Dictionary: Facetrack:

5. Face Tracks • We have: • Instead of:

6. Linear Combination Approach Training Images Test Image = x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9

7. Linear Combination Approach A x y = = Testing Training Coefficients

8. Linear Combination & Face Tracks

9. Linear Combination & Face Tracks

10. Linear Combination & Face Tracks

11. Linear Combination & Face Tracks

12. Linear Combination & Face Tracks

13. Linear Combination & Face Tracks

14. Linear Combination & Face tracks

15. Linear Combination & Face Tracks

16. Linear Combination & Face Tracks

17. Linear Combination & Face Tracks Sylvester Stallone

18. Variations Blurry images within face tracks Unknowns Facial Expressions

19. Variations Pose Occlusions

20. Preliminary Testing Ran SRC facial recognition on extracted facial tracks from Date Night. Choose the name that has the most “votes” in the track. Looking strictly at Steve Carell and Tina Fey tracks…

21. Results Correctly assessed 15 out of 34 face tracks correctly (44.12% accuracy). 6 out of 18 Tina Fey face tracks (33.33% accuracy). 9 out of 16 Steve Carell face tracks (56.25% accuracy).

22. Correct Result • Track 7- IDed CORRECTLY as: Tina Fey • Alyssa Milano had 2 votes • Drew Barrymore had 4 votes • Jennifer Love Hewitt had 1 votes • Salma Hayek had 3 votes • Tina Fey had 22 votes • 32 frames in this track • 68.75% identified as Tina Fey

23. Correct Result

24. Correct Result Good facial crop. Many frames. Good lighting Non blurry Generally straightforward pose

25. Incorrect Result (1) • Track 9- IDed as: Gael Garcia Bernal • Clive Owen had 2 votes • Cristiano Ronaldo had 1 votes • Gael Garcia Bernal had 3 votes • Steve Carell had 2 votes

26. Incorrect Result (1)

27. Incorrect Result (1) Blurry Not a great crop Only 7 frames

28. Incorrect Result (2) • Track 26 - IDed as: Nicole Kidman • David Duchovny had 2 votes • Lucy Liu had 1 votes • Michael Bloomberg had 1 votes • Nicole Kidman had 3 votes • Salma Hayek had 1 votes

29. Incorrect Result (2)

30. Incorrect Result (2) Profile view Poor crop Only 8 frames.

31. Incorrect Result (3) • track 40 - IDed as: Keira Knightley • Adriana Lima had 6 votes • Cindy Crawford had 8 votes • Eliot Spitzer had 1 votes • Eva Mendes had 1 votes • Gillian Anderson had 1 votes • Keira Knightley had 18 votes • Meg Ryan had 3 votes • Minnie Driver had 1 votes • Salma Hayek had 3 votes • Tina Fey had 15 votes

32. Incorrect Result (3)

33. Incorrect Result (3) Dark lighting Pose variation Motion blur toward the end 57 frames

34. What’s next? Continue to look at the coefficient vector. Look into confidence of each choice (SCI and residual error) Apply different facial recognition algorithms. Best way to choose a name for each track? (Mode, strongest confidence, etc.)