“15 SECONDS OF FAME” Use of Computer Vision in a Modern Art Installation

1. “15 SECONDS OF FAME”Use of Computer Vision in a Modern Art Installation Franc Solina Computer Vision Laboratory Faculty of Computer and Information Science University of Ljubljana, Slovenia

2. Motivation for this work • collaboration with the Academy of Fine Arts in Ljubljana since 1995 • new media, computer-based art installations (internet, virtual galleries, video, mobile robots, remote operation) • work of scientist and conceptual artist Ken Goldberg, UC Berkeley (TELEGARDEN) • COMPUTER VISION + ART INSTALLATION = ?

3. Video cameras in art installations • wooden mirror (Daniel Rozin) • touch me (Alba d’Urbano) • liquid views (Monika Fleischman) • … • TECHNICAL LIMITATIONS: precise positioning of the subject

4. “In the future everybody will befamous for 15 minutes.”Andy Warhol Marilyn Monroe (Andy Warhol, 1964)

5. Image mediated culture • people like to look at themselves (mirrors, photos, paintings, video) • vanity, self-discovery, self-assertion • a face in mass culture -> FAME • media attention - a mirror of the indivudual’s self-perception • WARHOL: celebrity photo -> portrait • warhol-like portrait -> instant celebrity

6. Faces in computer vision • images of people • find people, identify them, determine their activity • video surveillance • face recognition <- FACE DETECTION

7. 15 seconds of fame

8. Hardware LCD monitor Digital camera computer USB

9. learning 15 second loop Software input photo illumination compensation find faces + randomly select one transformation color filters pop-art portrait

10. Roadmap • color-based face detection • illumination compensation • pop-art color transformations • display and ordering of portraits over the Internet • conclusions

11. Our original face detection

12. Simplified face detection 1

13. Simplified face detection 2 • ADVANTAGES: faster, detected also faces from profile • DISADVANTAGES: faces of dark complexion not detected, other body parts can be detected

14. Eliminating the influence of non-standard illumination • different from daylight illumination • color constancy/compensation methods • eestimate the present illumination • reconstruct the image under standard illumination • run face detection algorithm

15. Color compensation methods • close to standard illumination • low time complexity • Grey World • Average surface color in the image is achromatic • Illumination estimation: average color • Mean gray value • Modified Grey World • Illumination estimation: each color is counted only once • White-Patch Retinex • On each image white surface is present • Illumination estimation: maximal color

16. Color compensation methods NO – original GW – Gray World MGW – Modified GW RET – White-Patch Retinex NO GW MGW RET

17. Color constancy methods • far from standard illumination • Color by Correlation • (1) LEARNING: Take images of the Macbeth color checker under present illum. and under standard illum. • Use correlation to compute the transform. Parameters • (2) APPLY TRANSFORMATION

18. Color comp. + correll. method NO GW MGW RET COR NO – original GW – Gray World MGW – Modified GW RET – White-Patch Retinex COR – Color by Correlation

19. Face detection results #1

20. Face detection after GW GW

21. Face detection results #2

22. Face detection after COR COR

23. Warhol’s celebrity portraits • segment the face from the background • delineate the contours • highlight some facial features (mouth, eyes, hair) • overlay with color screens • above transformations -> shape grammar • BUT: requires automatic segmentation into constituent face parts

24. pop-art color filters • color-balance • posterize • color-balance • posterize • hue-saturation • hue-saturation • 17 universal filters • random coloring

25. Display of portraits • different configurations • 1 big portrait • 4 smaller portraits • same filter • each with a different filter • horizontal flip • each time a different person • no detection -> last detected face with a different pop-art filter • 15 second counter

26. E-mail ordering of portraits • Beside the portrait is displayed an unique ID number • Sending e-mail to • Sending the requested picture Creating of the web page 15sec@lrv.fri.uni-lj.si Ordering system

27. The gallery of “famous” people from the project web page: black.fri.uni-lj.si/15sec

28. Audience interactions • people quickly realize that portraits of people present at the moment are displayed • if several people are present, becoming famous is elusive • subtle staging to get one’s most favourable image on the screen • subdued competition for “media” attention • narcissistic and voyeristic use of the “electronic mirror”

29. Exhibitions in art galleries • Forum Stadtpark, Graz, Austria, 19-26 Sep. 2003 • Finzgar Gallery, Ljubljana, 14-26 Nov. 2002 • 8th International Festival of Computer Arts, Maribor, 28 May-1 June 2002

30. Conclusions • well accepted by the audience • no visible interface • a group of people can interact at once • exact positioning of observers not necessary • at least one face should be found in the input image • -> high percentage of true positive face detections • -> percentage of true negative face detections can be low • a huge database for testing face detection is generated • The goal was not to mimic Andy Warhol’s portraits per se but to play upon the celebrification process and the discourse taking place in front of the installation.

31. From the first public showing