Privacy Protection for Life-log Video

1. Privacy Protection for Life-log Video Jayashri Chaudhari, Sen-ching S. Cheung, M. Vijay Venkatesh Department of Electrical and Computer Engineering Center for Visualization and Virtual Environment University of Kentucky, Lexington, KY 40507 SAFE 2007 (11-13 April), Washington, DC

2. Outline • Motivation and Background • Proposed Life-Log System • Privacy Protection Methodology • Face detection and blocking • Voice segmentation and distortion • Experimental Results • Conclusion

3. What is a Life-Log System? “A System that records everything, at every moment and everywhere you go” • Applications include • Law enforcement • Police Questioning • Tourism • Medical Questioning • Journalism • Existing Systems/work • “MyLifeBits Project”: At Microsoft Research • “WearCam” Project: At University of Toronto, Steve Mann • “Cylon Systems”: http::/cylonsystems.com at UK (a portable body worn surveillance system)

4. Technical Challenges • Security and Privacy • Information management and storage • Information Retrieval • Knowledge Discovery • Human Computer Interface

5. Technical Challenges • Security and Privacy • Information management and storage • Information Retrieval • Knowledge Discovery • Human Computer Interface

6. Why Privacy Protection? • Privacy is fundamental right of every citizen • There are no clear and uniform rules and regulations regarding video recording • Emerging technologies threaten privacy right • People are resistant toward technologies like life-log • Without tackling these issues the deployment of such emerging technologies is impossible

7. Research Contributions • Practical audio-visual privacy protection scheme for life-log systems • Performance measurement (audio) on • Privacy protection • Usability

8. Proposed Life-log System “A system that protects theaudiovisual privacyof thepersonscaptured by aportable video recording device”

9. × Usefulness √ Privacy √ Usefulness × Privacy √ Usefulness √ Privacy Privacy Protection Scheme • Design Objectives • Privacy • Hide the identity of the subjects being captured • Privacy verses usefulness: • Recording still should convey sufficient information to be useful • Speed • Protection scheme should work in real time.

10. Audio Segmentation Audio Distortion audio video Synchronization & Multiplexing storage Face Detection and Blocking System Overview S P S: Subject (The person who is being recorded) P: Producer (The person who is the user of the system)

11. Windowed Power, Pk Computation Pk <TS Pk <TU Y Y Statek=Producer Storage Statek=Subject Pitch Shifting Voice Segmentation and distortion Statek=Statek-1 orSubjectorProducer We use the PitchSOLA time-domain pitch shifting method. * “DAFX: Digital Audio Effects” by U. Z. et al.

12. Input Audio X1(n) X2(n) X2(n) X2(n) N Sa α*Sa Mixing Max correlation to preserve formant Km Pitch Shifting Algorithm Pitch Shifting : Steps 1) Time Stretching by a factor of α using window of size N and stepsize Sa Step 2) Re-sampling by a factor of 1/α to change pitch

13. camera Face detection is based on Viola & Jones 2001. Face Detection Face Tracking Subject Selection Audio segmentation results Selective Blocking Producer talking Subject talking Face Detection and Blocking

14. Experimental Results • Three types of experiments • Analysis of Segmentation algorithm • Analysis of Audio distortion algorithm • 1) Accuracy in hiding identity • 2) Usability after distortion

15. S P P S P S P Silence Transitions  S: Subject Speaking P: Producer Speaking Segmentation Experiment • Experimental Data: • Interview Scenario in quite meeting room • Three interviews recording of about 1 minute and 30 seconds long

16. Segmentation Results

17. Speaker Identification Experiment • Experimental Data • 11 Test subjects, 2 voice samples from each subject • One voice sample is used as training and the other is used for testing • Public domain speaker recognition software Script1 This script is used for training the speaker recognition software Train Script2 This script is used to test the performance of audio distortion in hiding the identity Test

18. Speaker Identification Results Distortion 1: (N=2048, Sa=256, α =1.5) Distortion 2: (N=2048, Sa=300, α =1.1) Distortion 3: (N=1024, Sa=128, α =1.5)

19. Usability Experiments • Experimental Data • 8 subjects, 2 voice samples from each subject • 1 voice is used without distortion and the other is distorted • Manual transcription (5 human tester) Manual Transcription 1.Wav (transcription1) This transcription is of undistorted voice --- stored in one dot wav file. 2.Wav (transcription2) This transcription is of distorted voice sample --- in two dot wav ---. Unrecognized words

20. Usability after distortion Word Error Rate: Standard measure of word recognition error for speech recognition system WER= (S+D+I) /N S = # substitution D = # deletion I = # insertion N = # words in reference sample Tool used: NIST tool SCLITE

21. Example Video

22. Conclusions • Proposed Real time implementation of voice-distortion and face blocking for privacy protection in Life-log video • Analysis of audio distortion for usability • Analysis of audio distortion for privacy protection Future Work: • Improvement in Segmentation and face blocking • Expanding to the larger dataset • Expanding to the noisy environment

23. Acknowledgment • People at Center of Visualization and Virtual Environment • Department of Homeland Security Thank you!

25. × Usefulness √ Privacy √ Usefulness √ Privacy √ Usefulness × Privacy