SoCKET Collaborative Project

1. SoCKET Collaborative Project (SoC toolKit for critical Embedded sysTems) Thales Use Case: Pedestrian tracking with smart cameras Paul Brelet paul.brelet@thalesroup.com 23/11/2011

2. Introduction • Thales interest on the project. • SOCKET Global Flow • Thales Use case description: • Application description. • Create Classifiers. • C application. • Architecture description. Workshop - November 2011

3. Thales Interest • Thales Research has several interests on the project: • Interaction between control and dataflow. • Predictability real time comportment. • Reduce the programming/re-programming cost. • Verification. Workshop - November 2011

4. SoCKET Flow System Requirements System Properties Global SoC Req. Metrics Metrics SoC Architecture Traffic generator HLS Requirements traceability Hardware properties Software properties Platform Assembly Headers generation IP-XactSoC C/C++/ASM TLMLT Software Functionality Functional validation Instruction Set Simulator TLMAT Timing + Functionality Software SW Performance Validation RTL Software HLS Co-simulation/Co-emulation Silicon Software Execution

5. Thales use case – Context Context Safety of critical infrastructures containing image processing Detection of intrusions  pedestrians Description Environment multi-camera (basic scenario using 2 cameras) having a specific visual field Centralized process of the video stream coming from a reduced number of camera in an System on Chip embedded system Workshop - November 2011

6. Use case TRT – Description App. • Pedestrians detection • Algorithm of classification [Viola&Jones] • Two steps: • Off-line: training by an image database • On-line: detection by using the training results • Pedestrians tracking in an multi-camera environment • Use of the visual covering of the cameras in order to carry out the tracking • Utilization of descriptors of forms and/or colors in order to improve the tracking and to manage occlusions Workshop - November 2011

7. Thales use case – Appl. description [Viola&Jones ICCV ’03] • Some examples Database Workshop - November 2011

8. Thales use case – App. description • Procedure • The training is carried out on a workstation • The SoC applies the classifiers and the tracking algorithm on multi-camera • Synoptic Application Bounding Box Descriptor Features Manage Descriptors Tracking result Camera Integral Image Classifiers Cascade Descriptor Table Tiler SoC Host IO Frontend processing Backend processing Workshop - November 2011

9. Thales use case – App. Description • Schematic view of the pedestrian detection Workshop - November 2011

10. To create the classifiers • From scratch: • A huge database is needed: • Minimum 10 000 positive and negative images to have good classifiers. • A good workstation and Time: • Take about 30 minutes on 8 cores Xeon 2.4GHz and 12Gb RAM. • 2 Weeks on Dual core and 2Gb RAM. • From existing classifiers: • Convert to the Thales classifiers by XML parsing. Workshop - November 2011

11. C Application • C Code (on-line) • Classifiers (off-line) • Result Workshop - November 2011

12. Thales use Case – Architecture Host Architecture details Workshop - November 2011

13. Thales use Case – Architecture Accelerator details (Engine) Workshop - November 2011

14. Questions? 23/11/2011 Workshop - November 2011