1 / 32

Pupil Detection and Tracking System

Pupil Detection and Tracking System. Lior Zimet Sean Kao EE 249 Project Mentors: Dr. Arnon Amir Yoshi Watanabe. Outline. Introduction and Goals Design Methodology Model of Computation Mapping and Implementation Verification Conclusions. Motivation.

bpoole
Download Presentation

Pupil Detection and Tracking System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pupil Detection and Tracking System Lior Zimet Sean KaoEE 249 Project Mentors: Dr. Arnon Amir Yoshi Watanabe

  2. Outline • Introduction and Goals • Design Methodology • Model of Computation • Mapping and Implementation • Verification • Conclusions EE 249 Project Presentation Lior Zimet and Sean Kao

  3. Motivation • Human-computer interfaces are becoming more important • New interfaces may benefit from knowledge of the location of the user’s eyes • Auto-stereoscopic displays • Virtual Reality interfaces • Facial recognition systems • Eye gaze tracking EE 249 Project Presentation Lior Zimet and Sean Kao

  4. Goals • Exercise design process from functional specification to implementation and verification • Develop an embedded system that will find the two-dimensional location of a user’s pupils • Apply various methodologies we have learned • Use a heterogeneous collection of various components in a real-time environment EE 249 Project Presentation Lior Zimet and Sean Kao

  5. Background • Human pupils may be found using two infrared light sources • An on-axis light source will give the “red-eye” effect. EE 249 Project Presentation Lior Zimet and Sean Kao

  6. Background • An off-axis light source will give a dark-pupil effect • We can synchronize the two light sources with the capturing device EE 249 Project Presentation Lior Zimet and Sean Kao

  7. Difference of the two Images EE 249 Project Presentation Lior Zimet and Sean Kao

  8. Design Methodology • Begin with a definition of the system with illustrations • Make a formal specification using the Unified Modeling Language • Describe the system using a behavioral model • Explore architectural space • Map functionality into chosen architecture • Verify implementation EE 249 Project Presentation Lior Zimet and Sean Kao

  9. System Definition • Take in an image illuminated with two different light sources • Find the pupils in the image • Calculate the position of the pupils • Output the coordinates EE 249 Project Presentation Lior Zimet and Sean Kao

  10. UML Diagrams • UML is used for formally describing a system • Use-case diagram shows functions of the system without implying how it is done • Class diagram shows what functional blocks are used • Sequence diagrams show how the use-cases are executed EE 249 Project Presentation Lior Zimet and Sean Kao

  11. Use Case Diagram EE 249 Project Presentation Lior Zimet and Sean Kao

  12. Class Diagram EE 249 Project Presentation Lior Zimet and Sean Kao

  13. System Behavior System Architecture Mapping Refine Y-Chart of the Project Implementation of System First, we’ll describe the system behavior with an appropriate model of computation EE 249 Project Presentation Lior Zimet and Sean Kao

  14. Model of Computation • Processes large amounts of data in a similar way • Dataflow model is the most appropriate • Our system is not control-centric • But some parts of the system are easier to describe using sequential algorithms • Mixed models of computation EE 249 Project Presentation Lior Zimet and Sean Kao

  15. Simulink versus Ptolemy (Virgil) • Ptolemy has the ability to mix models of computation and has support for synchronous dataflow • But Virgil does not have a simple way to integrate sequential algorithms • Simulink has extensive support for sequential algorithms (Matlab) • But lacks clearly defined semantics, combination of dataflow and discrete-event EE 249 Project Presentation Lior Zimet and Sean Kao

  16. Simulink Model of Computation • Use one clock to synchronize the system • No implicit definitions how many tokens are generated or used • We use counters and synchronous signals to determine how many “tokens” are on edges EE 249 Project Presentation Lior Zimet and Sean Kao

  17. Simulink Model Image capture model (source) Subtraction and threshold Properties of objects on a line FIFO Update properties of known objects in a frame Connect objects in different lines Calculate (X,Y) coordinates from those properties Display the coordinates EE 249 Project Presentation Lior Zimet and Sean Kao

  18. Verifying the Functionality • Obtained images from IBM Almaden Research Center • Run Simulink model on half-scale images and verify the behavioral model EE 249 Project Presentation Lior Zimet and Sean Kao

  19. System Behavior System Architecture Mapping Refine System Architecture on Y-Chart Implementation of System Next, we’ll define the system architecture EE 249 Project Presentation Lior Zimet and Sean Kao

  20. Architecture Space PC Image capture device Programmable hardware Interface controller Frame buffer FIFO System Architecture • Image capture device • Programmable hardware • Frame buffer FIFO • Interface controller • PC EE 249 Project Presentation Lior Zimet and Sean Kao

  21. System Behavior System Architecture Mapping Refine Mapping on Y-Chart Implementation of System Map the behavior onto the architecture EE 249 Project Presentation Lior Zimet and Sean Kao

  22. Mapping and HW/SW Partition • Frame buffer • RAM vs FIFO • Subtraction and thresholding of data • Hardware offers a fast, simple way • Software requires high memory bandwidth • Extracting objects from the video data • Algorithm is non-trivial to implement in hardware, a fully parallel implementation is costly • Software implementation is straightforward for sequential algorithms, but requires fast data transfers to processor EE 249 Project Presentation Lior Zimet and Sean Kao

  23. Mapping and HW/SW Partition • Calculating coordinates • Requires a hardware divider (expensive) • Needs lots of data to be passed, calculation is cheap in software • Displaying the XY coordinates • Hardware requires complicated interface to some type of display • Easy to display on monitor, only requires small amounts of data to transfer EE 249 Project Presentation Lior Zimet and Sean Kao

  24. FPGABlock FPGABlock Image capture device FPGABlock Communication • Deal with heterogeneous blocks • Different types of blocks require different types of communication USB Register Arrays FPGABlock PC Image capture device PC Parallel data stream I2C EE 249 Project Presentation Lior Zimet and Sean Kao

  25. Design Choices • Zoran CMOS Sensor • Zoran Video IP Evaluation Board • IBM Almaden BlueEyes LED Board • Altera APEX20K FPGA • Averlogic Frame Buffer FIFO • Cypress USB Controller EE 249 Project Presentation Lior Zimet and Sean Kao

  26. Design Choices Properties of objects on a line Image capture model (source) Subtraction and threshold FIFO CMOS Sensor Frame buffer Altera FPGA and Zoran Evaluation board Connect objects in different lines Update properties of known objects in a frame PC USB controller Calculate (X,Y) coordinates from those properties Display the coordinates EE 249 Project Presentation Lior Zimet and Sean Kao

  27. Implementation from Behavioral Model • No available tool that can effectively apply model of computation to various architectural implementations yet • Xilinx SysGen tool cannot handle data stream • Real-Time Workshop for Simulink • Does not necessarily generate efficient code • Various programs that generate Verilog from C • May not be synthesizable EE 249 Project Presentation Lior Zimet and Sean Kao

  28. System Behavior System Architecture Mapping Refine Implementation • Verilog (synthesized manually) is used for hardware blocks • C/C++ (synthesized manually) used for software blocks Simulink ModelSim Implementation of System LeonardoSpectrum and Quartus EE 249 Project Presentation Lior Zimet and Sean Kao

  29. Verification • ModelSim used to simulate Verilog • LeonardoSpectrum used to synthesize Verilog into FPGA netlist • Quartus used to map and place & route • Visual C++ used to compile and simulate software components EE 249 Project Presentation Lior Zimet and Sean Kao

  30. System Demo EE 249 Project Presentation Lior Zimet and Sean Kao

  31. Conclusions • Good design methodology allowed us to exercise the complete design process in a short time span (time-to-market) • Separating functionality from implementation can solve many errors early in the design process • Communication and interface-based design is vital for system integration • No available programs to automatically synthesize the process all the way down • Tried to apply EE 249 design methodologies, but implemented with traditional techniques EE 249 Project Presentation Lior Zimet and Sean Kao

  32. Acknowledgements • Dr. Arnon Amir (IBM Almaden Research Center) • Zoran Video IP team • Zoran CMOS Sensor team • Yoshi Watanabe • Prof. Sangiovanni-Vincentelli • Rong Chen EE 249 Project Presentation Lior Zimet and Sean Kao

More Related