Autonomous Tracking Unit

1. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Autonomous Tracking Unit John Berglund Randy Cuaycong Wesley Day Andrew Fikes Kamran Shah Professor: Dr. Rabi Mahapatra

2. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Project Status • Objective • Develop an autonomous camera that can identify and track an object in motion • Project Goals • Develop a 2-dimensional servo control system • Develop a camera platform • Implement an FPGA to camera interface • Develop a memory and camera control system • Develop a motion tracking algorithm • Implement the algorithm in hardware • Integrate components • Graduate ?

3. Autonomous Tracking Unit Autonomous Tracking Unit SRAM Camera Memory & Camera Control Camera Interface CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Algorithm Servos Servo Control System Diagram

4. Autonomous Tracking Unit Autonomous Tracking Unit 17-20ms 0º 160º 1.75ms 1.25ms CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Servos: Pulse Width Modulation

5. Autonomous Tracking Unit Autonomous Tracking Unit Bit 7 6 5 4 3 2 1 0 If (bit 7 == 0) { :0] == angle) { If (bit[6 Minimum pulse not reached Minimum+angle pulse length generated. } else { Pulse = 0; Minimum pulse reached Pulse = 1; Test for angle } } CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Servos: Final Design • 8-bit input angle with 9-bit internal counter • Test the MSB for minimum pulse length • Minimum pulse length => 1|0000|0000b - minimum number of pulses

6. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Servo Control Components

7. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Command Command (8) Memory and Camera Control Camera Interface Module QuickCam Start Reset Hold PCAck Nibble (4) CamRdy Data (8) Send Valid Camera Interface II Goals: • Remove asynchronous data exchange • Reduce complexity of controlling logic • Provide a configurable initialization sequence

8. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Camera Interface II Components

9. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Camera Interface II Success! • Testing Issues • BSII cannot trigger on an edge of a signal • Camera’s CCD decays at a rapid rate • Performance Measurements • Store a 128x120 image in 32 ms (31 fps) • Future Improvements • Implement the byte mode transfer • Implement memory storage logic inside the interface

10. Autonomous Tracking Unit CPSC 483 - Spring 1999 Memory & Camera Control (MCC) “Memory System: This will involve implementing an SRAM interface that will store image representations and allow for them to be accessed by other system components.” • Isolates the Algorithm from the details of memory access and camera control • This module went through several design changes

11. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Original MCC Design • Asynchronous communications with the Algorithm and the Camera Control Module • 256x243 pictures and 6-bit pixels • Expected to use 256K of memory to store 3 frames • Did not unpack the pixels • Complex initialization sequence • Tested in simulation

12. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Frame (3 bit) Vertical Offset (7 bits) Horizontal Offset (6 bits) MCC Final Design • Fifth and final design • One bit Command • Streams data from the camera synchronously • Camera Control module writes directly to memory • 64K of addressable memory, 8K needed • Reduced frame size to 128x120 Final Memory Addressing Scheme

13. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 MCC Final Design (Continued) • Tested with BS2 and Camera Control module • Observed inconsistent results • We made a separate project to unit test the MCC • Worked almost perfectly in testing • Needed to the camera’s cord! jiggle

14. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 MCC Final Block Diagram

15. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Our “Simple” Environment • Single rigid object • Object is darker than its background • Object is smaller than its background • Object is travelling at a reasonable pace

16. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Algorithm Test & Development • Goal was to develop and test several algorithms • Use software to help us visually compare solutions • Tests were used to compare performance in different scenarios

17. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Algorithm • Step 1: Create a histogram of the object • Step 2: Locate edges by subtracting • Step 3: Calculate the center of the object • Step 4: Adjust camera

18. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Algorithm Issues • Problems: • No object in frame • Partial object in frame • Handshaking timing issues • Solutions: • No object: Use buffers to filter out non-significant deltas • Partial object: Make frame extremes default boundaries of the object • Timing issues: Add more states to ensure correct propagation

19. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 System Integration • Phase 1: Integrate Camera Interface Module and Memory & Camera Control Module • Simulated in Xilinx • BSII used to simulate algorithm module • Capture data with PC serial port • Phase 2: Algorithm Module Revision • Image simulated with a Xilinx stub module • Unit integrated with one dimensional movement • Two dimensional movement implemented after testing horizontal and vertical tracking individually

20. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 System Integration • Phase 3: Performance Improvement. • Initial unit integration with the Algorithm Module on a 500kHz clock • Increasing Algorithm Module clock resulted in choppy movement, due to Servo operation • Frame clock used to synchronize Servo operation • Decreasing the frame clock period from 20.2ms to 17.0ms increased the image processing rate

21. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Performance(Frame Clock = 20.2 ms) 16.6 images per second

22. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Performance Improvement(Frame Clock = 17 ms) 19.7 images per second 18.7% improvement

23. Autonomous Tracking Unit Autonomous Tracking Unit CPSC 483 - Spring 1999 CPSC 483 - Spring 1999 Possible Improvements • Use 2 SRAM chips simultaneously • Image is retrieved and processed from one chip while the next image is being stored • Pin limitations • Implement entire unit in Xilinx • Use internal RAM to store histogram information • 130 11-bit storage locations required • 45 CLBs required to store histogram information