研 究 生：方瓊瑤

1. A Vision-Based Driver Assistance System Based on Dynamic Visual Model 指導教授：陳世旺博士 傅楸善博士 研 究 生：方瓊瑤

2. Outline • Introduction • Dynamic visual model (DVM) • Neural modules • Road sign recognition system • System to detect changes in driving environments • System to detect motion of nearby moving vehicles • Conclusions

3. Introduction (1) -- ITS • Intelligent transportation system (ITS) • Advanced traffic management systems (ATMS) • Advanced traveler information systems (ATIS) • Commercial vehicle operations (CVO) • Advanced public transportation systems (APTS) • Advanced rural transportation systems (ARTS) • Advanced vehicle control and safety systems (AVCSS) • Driver assistance systems (DAS)

4. Introduction (2) -- DAS • Driver assistance systems (DAS) • Safety • Passive • Active • Driving is a sophisticated process • The technology of vehicle • The temperament of the driver

5. Introduction (3) -- VDAS • Vision-based driver assistance systems (VDAS) • Difficulties of VDAS • Weather and illumination • Daytime and nighttime • Vehicle motion and camera vibration

6. Introduction (4) • Subsystems of VDAS • Road sign recognition system • System to detect changes in driving environments • System to detect motion of nearby moving vehicles

7. Introduction (5) -- DVM • DVM: dynamic visual model • A computational model for visual analysis using video sequence as input data • Two ways to develop a visual model • Biological principles • Engineering principles • Artificial neural networks

8. Video images Data transduction Sensory component Episodic Memory Information acquisition Spatialtemporal information Perceptual component STA neural module No Focuses of attention Yes Feature detection Categorical features Conceptual component CART neural module Category Pattern extraction Patterns CHAM neural module Action Dynamic Visual Model

9. Physical stimuli Data compression Transducer Low-level feature extraction Sensory analyzer High-level feature extraction Perceptual analyzer Classification and recognition Conceptual analyzer Class of input stimuli Human Visual Process

10. Neural Modules • Spatial-temporal attention (STA) neural module • Configurable adaptive resonance theory (CART) neural module • Configurable heteroassociative memory (CHAM) neural module

11. STA Neural Network (1) ak ai Output layer (Attention layer) nk ni Inhibitory connection wij Excitatory connection xj nj Input layer

12. Gaussian function G Attention layer ni rk nk corresponding neurons wkj nj Input neuron The linking strengths between the input and the attention layers STA Neural Network (2) • The input to attention neuron nidue to input stimuli x:

13. Interaction + Lateral distance “Mexican-hat” function of lateral interaction STA Neural Network (3) • The input to attention neuron ni due to lateral interaction:

14. STA Neural Network (4) • The net input to attention neuron ni : : a threshold to limit the effects of noise where 1< d <0

15. STA Neural Network (5) stimulus activation t 1 1 p pd The activation of an attention neuron in response to a stimulus.

16. Orienting subsystem Attentional subsystem Category representation field F2 y Signal generator Reset signal S Input representation field F1 ＋ q ＋ ＋ r ＋ － ＋ － － ＋ ＋ G p ＋ G ＋ ＋ G G － v ＋ ＋ ＋ u ＋ ＋ x － ＋ G ＋ w ＋ Input vector i ART2 Neural Network (1) CART

17. ART2 Neural Network (2) • The activities on each of the six sublayers on F 1: where I is an input pattern where where the J th node on F 2 is the winner

18. ART2 Neural Network (3) • Initial weights: • Top-down weights: • Bottom-up weights: • Parameters:

19. v1 v2 vi vn Output layer (Competitive layer) i Excitatory connection wij xj j Input layer HAM Neural Network (1) CHAM

20. HAM Neural Network (2) • The input to neuron nidue to input stimuli x: nc: the winner after the competition

21. Road Sign Recognition System

22. Objective • Get information about road • Warn drivers • Enhance traffic safety • Support other subsystems

23. Problems

24. Perceptual Component

25. Conceptual Component— Classification results of CART Training Set Test Set

26. Conceptual Component— Training Patterns for CHAM

27. Experimental Results

30. Other Examples

31. Discussion • Vehicle and camcorder vibration • Incorrect recognitions Input patterns Recognition results Correct patterns

32. System to Detect Changes in Driving Environments

33. Definition • The environmental changes in expressways: • Left-lane-change • Right-lane-change • Tunnel-entry • Tunnel-exit • Expressway-entry • Expressway-exit • Overpass-ahead

34. Objective • Coordinate DAS subsystems • Update parameters • Detect unexpected changes • Detect rapid changes

35. Results of the Sensory Component

36. Results of the Perceptual Component

37. The Prototypical Attention Patterns

38. Experimental Results

39. Experimental Results

40. Experimental Results

41. Experimental Results

42. Experimental Results

43. Discussion • Curved roads • Shadows • Multiple environmental changes

44. System to Detect Motion of Nearby Moving Vehicles

45. Introduction • Motions of the Vehicles • Lane change • Speed change • Objective • Simple motion detection • Complex motion detection

46. Simple Motion Patterns

47. Improved DVM • Two problems: • The motions of vehicles may occur anywhere on the road. • Training a CART neural network to recognize various complex motions is quite difficult. • Solutions: • Feature extraction • Attention map partition • Collection of classification results • Temporal integral process

48. Attention maps Windowing b1 b2 bn-1 bn Feature extraction Feature extraction Feature extraction Feature extraction CART1 CART2 CARTn-1 CARTn st1 st2 stn-1 stn Decision making No Confirm? Yes Output Flowchart for Conceptual Component

49. b5 b4 b1 b2 b3 Attention Map Partition

50. 1 2 3 4 5 6 7 8 9 10 gi1 i i gi1 1 2 3 4 5 6 7 8 9 10 Feature Extraction (1) ---Skewness features