1 / 27

NEURAL NETWORK THEORY

NEURAL NETWORK THEORY. NEURAL DYNAMIC1: ACTIVATIONHS AND SIGNALS. MAIN POINTS:. NEURONS AS FUNCTIONS( 神经元函数 ) SIGNAL MONOTONICITY （信号单调性） BIOLOGICAL ACTIVATIONS AND SIGNALS （生物激励与信号） NEURON FIELDS （神经域） NEURONAL DYNAMICAL SYSTEMS （神经诊断系统） COMMON SIGNAL FUNCTION （一般信号方程）

gabriel-cardenas
Download Presentation

NEURAL NETWORK THEORY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NEURAL NETWORK THEORY NEURAL DYNAMIC1: ACTIVATIONHS AND SIGNALS

  2. MAIN POINTS: • NEURONS AS FUNCTIONS(神经元函数) • SIGNAL MONOTONICITY（信号单调性） • BIOLOGICAL ACTIVATIONS AND SIGNALS（生物激励与信号） • NEURON FIELDS（神经域） • NEURONAL DYNAMICAL SYSTEMS（神经诊断系统） • COMMON SIGNAL FUNCTION（一般信号方程） • PULSE-CODED SIGNAL FUNCTION（脉冲编码信号方程）

  3. NEURONS AS FUNCTION , Figure 1. Neuron Structure Model Relationship of input-output:

  4. NEURONS AS FUNCTION • Common nonlinear transduction description: a sigmoidal or S-shaped curve Fig.2 s(x) is a bounded monotone-nondecreasing function of x Signal Function: Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t)).

  5. SIGNAL MONOTONICITY • In general, signal functions are monotone nondecreasing S’>=0. In practice this means signal functions have an upper bound or saturation value. • An important exception: bell-shaped signal function or Gaussian signal functions The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise.

  6. SIGNAL MONOTONICITY • Generalized Gaussian signal function define potential or radial basis function: input activation vector: : variance: mean vector: Because the function depend on all neuronal activations not just the ith activation, we shall consider only scalar-input signal functions:

  7. SIGNAL MONOTONICITY • A property of signal monotonicity: semi-linearity • Comparation: • Linear signal functions: • computation and analysis is comparatively easy; do not suppress noise. b. Nonlinear signal functions: Increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;

  8. SIGNAL MONOTONICITY • Signal and activation velocities the signal velocity: =dS/dt Signal velocities depend explicitly on action velocities. This dependence will increase the number of unsupervised learning laws.

  9. BIOLOGICAL ACTIVATIONS AND SIGNALS • Introduction to units ： Dendrite: input Axon: output Synapse: transduce signal Membrane: potential difference between inside and outside of neuron Fig3. Key functional units of a biological neuron

  10. BIOLOGICAL ACTIVATIONS AND SIGNALS • Competitive Neuronal Signal Signal values are usually binary and bipolar. Binary signal functions : Bipolar signal functions :

  11. NEURON FIELDS In general, neural networks contain many fields of neurons. Neurons within a field are topological. Denotation: : input field : output field Neural system samples the function m times to generate the associated pairs • Classification: Zeroth-order topological (simplest) Three-dimensional and volume topological (complex)

  12. NEURONAL DYNAMICAL SYSTEMS • Description: A system of first-order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials Activation differential equations: denote the activation time functions of the ith neuron in and jth neuron in • Classification: Automomous systems: activations are independent of t Nonautonomous systems: depend on t

  13. NEURONAL DYNAMICAL SYSTEMS • Neuronal State spaces So the state space of the entire neuronal dynamical system is: Augmentation : Concatenate fields have different computational, metrical or other properties

  14. NEURONAL DYNAMICAL SYSTEMS Signal state spaces as hypercubes Fig.4 Neural and fuzzy computations conincide.

  15. NEURONAL DYNAMICAL SYSTEMS • Neuronal activations as short-term memory Short-term memory(STM) : activation Long-term memory(LTM) : synapse

  16. S k x o COMMON SIGNAL FUNCTION 1、Liner Function S(x) = cx + k , c>0

  17. S r -θ θ x -r COMMON SIGNAL FUNCTION 2. Ramp Function r if x≥θ S(x)= cx if |x|<θ -r if x≤-θ r>0, r is a constant.

  18. COMMON SIGNAL FUNCTION 3、threshold linear signal function: a special Ramp Function Another form:

  19. COMMON SIGNAL FUNCTION 4、logistic signal function: Where c>0. So the logistic signal function is monotone increasing.

  20. COMMON SIGNAL FUNCTION 5、threshold signal function: Where T is an arbitrary real-valued threshold,and k indicates the discrete time step.

  21. COMMON SIGNAL FUNCTION 6、hyperbolic-tangent signal function: Another form:

  22. COMMON SIGNAL FUNCTION 7、threshold exponential signal function: When

  23. COMMON SIGNAL FUNCTION 8、exponential-distribution signal function: When

  24. COMMON SIGNAL FUNCTION 9、the family of ratio-polynomial signal function: An example For

  25. PULSE-CODED SIGNAL FUNCTION • Description: • Pulse trains arriving in a sampling interval seems to be the bearer of neuronal signal information. Pulse-coded formulation: where denote binary pulse functions that summarize the excitation of membrane potential.

  26. PULSE-CODED SIGNAL FUNCTION • Velocity-difference property of pulse-coded signals A simple form for the signal velocity: Current pulse and current signal or expected pulse frequency are available quantities. Another computational advantage: If

  27. 欢迎大家提出意见建议！ 常虹 2006.9.25

More Related