1 / 27

Spiking-Timing-Dependent Learning in Memristive Device

Spiking-Timing-Dependent Learning in Memristive Device. Yi Kaijun DSI Jan 12 2011. Background. Current CMOS scaling problem: tunneling effect, defects, faults, failures, variability, drift, etc.

tyra
Download Presentation

Spiking-Timing-Dependent Learning in Memristive Device

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spiking-Timing-Dependent Learning in Memristive Device Yi Kaijun DSI Jan 12 2011

  2. Background • Current CMOS scaling problem: tunneling effect, defects, faults, failures, variability, drift, etc. • Adaptive, neuromorphic circuits and system (intelligent, low power consumption, error tolerance, etc.)

  3. Neuromorphic Device • Very-large-scale integration (VLSI) systems containing electronic analog circuits that mimic neuro-biological architectures present in the nervous system. • Both analog, digital or mixed-mode analog/digital VLSI systems that implement models of neural systems (for perception, motor control, sensory processing, learning, etc.) as well as software algorithms.

  4. Hebbian Learning • If two neurons on either side of synapse (connection) are activated simultaneously, then the strength of that synapse is selectively increased. • If two neurons on either side of a synapse are activated asynchronously, then that synapse is selectively weakened or eliminated.

  5. Spiking Timing Dependence Spiking • Spike-timing-dependent plasticity (STDP) is a biological process that adjusts the strength of connections between neurons in the brain. The process adjusts the connection strengths based on the relative timing of a particular neuron's output and input action potentials (or spikes). • Timing-based learning law

  6. Spiking Timing Dependence Spiking

  7. STDP with CMOS J. V. Arthur and K. Boahen, “Learning in Silicon : Timing is Everything”, NIPS, 2006

  8. Memristor in a nutshell • A two-terminal device • Resistance state (RHIGH/RLOW) depends on the voltage applied across • Resistance state remains after the removal of the electrical stimulus • Reversible switching of resistance states RLOW RHIGH

  9. Memristor – Device realization • Resistance change (∆R) as a result of morphological change of materials • Can be initiated by the electrode or within the insulator • Global vs. local • Semi-permanent, i.e. R remains after removal of electrical bias • Reversible metal electrode resistive layer conductive filament conductive layer

  10. Memristor – Two basic operation schemes1 • Unipolar: • Switching depends on amplitude of the applied voltage but not the polarity • VSET always higher than VRESET • Bipolar • Directional switching depending on the polarity of the applied voltage • VSET and VRESET at opposite polarities Unipolar and bipolar switching.2 CC denotes the compliance current, often needed to limit the ON current. 1Waser, R. et al., Adv. Mater., 21, 2632 (2009). 2Waser, R. , Microelectron. Eng., 86, 1925 (2009).

  11. STDP with Memristor

  12. Neurons and Synapse CMOS Memristor

  13. STDP Device

  14. Neuromorphic Network

  15. Key Ideas • Define two synaptic state variables for pre- and post-synaptic neurons • To separate computational communication from learning by time-division multiplexing (TDM) of pulse-width modulation (PWM) signal through synapses

  16. Two State Parameters • Long Term Potentiation (LTP): a long-lasting enhancement in signal transmission between two neurons that results from stimulating them synchronously. • Long Term Depression (LTD): an activity-dependent reduction in the efficacy of neuronal synapses lasting hours or longer.

  17. TDM and PWM • TDM: a type of digital or (rarely) analog multiplexing in which two or more signals or bit streams are transferred apparently simultaneously as sub-channels in one communication channel, but are physically taking turns on the channel. • PWM: A technique to provide logic “1” and logic “0” for a controlled period of time. The information is encoded as pulse duration.

  18. Time-division Multiplexing of Signals through Synapses

  19. Signal Definition • COMM (Time Slot 0): communication of spikes from pre to post in order to compute inner products or matrix products. • LTP+/- (Time Slot 1/2): communicate LTP timing information from Pre to post with PWM encoding. • LTD+/- (Time Slot 3/4): communicate LTD timing information from post to pre with PWM.

  20. Pulse-width Modulation within a timeslot

  21. Transmission of LTD and LTP Timing Curves Through Synapses

  22. Post Spiking After Pre Spiking Induces LTP

  23. Pre Spiking After Spiking Induces LTD

  24. Block Diagram of A Neuron

  25. A MemristorNanodevice at HP

  26. Advantages • Better control over power consumption • Fewer constraints on memristor used for synapses • Greater freedom in learning algorithm • Greater control over precise timing control for STDP learning • Adaptable for other learning law • Better circuit diversity

  27. Drawbacks • Neuron element is too complicated to be integrated in large scales. • This is only the concept, more realistic protocols and concrete devices should be developed. • No demonstration is provided.

More Related