340 likes | 718 Views
Business School Institute of Business Informatics. Artificial Neural Networks - Basics. Uwe Lämmel. www.wi.hs-wismar.de/~laemmel U.laemmel@wi.hs-wismar.de. Neural Networks. Idea Artificial Neuron & Network Supervised Learning Unsupervised Learning Data Mining – other Techniques. Idea
E N D
Business SchoolInstitute of Business Informatics ArtificialNeural Networks - Basics Uwe Lämmel www.wi.hs-wismar.de/~laemmel U.laemmel@wi.hs-wismar.de
Neural Networks • Idea • Artificial Neuron & Network • Supervised Learning • Unsupervised Learning • Data Mining – other Techniques
Idea An artificial Neuron Neural Network Example Learning Application Artificial Neuron & Network
Idea • A human being learns by example “learning by doing” • seeing(Perception), Walking, Speaking,… • Can a machine do the same? • A human being uses his brain. A brain consists of millions of single cells.A cell is connected with ten thousands of other cell. • Is it possible to simulate a similar structure on a computer?
Idea Artificial Neural Network • Information processing similar to processes in a mammal brain • heavy parallel systems, • able to learn • great number of simple cells ? Is it useful to copy nature ? • wheel, aeroplane, ...
Idea An artificial neural network functions in a similar way a natural neural network does. • we need: • software neurons • software connections between neurons • software learning algorithms
Dendrits cell and nucleus Axon (Neurit) Synapsis A biological Neuron • Dendrits: (Input) Getting other activations • Axon: (Output ) forward the activation (from 1mm up to 1m long) • Synapse: transfer of activation: • to other cells, e.g. Dendrits of other neurons • a cell has about 1.000 to 10.000 connections to other cells • Cell Nucleus: (processing) evaluation of activation
w1i w2i ... wji Dendrits cell and nucleus Axon oi (Neurit) Synapsis Natural vs. Artificial Neuron
Abstraction • Dendrits: weighted (real number) connections • Axon: output: real number • Synapse: --- (identity: output is directly forwarded) • Cell nucleus: unit contains simple functions input = (many) real numbers processing = activation function output = real number (~activation)
w1i w2i ... wji oi An artificial Neuron net : input from the network w : weight of a connection act : activation fact : activation function :bias/threshold fout : output function (mostly ID) o : output
Exercise: AND/OR -LTU • Built a „network“ that works like an AND-function • Built an OR-network • Try to built an XOR-network LTU – Linear Threshold Unit
a1=__ a2=__ w1=__ w2=__ net = o1w1+o2 w2 a = 1, if net > = 0, otherwise o = a o=__ A simple switch • Neuron = AND-function • Find parameters: • Input neurons 1,2 :a1,a2 input pattern, • weights of edges: w1, w2 • bias • Now evaluate output o !
Mathematics in a Cell • Propagation function (neuron input)neti(t) = ojwj = w1i o1 + w2i o2 + ... • Activationai(t) – Activation at time t • Activation functionfact :ai(t+1) = fact(ai(t), neti(t), i) i – bias • Output functionfout :oi = fout(ai)
Bias function 1,0 0,5 0,0 -4,0 -2,0 0,0 2,0 4,0 -0,5 Identity -1,0 4,0 2,0 0,0 -4,0 -2,5 -1,0 0,5 2,0 3,5 -2,0 -4,0 Activation Functions activation functionsaresigmoid functions
y = tanh(c·x) c=3 1,0 c=2 c=1 0,5 y = 1/(1+exp(-c·x)) -0,6 0,6 1,0 -1,0 c=10 1,0 -0,5 c=3 c=1 -1,0 0,5 1,0 0,0 -1,0 Activation Functions activation functionsaresigmoid functions Logistic function:
Structure of a network • layers • input layer – input neurons • output layer – output neurons • hidden layer – hidden neurons • An n-layer network has: • n layer of connections which can be trained
Definition: A Neural Network … • … is characterized by • connections of many (a lot of) simple units (neurons) and • units exchanging signals via these connections • … is a • coherent, directed graph which has • weighted edges and • each node (neuron, unit ) contains a value (activation).
XOR-network input output • weights are set by hand • fact = 1, net > = 0, otherwise • = 1.0
XOR-Example • standard propagation function:neti(t) = oj(t)wji • activation function = bias function • ai = 1, if neti(t)> i e.g. =0.5 0, otherwise • output function = Identity: oj = aj • use EXCEL and built the XOR-example network!
Network evaluation network error learning examples changingnetwork parameters Learning
Learning - can be done by: • Modification of the weight of a connection • most frequently used • Deleting connections • can be done by (1): w=0 w0 • Modification of the bias of a neuron • Can be done by (1) using an extra neuron • changing functions (activation, propagation, output function) • Building new cells (GNG) • Building new connections • Deleting cells
Learning • supervised learning • We know the results for certain input pattern:teaching input teaching output • Network error is used to adapt weights • Fast, but not natural • reinforced learning • We know whether output is right or wrong • Information is used to adapt weights • Slower than supervised; natural • unsupervised learning • Network has to learn by itself; • Slow , natural
Applications • Pattern recognition (text, numbers, faces) • Checking the quality of a surface • Control of autonomous vehicles • Monitoring of credit card accounts • Data Mining
Applications • Speech recognition • Control of artificial limbs • classification of galaxies • Product orders (Supermarket) • Forecast of energy consumption • Stock value forecast
Application - Summary • Classification • Clustering • Forecast • Pattern recognition • Learning by examples, generalization • Recognition of not known structures