HIGH PERFORMANCE MULTILAYER PERCEPTRON ON A CUSTOM COMPUTING MACHINE

1. HIGH PERFORMANCE MULTILAYER PERCEPTRON ON A CUSTOM COMPUTING MACHINE Authors: Nalini K. Ratha, Anil K. Jain H. GÜL ÇALIKLI 2002700743

2. INTRODUCTION • Artificial Neural Networks(ANNs)attempt to mimic biological neural networks. • One of the main features of biological neural networks is the massively parallel interconnections among the neurons.

3. INTRODUCTION • Computational model of a biological neural network: • simple operations such as: • inner product computation • thresholding • design parameters • 1.network topology: • i.number of layers • ii.number of nodes in a layer • 2.connection weights • 3.property at a node; e.g.type of non-linearity to be used.

4. INTRODUCTION Schematic of a Perceptron d-dimensional input vector X=(x1,x2,...,xd) weight vector W =(w1,w2,w3,....,wd) “Output y” used to determine the category of input x1 w1 ∑ wi .xi ∑ y w2 Non-Linearity x2 w3 x3 wd İnner product of input vector X and weight vector W .... xd

5. INTRODUCTION: • Multilayer Perceptrons: (MLPs) • one of the most popular neural network models for solving pattern classification and image classification problems • consists of several layer of perceptrons • nodes in the i layer are connected to nodes in the (i+1) layer through suitable weights • no interconnection among the nodes in a layer. th th

6. INTRODUCTION:Multilayer Perceptrons Multilayer Perceptron Biological neuron

7. INTRODUCTION: • Training of a MLP: • 1.Feedforward Stage: • training patterns with known class labels are presented at the input layer • at the start weight matrix is randomly initialised • the output is computed at the output node. • 2.Weight Update Stage: • weights are updated in a backward fashion starting with the output layer • weights are changed proportional to the error between the desired output and the actual output. • 3.Repeat 1 and 2 until the network converges

8. INTRODUCTION: A Multilayer Perceptron x1 x2 x3 . . . . . . . . Feature Vector . . . xd hidden layer output layer input layer

9. INTRODUCTION: • For an n-node MLP, O(n2) interconnections are needed. • THUS: • mapping a MLP onto a parallel processor is a real challenge. • on a uniprocessor, the whole operation proceeds sequentially one node at a time. (no complex communications involved) • HOWEVER: • for a high performance implementation, efficient communication capability must be supported.

10. INTRODUCTION: • Typical Pattern Recognition and Computer Vision Applications: • applications have >100 input nodes • classification process involving complex decision boundaries demands a large number of hidden nodes

11. INTRODUCTION: • Real Time Computer Vision Applications: • The network training can be carried out offline. • Recall Phase:High input/output bandwidth is required along with fast classification (recall) speeds.

12. INTRODUCTION: • For a three layer network (excluding the input layer),let : • m:input nodes • n1:nodes in the first hidden layer • n2:nodes in the second hidden layer • k:output nodes (classes) • Nm: the number of multiplications • Na:the number of additions Nonlinearity not included • Nm=(m*n1)+(n1*n2)+(n2*k) • Na=Nm-(n1+n2+k)

13. INTRODUCTION: • Example- a Practical Vision System • Process a 1024 x 1024 image in “real time” • 30 frames to be processed per second • 30 x 1024 x 1024 =30 x 106 input patterns/sec • THUS, a real time neural network classifier is expected to perform billions of operations per second. • Connection weights are floating point numbers floating point multiplications and additions • Result:Throughputs of this kind are difficult to achieve with today’s most powerful uniprocessors.

14. INTRODUCTION: • Parallel Architectures for ANNs • Types of Parallelism Available in a MLP: • 1. Training session parallelism • 2. Training example parallelism • 3. Layer and forward/backward parallelism • 4. Node parallelism • 5. Weight parallelism • 6. Bit parallelism easily mapped onto a parallel architecture

15. INTRODUCTION: • Parallel Architectures for ANNs (cont’d) • Complexities involved: • 1.computational complexity • 2.communication complexityinner product computation involves a large number of communication steps. • THUS,special purpose neurocomputers have been built using; • 1.commercially available special purpose VLSIs • 2.special purpose VLSIs provides the best performance

16. INTRODUCTION: • Parallel Architectures for ANNs (cont’d) • Dynamically changing architecture • number of nodes • number of layers from application to application • Expensive to design a VLSI architecture for individual applications • Typically, architectures with a fixed number ofnodes and layers are fabricated.

17. INTRODUCTION: • Parallel Architectures for ANNs (cont’d) • Special Purpose ANN Implementations in the Literature • Ghosh and Hwang: • investigate architectural requirements for simulating ANNs using massively parallel multipocessors • propose a model for mapping neural networks onto message passing multicomputers.

18. INTRODUCTION: • Liu: • presents an efficient implementation of backpropagation algorithm on the CM-5 that avoids explicit message passing •  results of CM-5 implementation compared with those of Cray-2,CrayX-MP,and CrayY-MP • Chinn: •  describe a systolic algorithm for ANN on MasPar-1 using a 2-D Systolic Array-Based Design

19. INTRODUCTION: • Onuki: • present a parallel implementation using a set of sixteen standard 24 bit DSPs connected in a hypercube • Kirsanov: • discusses a new architecture for ANNs using Transputers • Muller: • presents a special purpose parallel computer using a large number of Motorola floating point processors for ANN implementation.

20. INTRODUCTION: • Parallel Architectures for ANNs (cont’d) • Special Purpose VLSI chips designed & fabricated for ANN implementations: • Hamerstorm: •  a high performance and low cost ANN with: • 64 processing nodes per chip • hardware based multiply and accumulator operators • Barber: • used a binary tree adder following parallel multipliers in SPIN-L architecture

21. INTRODUCTION: • Shinokawa: • describe a fast ANN with billion connections per second using ASIC VLSI chips • Viredez: • describes MANTRA-I neurocomputer using 2x2 systolic PE blocks • Kotolainen: •  proposed a tree of connection units with processing units at the leaf nodes for mapping many common ANNs.

22. INTRODUCTION: • Asanovic: •  proposed a VLIW of 128 bit instruction width and a 7 stage pipelined processor with 8 processors per chip. • Ramacher: •  describes the architecture of SYNAPSE • SYNAPSE:a systolic neural signal processorusing a 2D array of systolic elements • Mueller & Hammerstrom: •  describe design and implementation of CNAPS

23. INTRODUCTION: • CNAPS: • a gate array implementation of ANNs • a single CNAPS chip: • consists of 64 processing nodes • each node connected in a SIMD fashion • using broadcast interconnect. •  each processor has: • 4K bytes of local memory • a multipler • ALU • dual internal buses

24. INTRODUCTION: • Cox: • describes the implementation of GANGLION • GANGLION: •  a single neuron caters a fixed neural architecture of • 12 input nodes •  14 hidden nodes •  4 output nodes •  using CLBs 8x8 multipliers have been built •  a lookup table is used for the activation function.

25. INTRODUCTION: • Stochastic Neural Architectures: • There is no need for a time-consuming and area costly floating point multiplier. • Suitable for VLSI implementations • Examples: • Armstrong & Thomas: • proposed a variation of ANN called Adaptive Logic Network (ALNs) • ALNs: similiar to ANNs •  costly multiplications replaced by logical and operations •  additions replaced by logical or operations

26. INTRODUCTION: • Masa et. al.: • Describe an ANN, • with  a single output  six hidden layers  seventy inputs • can operate at 50 MHz input rate

27. CUSTOM COMPUTING MACHINES • Uniprocessor: • instruction set available to a programmer is fixed • an algorithm is coded using a sequence of instructions • processor can serve many applications by simply reordering the sequence of instructions • Application Specific Integrated Circuits (ASICs) • used for a specific application • provide higher performance compared to the general purpose uniprocessor

28. CUSTOM COMPUTING MACHINES • Custom Computing Machine (CCM) • a user can customize the architecture and instructions for a given application  programming at a gate level • by programming at a gate level, high performance can be achieved. • using a CCM, a designer can tune and match the architectural requirements of the problem

29. CUSTOM COMPUTING MACHINES • A CCM can overcome the limitations of ASICs • Limitations of ASICs • fast but costly • nonreconfigurable • time consuming • Advantages of CCMs • cheap: • CCMs use Field Programmable Gate Arrays(FPGAs) as compute elements • FPGAs are off-the-shelf components, thus relatively cheap.

30. CUSTOM COMPUTING MACHINES • Advantages of CCMs (cont’d) • reconfigurable: since FPGAs are reconfigurable, CCMs are easily reprogrammed. • time saving: • CCMs do not need to be fabricated with every new application since they are often employed for fast prototyping • THUS they save a considerable amount of time in design and implementation of algorithms.

31. SPLASH 2 – ARCHITECTURE and PROGRAMMING FLOW • Splash 2 is one of the leading FPGA basedcustom computing machine designed and developed by Supercomputing Research Center

32. SYSTEM LEVEL VIEW of the SPLASH 2 ARCHITECTURE interface board:1.connects Splash 2 to the host 2.Extends the address and data buses Processing Element (PE): Each PE has 512 KB of memory The host can read/write this memory PE X0:controls the data flow into the processor board PEs (X1-X16) Splash 2 Processing Board The Sun host can read/write to memories and memory mapped control registers of Splash 2 via these buses.

33. 512 K 16-bit Memory RD RD WR WR 32 18 16 Processing Element (PE) Xilinx 4010 32 36 36 36 SPLASH 2 – ARCHITECTURE and PROGRAMMING FLOW Processing Element in Splash 2 SBus Read SBus Write individual memory available with each PE makes it convenient to store temporary results and tables. Address Data SBus Address SBus Data Processor inhibit To left neighbor To right neighbor To crossbar

34. SPLASH 2 – ARCHITECTURE and PROGRAMMING FLOW Programming Flow for Splash 2 VHDL source Simulation Logic Synthesis (Gate level decription) Main concern: achieve the best placement of logic in an FPGA in order to minimize timing delay. İf timing obtained is not acceptable then design process is repeated. Logic designed using VHDL is verified. • To program Splash 2, we • need to program: • Each of the PEs • Crossbar • Host interface Partition, place and route (Logic placement) Timing of logic if the logic circuit can not be mapped to CLBs and flip flops which are available internal to an FPGA , then designer needs to revise the logic in the VHDL code and the process is repeated. Splash 2 Once the logic is mapped to CLBs, the timing for the entire digital logic is obtained.

35. SPLASH 2 – ARCHITECTURE and PROGRAMMING FLOW Steps in Software Development on Splash 2 Design Entry (VHDL) simulation Functional Verification Verified Design Partition, place and route Delay Analysis synthesis Generate Control Bİts Debugging Host interface improvement Integration Host-splash 2 Executable code

36. MAPPING an MLP on SPLASH 2 • In implementing a neural network classifier on Splash-2: building block  perceptron implementation • For mapping MLP to Splash-2 2 physical PEs serve as a neuron. • ith PE handles the inner product phase ∑wijxi • (i+1)thPE computes nonlinear function tanh(βx) with β=0.25 • where i is odd and (i+1) is even

37. MAPPING an MLP on SPLASH 2 • Assume:perceptrons have been trained  connection weights are fixed. • Thus, an efficient way of handling the multiplication is to employ a Look-up Table.Since a large external memory (512 KB),the lookup table can be stored. • A pattern vector component xiis presented at every clock cycle • 1. Inner Product Calculation: • The ith (odd) PE look up the multiplication table to obtain the weighted product

38. MAPPING an MLP on SPLASH 2 • The sum ∑wijxi is computed using an accumulator. • After all the components of a pattern vector have been examined, we have computed the inner product. • 2. Application of nonlinear function to the inner product • The nonlinearity is again stored as a lookup table in the second PE. • On receiving the inner product result from the first PE, the second uses the result as the address to the non-linearity look-up table and produces the output.

39. MAPPING an MLP on SPLASH 2 • 3. Thus the output of a neuron is obtained: • The output is written back to the external memory of the second PE starting from a prespecified location. • 4. After sending all the pattern vectors, the host can read back the memory contents. • A layer in the neural network is simply a collection of neurons working synchronously on the input. • On Splash-2 this can be achived bybroadcasting the input to as many physical PEs as desired.The output of a neuron is written into a specified segment of external memory and read back by the host.

40. MAPPING an MLP on SPLASH 2 • For every layer in MLP stages 1-4 is repated until the output layer is reached. • NOTE: For every layer, there is a different look-up table. • Look-up Table Organization: • There are m multiplications to be performed per node corresponding to the m-dimensional weight vector. • Look-upTable is divided into m segments.

41. Table m : 12 17 Block Offset 11 0 Table 1 MAPPING an MLP on SPLASH 2 • Look-up Table Organization (cont’d) • A counter is incremented at every clock which forms the higher order (block) address for the lookup table. • Note:The offset can also be negative correponding to a negative input to the look up table. • Pattern vector component • forms the lower order address bits. • Splash-2 has 18 bit adress bus for the external memory. • Higher order 6 bits for the block address • Lower order 12 bits for the offset address within the block.

42. MAPPING an MLP on SPLASH 2 • Look-up Table Organization (cont’d) • The numbers have been represented by 12 bits 2’s complement representation. Hence; • The resolution of this representation is eleven bits. • accumulator: • within PE • 16 bit wide • After accumulation, the accumulator result is scaled down to 12 bits.

43. MAPPING an MLP on SPLASH 2 Lookup Table Organization

44. PERFORMANCE EVALUATION • The requirements for mapping a MLP required to complete a classification process: • in terms of PEs required • number of PEs required is equal to twice the number of layers in each layer. • number of clock cycles required = m*K* l • where: • m: number of input layer nodes. • K: number of patterns • l :number of clock cycles

45. PERFORMANCE EVALUATION • in the implementation by authors of the paper: • m = 20 • K = 1024 X 1024 =1 MB (Total number of pixels in the input şmage) • l = 2 • THUS: • no. of clock-cycles = 20*2*106 = 40 million • with a clock rate of 22 MHz., time taken for 40 million clock ticks =1.81 secs.

46. PERFORMANCE EVALUATION • When the number of PEs required is larger than the available PEs; • either more processor boards need to be added • or PEs need to be time shared. • NOTE: •  neuron outputs are produced independent of other neurons •  algorithm waits till the computations in each layer is completed.

47. PERFORMANCE EVALUATION • A MLP has communication complexity of O(n2) where n is the number of nodes. • As n grows, it will be difficult to get good timing performance from a single processor system. • with a large number of processor boards, the single input data bus of 36 bits can cater to multiple input patterns. • Note: • In a multiboard system, all boards receive the same input. •  This parallelism can give rise to more data streaming into the system, •  thus the number of clock cycles is reduced.

48. PERFORMANCE EVALUATION SCALABILITY: only a single layer is considered network size is represented by the # of nodes in that layers multilayered networks are considered to be linearly scalable in Splash 2 architecture performance measure is processing time as measured by # of clock cycles for Splash 2 with 22 MHz. Clock. sparc20 Time (Log Scale) splash Size of network

49. PERFORMANCE EVALUATION • Speed Evaluation: • 20 input nodes implemented on a 2-board system • 176 million connections per second (MCPS) is achieved per layer by running the Splash clock at 22 MHz. • A 6- board system can deliver more than a billion connections per second. • Comparable to the performance of many high level VLSI-based systems such as Synapse, CNAPS which perform in the range of 5 GCPS.

50. PERFORMANCE EVALUATION • Network-based Image Segmentation: • Image Segmentation: The process of partitioning an image into mutually exclusive connected image regions. • In an automated image document understanding system, page layout segmentation plays an important role for segmentingtext, graphics and background areas. • Jain and Karu proposed an algorithm to learn texture discrimination masks needed for image segmentation.