Building an Artificial Brain For Less Than $10,000 Prof. Dr. Hugo de GARIS Head of Artificial Intelligence Group, Inter

1. Building an Artificial Brain For Less Than $10,000 Prof. Dr. Hugo de GARIS Head of Artificial Intelligence Group, International School of Software, Wuhan University, Wuhan, Hubei Province, CHINA profhugodegaris@yahoo.com http://www.iss.whu.edu.cn/degaris

2. What is an Artificial Brain? Definition : A set of 10,000 – 20,000 evolved neural net circuit modules that are interconnected according to the designs of human “BAs” (Brain Architects). Brain Building Course : Prof de Garis has taught a PhD level course on “Brain Building” at his American university (Utah State) over the period 2001-2006, and will continue to teach this course at his new professorial job in China.

3. Neural net circuit modules can be evolved conventionally on a PC • but this can typically take hours to a day or more per module. • Obviously an artificial brain (A-Brain) consisting of several • 10,000s of evolved neural net (NN) modules will take too long to • build using only a PC to evolve them. • Hence the need for a neural net module evolution ACCELERATOR. • There are two major approaches to accelerating NN evolution – • The Software Approach • The Hardware Approach

4. The Software Approach • The typical algorithm for evolving a NN is the Genetic Algorithm • (GA), but a GA can be very inefficient, because its “genetic • operators” e.g. mutation, crossover are random, and hence most • of the time they produce wasted effort. • There have been new attempts lately in GAs to generate the next • generation more efficiently, using statistical or “machine learning” • techniques, • e.g. • EDA (Estimation of Distribution Algorithms) • LEM (Learnable Evolution Method), etc

5. b) The Hardware Approach Perform the GA based evolution of a NN module in special hardware that speeds the evolution by a factor of 10s to 100s of times compared to using software in a PC. e.g. use a Celoxica electronic board (www.celoxica.com) that contains a Xilinx FPGA with (today) 6 mega-gates. This FPGA is programmed using a “C-like” high level language called HANDEL-C (Handel the composer, not Handle, as in handbag). Handel-C code is “hardware compiled” directly into the FPGA on the Celoxica board.

7. The GA based evolution of a neural net (with a given fitness definition, i.e. a performance measure) is programmed in Handel-C and executed on the Celoxica board. The best evolved NN on the Celoxica board is downloaded into the PC’s memory. The 10s – 100s of times speed up factor is CRITICAL. It makes brain building PRACTICAL and CHEAP. The Celoxica board (6 megagate FPGA) ~ $7000 A robot to be controlled by the A-Brain ~$1000 So the total cost is less than $10,000

8. How to Connect the NNs into a Useful A-Brain? Answering this question is one of the major research challenges of Brain Building. This “Brain Building strategy for less than $10,000” is aimed at creating a methodology for creating “Artificial Brains”, that any Artificial Intelligence (AI) research lab can afford. Hopefully, a whole community of Artificial Brain researchers can now arise, which will share its ideas on how to “architect” A-Brains, leading to workshops, conferences, and journals.

9. A current alternative, is to use expensive PC clusters, costing large amounts of money, that only rich labs and companies can afford. e.g. a) Switzerland’s (EPFL) and IBM’s “Blue Brain Project” that aims to simulate a cortical column at the molecular level, and scale up to the full cortex over 10-15 years, using IBM’s “Blue Gene” supercomputer (that consists of a large PC cluster). b) Artificial Development (AD)’s “CCortex Project” with its 1000 processor PC cluster. The Celoxica evolved NN route offers a much cheaper alternative.

10. Individual NN Modules • There are various categories of NN modules that can be evolves – • Pattern recognition modules • Motion controller modules • Decision making modules • Timing modules • Memory modules etc • The art of being a Brain Builder (or Brain Architect) is to conceive • how to evolve these individual modules, and how to connect • them together to build Artificial Brains (A-Brains).

11. A Concrete Example of the Evolution of a NN Module – Imagine a grid of receptors whose outputs connect to a NN as external inputs. If light falls on a grid pixel, a strong output signal is sent to the connecting neuron in the NN. If no light falls on the pixel, a weak output signal is sent. The output signal of the NN is used to evolve the fitness. A “multi-task” evolutionary approach is typically used for the evolution of pattern detectors.

12. Assume we have a pattern on the grid of pixels we want to detect. Call this pattern P, the “positive example”. Let Q be a pattern similar to P. Call it the “negative example”. (There can be many positive and negative examples). The example P is radiated onto the grid for e.g. 100 ticks of the clock, and then pattern Q for a further 100 ticks. The target (desired) output of the output neuron of the NN is to be high if the pattern P is shown, and low if pattern Q is shown.

13. A suitable fitness definition for such a “P Detecting” NN could be - fitness = 1/(St=1to100 (Tt – At)2 + St=101to200 (Tt – At)2 ) where At is the actual output signal at tick t, and Tt is the desired or target signal at tick t. At = 0.8 (high) or 0.2 (low) Brain Architects soon develop considerable creativity in designing many individual NN modules. For details, see the PowerPoint notes of Prof. Hugo de Garis’s PhD course (CS7940) on Brain Building at http://www.iss.whu.edu.cn/degaris

14. The BAs (Brain Architects) evolve large numbers of NN modules (10,000-20,000 of them), and download them one by one from the Celoxica board into the PC’s memory. The BAs then need to specify how the NN modules connect up to form the A-Brain. To do this, special software in the PC, called “IMSI” (Inter Module Signaling Interface) is used. This software is essentially a set of look up tables (LUTs) that tell the PC which NN module is connected to which other NN module. These LUTs are used for a given module to find the external signals from other NN modules.

15. The IMSI also uses the downloaded evolved weights Wij of • each module to calculate the neural signals of each module. • Suggested Tasks of A-Brains • Control of Autonomous Robots • Visual Processing • Speech Processing • etc • The immediate task we have chosen is to use a 4 wheeled • robot with a CCD camera, a gripper, and a 2-way radio antenna • to send/receive signals from the antenna at the PC.

17. The A-Brain is contained in the PC. It sends control signals via its radio antenna to the radio antenna on the robot, to control its actions. The task of the robot is to detect with its camera eye, UXO (UneXploded Ordnance), i.e. small cluster bomblets. It approaches them, picks them up and deposits them in some central place, so that friendly troops can march through an area cleared of UXO. There will be many A-Brain applications.

18. Disadvantages of this Approach – a) Undesired Inter-Modular Synergy The modules are evolved individually, and then assembled into networks of networks. Therefore it is possible that undesirable “synergistic signaling” effects could happen. What to do? If adding a new module to an existing architecture causes undesirable side effects, then one could re-evolve another module.

19. b) No Multi-Modular Evolution Evolving 10,000-20,000 modules can be done with a team of BAs, but Moore’s Law will make A-Brains possible in PCs with 100,000 modules. Obviously multi-module evolution needs to be automated. This is a new research challenge that will have to be faced in the next few years. But first, A-Brains of 10,000-20,000 modules need to be built to show the approach is feasible, and that useful A-Brains can be architected and built, in a reasonable time.

20. Why 10,000 – 20,000 modules in a PC? Experiments have shown that an ordinary PC can update the signaling of all the simulated neurons in an A-Brain at the rate of at least 25 Hz (signals per second per neuron), if there are no more than 10,000-20,000 modules, with roughly 20 neurons per module. As the processing speed doubles, double the number of modules can be placed in the A-Brain. Even an A-Brain of 10,000 NN modules can be quite sophisticated, with hundreds of behaviors, and hundreds of pattern recognizers, etc.

21. By the year 2015, only 9 years away, i.e. 6 Moore’s Law doublings, would allow an A-Brain of over half a million modules to be built. This is a huge effort, and would require a large team of BAs. e.g. if one BA can design and evolve 1 NN module in an hour then how many BAs would be needed to build an A-Brain of 500,000 modules in 5 years? Under reasonable assumptions (e.g. 8 hour working days etc) The answer is about 50 people, i.e. the scale that a large private company could afford. With $100,000 salaries, that’s a salary budget of $25 million.

22. National Brain Projects By the year 2021, i.e. another 4 Moore doublings, will mean about 10 million modules. This implies about 1000 people for a 5 year project, and a salary budget of about $1 Billion. This is the size of a national (government) project. Prof de GARIS, intends spearheading China’s national Brain Building project, to build the “C-Brain”, and challenges other major brain building countries/regions to do the same, e.g. A-Brain (America’s Brain), E-Brain (Europe’s Brain) J-Brain (Japan’s Brain), I-Brain (India’s Brain), etc.