SELF-ORGANIZING LEARNING ARRAY (SOLAR) AND ITS APPLICATION TO ECONOMIC AND FINANCIAL PROBLEMS

1. SELF-ORGANIZING LEARNING ARRAY(SOLAR)AND ITS APPLICATION TO ECONOMIC AND FINANCIAL PROBLEMS by Janusz Starzyk, Zhen Zhu, Haibo He and Zhineng Zhu School of EECS Ohio University, Athens, OH 3rd International Workshop on Computational Intelligence in Economics and Finance Cary, NC, September 30th, 2003

2. OUTLINE • Motivation • SOLAR and its organization • Self-organizing principle • Evolution of SOLAR structure • Data processing • Examples of economic and financial applications • Future work and conclusion

3. SELF-ORGANIZING LEARNING ARRAYWhat is SOLAR? • New Biologically Inspired Network Organization Basic Fabric: A fixed lattice of distributed, parallel processing units (neurons) Self-organization Algorithm: • Interconnections among neurons are dynamically refined. • Neurons are dynamically re-configured. • Number of neurons used is decided by problem complexity.

4. Why do we need SOLAR? • Needed a general purpose learning network • Network that learns without algorithm • Network that runs without software • Network that is data driven • Network that self-organizes • Network that learns through associations • Network that acts with self awareness • Network that scales to a very large system • Network that is fault tolerant • Network that is modular

5. SOLAR-Organization • Neurons organized in a cell array • Sparse randomized connections • Local self-organization • Data driven • Entropy based learning • Regular structure • Suitable for large scale circuit implementation

6. Structure of a single neuron • RPU:reconfigurable processing unit • CU: control unit • DPE: dynamic probability estimator • EBE: entropy based evaluator • DSRU: dynamic self-reconfiguration memory. • NI/NO: Data input/output • CI/CO: Control input/output

7. Self-organizing connections Each neuron is pseudorandomly connected to other neurons or primary inputs. O: processing unit : data : control

8. Learned Structure Using a entropy-based metric, some of the connections would be found carrying more information and thus saved while other cut out.

9. Self-organizing Principle Information deficiency Information index If the space was divided, then the information index and information deficiencies are related as When subsequent space partitions take place, the information deficiency is expressed as a product of information deficiencies in each subsequent partition.

10. Neuron and its I/O Ports • A priori information – class probabilities and input information deficiencies Transformation functions with thresholds, cut input space into two sub-spaces. Neuron self organize to maximize information deficiency reduction: Learning stops when:

11. Neuron’s action • Through learning, one pair of function and threshold will be chosen and solidified. • This will provide a cut in the input space to separate different classes.

12. Neuron’s action Other neurons can provide other cuts. Most confident neurons vote on final classification.

13. SOLAR Behavior • Behavior of a single neuron: • Calculates its data and control outputs and provides them as inputs to others. • Computes statistical information (for example, entropy based information deficiency) in its subspaces. • Makes associations with other neurons. • Behavior of the network: • Clusters of neurons solve the problem. • Network connections are active elements of learning. • The system gathers information from all neurons and makes final decision based on well trained neurons

14. SOLAR-Data Processing • SOLAR receives input data as a 2-D matrix of feature samples. • This matrix is pre-processed to make sure: • Missing parts are recovered • Symbolic data is represented in a proper way • Data into each neuron is re-scaled for full resolution

15. Training: Each neuron selects most efficient inputs, function, threshold and solidify them. Testing: Neurons calculate outputs and pass them to others. F1=f1(x,y) F2=f2(x,y) F3=f3(x,y) F4=f4(x,y) F5=f5(x,y) Decision Making SOLAR-Data Processing Pre-processing input data Initialize SOLAR Testing: Neurons calculate outputs and pass them on. Neurons work in parallel System collects information and makes final decision

16. SOLAR-Data Processing • We may use an ensemble of SOLAR networks to vote on the same target:

17. Biologically Inspired NN

18. SOLAR-Examples • Bankruptcy Prediction • Amir F.Atiya, “Bankruptcy Prediction for Credit Risk Using Neural Networks: A survey and New Results,” IEEE Trans. on Neural Networks, Vol. 12, No. 4, July, 2001. • Credit Card Approval Decision • Loan Decision-Australian Adult Income Classification • D. Michie, D. J. Spiegelhalter, and C. C. Taylor, “Machine Learning, Neural and Statistical Classification” London, U. K. Ellis Horwood Ltd. 1994. • ftp at cs.uci.edu (128.195.1.1)

19. SOLAR-Example • The Bankruptcy Dataset: • 716 solvent US corporations • 195 defaulted ones (within 1 to 36 months) • Expanded into 1160 points by taking different instances before the default • Over 60 available indicators (features) • Financial Ratio and Equity-based Indicator System: • Based on financial ratios and prices • Proved superior to traditional indicators • Optimal indicator pool based on traditional ANNs

20. Time to default Reported correct rate % correct rate % of SOLAR correct rate % of SOLAR using all 6 month or less 86.15 85.11 87.23 6 to 12 months 81.48 84.09 86.36 12 to 18 months 74.60 76.19 90.24 18 to 24 months 78.13 55.17 72.24 more than 24 months 66.67 64.29 75.00 total defaulted 78.13 75.13 83.96 solvent 90.07 92.74 93.42 total 85.50 85.80 90.04 SOLAR-Example Bankruptcy prediction using SOLAR

21. SOLAR-Example • Using the same “Financial Ratio and Equity-based Indicator System”, SOLAR shows equal performance to traditional ANNS. • SOLAR is capable of handling all indicators and gives even better performance.

22. SOLAR-Exampleprewired structure

23. SOLAR-Examplelearned connections

24. SOLAR-Example Detail of prewired structure Detail of learned connections

25. SOLAR-Examples • Credit Card Approval Decision and Loan Decision • Benchmark problems used to compare performance various methods in literature • learning algorithms, neural networks, statistical methods and SOLAR are compared

26. Algorithm Mis-prob. Algorithm Mis-prob. CAL5 0.131 CART 0.145 SOLAR 0.135 RBF 0.145 Itule 0.137 CASTLE 0.148 DIPOL92 0.141 Naivebay 0.151 Logdisc 0.141 IndCART 0.152 Discrim 0.141 Bprop 0.154 SOLAR-Example Performance Comparison on Credit Card Approval Decision

27. Algorithm Mis-prob. Algorithm Mis-prob. FSS Naive Bayes 0.1405 CN2 0.1600 NBTrees 0.1410 Naive-Bayes 0.1612 C4.5-auto 0.1446 Voted ID3 (0.8) 0.1647 IDTM(Decision table) 0.1446 T2 0.1687 HOODG/SOLAR 0.1482 1R 0.1954 C4.5 rules 0.1494 Nearest-Neighbor (3) 0.2035 OC1 0.1504 Nearest-Neighbor (1) 0.2142 C4.5 0.1554 Pebls Crashed Voted ID3 (0.6) 0.1564 SOLAR-Example Performance Comparison on Loan Decision

28. Conclusion • SOLAR is a new biologically inspired learning network organization. • Efficient, general purpose, capable of large data sets, suitable for hardware implementation • Used to solve financial and economic problems, prediction, identification, and decision making.

29. Future work • Applications to other fields • Hardware implementation in FPGA • real time applications • modular and expandable structures • Associative learning • Temporal learning • Stability issues • Autonomous, goal driven behavior

30. Questions ?