Using Loops in Genetic Programming for A Two Class Binary Image Classification Problem

1. Using Loops in Genetic Programming for A Two Class Binary Image Classification Problem Xiang Li and Vic Ciesielski {xiali, vc}@cs.rmit.edu.au School of Computer Science and Information Technology RMIT University, Australia Using Loops in GP for A Two Class Binary Image Classification Problem

2. Overview • Introduction • Why do we experiment loops in image classification? • Our research questions • Related works • Syntax and Semantic of the Loops Used • Experimental Problem - Binary Image Classification • Brief description • Functions and terminals used and genetic environment settings • Results and analysis • Conclusion • What have we learned from the results and analysis? Answers for the research questions • In what situations (when) and in what way (how) can we used loop to a success? Using Loops in GP for A Two Class Binary Image Classification Problem

3. Introduction-Whydo we experiment loops in image classification ? • Powerful • Provides a mechanism for repetition • Natural repetition in images • Seldom used previously in GP (as discussed in our previous paper) • Hard to evolve {start, end, body, update branch} • Takes longer to evaluated and some mechanism must be used to avoid infinity • Large class of problems can be solved without loops • Implicit loops are used. {Santa Fe Ant} Using Loops in GP for A Two Class Binary Image Classification Problem

4. Introduction-The Research Questions • Research Questions • How can for-loops be incorporated into evolved programs for image classification? • Will the classifiers with loops need more/fewer generations to evolve? • Are the classifiers with loops better than the classifiers without loops, that is, more accurate, smaller, more understandable? • What are the differences between decision strategies in the evolved loop and non-loop programs? • Expectations • For-loops can be applied to the classification problem and that programs with for-loops will be smaller in size and easier to analyse and thus more understandable. Using Loops in GP for A Two Class Binary Image Classification Problem

5. Introduction- Related Works [1]John R. Koza, Forrest H Bennet III, David Andre, and Martin A. Keane. Genetic Programming III; Darwinian invention and problem solving. Morgan Kaufmann, 1999. • ADL – automatically defined loops – pre-established maximum number of execution [2] Kenneth E. Kinnear, Jr. Generality and difficulty in genetic programming: Evolving a sort. In Stephanie Forrest, editor, Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93,pages 287–294, University of Illinois at Urbana-Champaign, 17-21 1993.Morgan Kaufmann. • A sorting algorithm – restricted total and individual loops times. – Inverse sized fitness measure [3] William B. Langdon. Data structures and genetic programming. In Peter J. Angeline and K. E. Kinnear, Jr., editors, Advances in Genetic Programming 2, pages 395–414. MIT Press, Cambridge, MA, USA, 1996. • A list data structure- for-while – no nested loops and maximum iteration 32 [4] Mengjie Zhang and Victor Ciesielski. Genetic programming for multiple class object detection. In Norman Foo, editor, Proceedings of 12th Australian Joint Conference on Artificial Intelligence, volume 1747 of LNAI, pages 180{192, Sydney, Australia, 6-10 December 1999. Springer-Verlag. • Image classification Problem [5] Vic Ciesielski and Xiang Li. Experiments with explicit for-loops in genetic programming. In Proceedings of Congress on Evolutionary Computation (CEC2004), pages 494{501. IEEE Press, June 2004. • Different format and constraints on loops usage in GP Using Loops in GP for A Two Class Binary Image Classification Problem

6. Syntax and Semantics of the For-Loops • Pixels are stored in a one dimensional array and can be referenced by image[i] • Loop format used in this paper • (FOR-LOOP START END METHOD) • METHOD {plus, minus} • START, END represents start position of image[i] and end position of image[i] • The FOR-LOOP will calculate the pixels values from start to end using the specified method {- or +}. • STGP is used to allows multiple data types and enforces closure by only generating parse trees which satisfy the type constraints • START, END are of POSTION type • METHOD is of METHOD type • Infinite loops are not possible Using Loops in GP for A Two Class Binary Image Classification Problem

7. Two Class Binary Image Classification - Task Descriptions • Distinguishing two class of objects – circles and squares • Objects are of the same size and in a 16x16 grid • pixels are randomly removed to generate noises • In the first task, objects are centred, see Figure 1 • In the second task, objects are randomly shifted, see Figure 2 Using Loops in GP for A Two Class Binary Image Classification Problem

8. Functions & Terminals Used(without loops) • Original approach Using Loops in GP for A Two Class Binary Image Classification Problem

9. Functions & Terminals Used(with loops) • Loops approach (extra nodes) Using Loops in GP for A Two Class Binary Image Classification Problem

10. Genetic Environments Settings • Parameter Settings : • Fitness Function : Using Loops in GP for A Two Class Binary Image Classification Problem

11. Experiment Results- Success Ratio • The cumulative probability of success of 100 runs for getting a perfect classifier. A perfect classier means that the evolved program that classifies all the training and test images correctly. If a classier passes the training set, but fails to identify all the testing images, it is considered a failure. The graph shows that for the centred images the loop method is much more likely to generate a successful classier. At 600 generations, 82 of the 100 loop runs had succeeded while only 52 of the normal runs without loop were successful. The difference is even more pronounced on the more difficult shifted image problem. After 2,000 generations, 36 of the 100 loop runs had succeeded while only 2 of runs without loops were successful. Using Loops in GP for A Two Class Binary Image Classification Problem

12. Experiment Results - Mean Average Fitness Comments: It is expected that loops approach will give a larger variance in fitness. This is good in a way it creates a diversity and helps to identify what is good and what is bad. Using Loops in GP for A Two Class Binary Image Classification Problem

13. Decision Strategies AnalyseBest Classifier ::Centred Images::Non-Loops vs. Loops A classifier utilizing loops: (d- (ForLoop PosImg228 PosImg188 plus) drand9.260122) A classifier without using loops: (d+ (d- image[37] drand0.441534) f203) Using Loops in GP for A Two Class Binary Image Classification Problem

14. Decision Strategies AnalyseTypical Classifier ::Centred Images::Non-Loops vs. Loops Cons and Pros (no free lunch): • Non-Loops decision points are hard to understand and randomized, but uses smaller number of pixels • Loops decision points are easy to understand, but uses up too many pixels. Using Loops in GP for A Two Class Binary Image Classification Problem

15. Decision Strategies AnalyseTypical Classifier ::Centred Images::Non-Loops vs. Loops A typical successful classifier without using loops: d+ (d- (d+ (d+ (d+ (d+ drand70.929252 image[188]) drand70.929252) (d- (d- drand22.060454 drand70.917456) (d+ drand29.415353 drand89.236116))) (d+(d- (d- image[2] image[155]) (d- image[11] image[26])) (d+ (d+ image[150] image[37]) (d- drand52.450194 drand38.299516)))) (d+ (d (d+ (d+ image[133] drand72.779942) (d+ image[139] image[130])) (d- (d+ drand72.943129 drand86.640064) image[114])) (d+ (d- (d- image[170] image[83]) (d- image[194] image[133])) (d+ (d- image[225] image[172]) (d drand29.415353 f205))))) (d+ (d+ (d+ (d- (d- f18 f194) (d- drand85.580583 image[209])) (d+ (d+ drand61.098601 image[60]) (d+ image[93] drand59.032376))) (d+ (d+ (d+ image[224] drand2.089882) (d- image[229] drand82.981664)) (d- (d- image[135] image[209]) (d- image[187] image[14])))) image[56])) A typical successful classifier utilizing loops: (d- (ForLoop PosImg161 PosImg228 plus) (d- (d+ (d+ drand76.701336 (ForLoop PosImg144 PosImg172 plus)) (d- (d- PosImg152 drand54.382222) PosImg157)) drand14.021874)) ::SIZE MATTERS:: Using Loops in GP for A Two Class Binary Image Classification Problem

16. Decision Strategies AnalyseBest Classifier ::Shifted Images::Non-Loops vs. Loops One of two successful classifiers evolved without using loops One of the smallestclassifiers evolved by the loops method :: CHECKING UP AND DOWN OF THE IMAGES :: Using Loops in GP for A Two Class Binary Image Classification Problem

17. Conclusion • Successful to apply loops for an image classification problems • A loop syntax and semantics which leads to successful evolution for a non-trivial problem. • Programs with loops took fewer generations to evolve • Much more better for more difficult task – shifted images classification • Programs with loops are easier to understand • They are small, understandable, robust • Taking advantage of adjacent pixel information • Suggestions • Loops are good operators in GP when domain knowledge is available to provide some restriction on the semantics Acknowledgement: This work was partially supported by grant EPPNRM054 from the Victorian Partnership for Advanced Computing. Using Loops in GP for A Two Class Binary Image Classification Problem

18. Questions & Suggestions Using Loops in GP for A Two Class Binary Image Classification Problem