Data Evolution in General Formal Problem Statement

1. Data Evolution inGeneral Formal Problem Statement A General Approach to Evolutionary Problem Solving, Implemented in the Computer Language GENETICA

2. Background

3. CLASSICAL PROBLEM SOLVING According to the general notion of "problem solving" there is a set ofstates (blue dots) …

4. Search Space Solution Space CLASSICAL PROBLEM SOLVING … within which is defined a subset of states (red dots) having some desired properties. The former set—structured by an appropriate search procedure—is called "the search space"… … while the subset is called "the solution space". The objective of a problem solving procedure is to reach an element of the solution space, given the search space.

5. CLASSICAL PROBLEM SOLVING This is the case of "confirmation" problems, i.e. problems where a state either belongs to the solution space or not. Confirmation

6. CLASSICAL PROBLEM SOLVING In another class of problems, membership in the solution space is considered as quantitative: all the elements of the search space belong to the solution space, each one having a different membership value which expresses the acceptability of the element as a solution. Then the problem becomes one of finding a state having as high membership value as possible. Optimization This is the case of the optimization problems

7. Although the current Artificial Intelligence (AI) research has focused on different issues, such as "behavioral" intelligence, adaptability, cellular or connectionist systems, evolutionary systems etc, the classical notion of "problem solving" remains valid as—in its most abstract and general form—it is encountered in virtually any AI system. For instance, adaptability can be conceived as convergence to goal-states, while "behaviors" can be viewed as states within a search space. Due to the critical role of "problem solving" in various AI methods, the development of AI systems can be effectively supported by general problem solving tools. Such tools should offer both expressive and creative power. The former has been traditionally dealt with by means of formal languages, within the field of symbolic AI, while the latter—viewed as performance in searching within complex and "unpredictable" search spaces—has been extensively investigated within the field of Evolutionary Computation (EC). As any AI system, even if it is not "algorithmic" in nature, is subjected to a formal description (otherwise it could not be developed in the computer), while several forms of "creative" behavior could be based on (or enhanced by) evolutionary dynamics, the development of AI systems could be effectively supported by tools that integrate formal expression and evolution methods. According to this direction, the computer language GENETICA is proposed here as a general tool for the development of AI systems. First we are going to identify some key-features of the original approaches to "expressive" and "creative" problem solving, respectively represented by symbolic AI and EC methods.

8. Symbolic Language Search SYMBOLIC ARTIFICIAL INTELLIGENCE A major class of problem solving systems falls into the framework of symbolic Artificial Intelligence Physical symbol structures ClassicalAI problem-solving systems are based on "physical symbol structures". Symbols Relations Processes Such systems include symbols, relations and "processes", where symbols and relations form a symbolic language, while "processes" generate the elements of the search space. AI languages include predicate logic, which makes possible to use any logical expression concerning countable classes in the problem representation. Given this representation, AI problem solvers scan the search space until a solution is reached. A solution is a "symbol structure" that confirms the problem statement. The strong point is expressive power, which allows reduction of the search space through the incorporation of problem specific knowledge in the problem formulation. Expressive power Exhaustive Search : Complexity limitations Low Creativity A weak point of classical AI problem-solving is that exhaustive search suffers from the "combinatorial explosion" effect which could make solution practically inefficient. However, this approach reduces "creativity", i.e. the potential of the AI system to create innovative solutions unpredictable in terms of existing problem specific knowledge. Furthermore, AI systems based on symbol structures cannot cope with optimization problems where the search space is continuous.

9. EVOLUTIONARY COMPUTATION Other problem solving systems include optimization methods based on either numerical techniques or stochastic search. Within this framework, Evolutionary Computation systems exhibit a remarkable combination of generality of application and effectiveness. Fitness evaluation Evolutionary problem solving methods evolve elements of the search space aiming to the optimization of "fitness": a magnitude representing the "desirable attribute" of a solution. in a population of individuals A "population" of individuals, i.e. formal representations of potential solutions, is evolved… subjected to genetic operations …as"genetic operations" affect individuals, while emerging high-fitness individuals substitute lower-fitness ones in the population. High computational efficiency The strong point of evolutionary problem-solving is computational efficiency: evolution may result in solutions, in cases where this would be practically inefficient by exhaustive search. High creativity Unfortunately, even if the implementation of problem-specific knowledge is desirable (in order to achieve a more accurate problem statement or a more effective solution procedure) it is often very hard to be attained. Computational efficiency supports creativity, as the need for search space reduction through problem-specific knowledge becomes less intensive. Low expressive power This is because the problem statement is not supported by a universal formal language (such as predicate logic). As a consecuence, evolutionary methods tend to be highly "problem-specific". Furthermore, the quantitative nature of "fitness" makes hard for these methods to cope with general confirmation problems.

10. EVOLUTIONARY COMPUTATION Major fields of investigation within the framework of Evolutionary Computation are Genetic Algorithms (GAs) and Genetic Programming (GP) Genetic Algorithms Binary strings Classical GAs represent to-be-evolved objects as fixed-length binary strings, the latter viewed as genotypes.

11. 1 1 0 0 1 0 1 0 1 0 0 1 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 EVOLUTIONARY COMPUTATION Consider a population of genotypes, each one presented here as a colored dot. Colors represent fitness values: higher fitness genotypes (biased to red) are shown at the top of the oval shape, while lower fitness genotypes (biased to blue) are shown at the bottom. Each genotype is a binary string A new genotype could emerge by the alteration of a limited number of bits within a string. Such alterations constitute "genetic operations" on genotypes, known as "mutations".

12. EVOLUTIONARY COMPUTATION 1 1 0 0 0 0 0 1 1 0 0 1 1 0 1 0 Homologous bits of different genotypes can be interchanged. This is a genetic operation, named "crossover", which results in new genotypes.

13. EVOLUTIONARY COMPUTATION The application of either "crossover" or "mutation" on high fitnesss genotypes may result in higher fitness genotypes. Such genotypes are introduced in the population… …substituting low-fitness genotypes. As this process is repeated, the population is successively enriched in higher-fitness genotypes.

14. EVOLUTIONARY COMPUTATION Genetic Algorithms Binary strings Tree-structured genotypes As binary strings have very limited expressive power… …the expressive power of genetic methods has been enhanced by more sophisticated representations, such as tree-structured genotypes, which reflect domain-specific knowledge. Genetic Programming Computer programs Genetic Programming (GP) considers computer programs as the objects to be evolved. This largely improves expressiveness, as the expressive power of a computer language allows to represent virtually anything as a computer program.

15. EVOLUTIONARY COMPUTATION Consider the syntax structure of a computer program (schematically represented here as a predicate logic expression). ((P AND Q) OR ( x (R)))AND(NOT (F)) (P AND Q)OR( x (R)) NOT(F) PANDQ x (R) P Q R F

16. (P OR Q) AND R P OR Q R P Q EVOLUTIONARY COMPUTATION ((P AND Q) OR ( x (R)))AND(NOT (F)) NOT (F) F

17. (P OR Q) AND R P OR Q R P Q EVOLUTIONARY COMPUTATION Parts of this structure can be interchanged with syntactically compatible parts of the syntax structure of another computer program. This can be viewed as an extension of the "crossover" operation. Partial regeneration of the structure can be viewed as an extension of the "mutation" operation. GP methods use such operations to evolve a population of computer programs, where the fitness of each program is calculated by the program execution. ((P OR Q) AND R) AND (NOT (F)) NOT (F) F

18. EVOLUTIONARY COMPUTATION Although the programming language of a GP application makes possible high expressiveness in solution representation, GP does not provide an equally effective representation of the problem to be solved: in classical GP the user is only allowed to define the elements of the programming language (i.e. the function set and the terminal set), the fitness function, and potentially some aspects of the program architecture. Implementation of problem-specific knowledge has been attempted through specifications of the solution representation language, which results in highly problem-specific GP methods. Another limitation of expressiveness concerns the quantitative nature of "fitness", which severely limits the possibility of a general GP approach to confirmation problems. Evolution in formal logic A class of evolutionary methods—some of them using GP techniques—consider abstract objects, which reflect the syntax of a fixed predicate logic expression, as the objects to be evolved. The logical expression can be viewed as a problem statement, while the objects to be evolved can be viewed as solution instances. In this case the problem representation is supported by the expressive power of predicate logic. Evolutionary machine learning Evolutionary abstract interpretations Evolutionary theorem proving Applications have been developed within the fields of machine learning, abstract interpretation and theorem proving. These applications exhibit high expressiveness in problem representation, while they can cope with confirmation problems. However, they are not appropriate for general problem solving tasks, as their application domain is limited, while they do not exhibit creativity, as they are strictly based on existing knowledge.

19. SOME REMARKS ON COMPUTER AIDED PROBLEM SOLVING Under a general conceptualization, problem-solving in the computer can be viewed as a process of construction and evaluation of data structures during the program execution. Construction of different data structures creates the search space while evaluation selects the elements of the solution space. Elementary decisions Selection method Evaluation method Selection in Countable classes Deterministic Logical Continuous classes Stochastic Numerical The elementary decisions leading to data construction concern selection of an element of either a countable class (which is represented either as a list, an array or an integer interval) or a continuous class (which is represented as a real interval). Selection may be either deterministic (e.g. exhaustive enumeration of potential solutions in AI systems) or stochastic (e.g. random selections encountered in stochastic optimization or genetic methods). Data structures may be evaluated either in a Boolean manner (confirmation criteria) or in a numerical manner (optimization criteria). A general application of evolutionary heuristics in problem solving could be achieved if stochastic elementary decisions within (arbitrarily complex)formal expressions could be treated as genes, while either logical or numerical evaluation of (arbitrarily complex) data structures—the latter viewed as potential solutions—could be formulated as a fitness function.

20. Continued in Section_2.PPS