Introduction to Artificial Intelligence: Applications in Computational Biology

1. Introduction to Artificial Intelligence: Applications in Computational Biology Susan M. Bridges bridges@cs.msstate.edu

2. Outline • What is AI? • Search • Expert systems • Uncertainty • Machine learning • Data mining

3. Intelligent Systems and Computational Biology • First applications (DNA) in which great progress was made were digital • Signal processing algorithms • Text processing techniques • Many of the most interesting and difficult problems to be tackled are analog • Protein structure • Gene expression • Metabolic networks

4. Definitions of AI(What is AI?) • Rich, E. and K. Knight . 1991. Artificial Intelligence. New York: McGraw-Hill. “Artificial intelligence (AI) is the study of how to make computers do things which at the moment, people do better.”

5. Another definition of AI • Winston, Patrick Henry. 1984. Artificial Intelligence. 1984. Addison-Wesley, Reading, MA. “Artificial Intelligence is the study of ideas that enable computers to be intelligent. Intelligence includes: ability to reason, ability to acquire and apply knowledge, ability to perceive and manipulate things in the physical world, and others.”

6. Why Study AI? • Understand human human intelligence • Develop “intelligent” machines • Robotics • Programs with intelligent properties

7. Acting Rationally:Turing Test Approach Interrogator

8. Mundane tasks Perception Vision Speech Natural Language Understanding Generation Translation Common sense reasoning Robot control Formal tasks Games Mathematics Geometry Logic Integral calculus Expert tasks Engineering Scientific analysis Medical diagnosis Financial analysis AI Tasks

9. Intelligent Agents • Agent • Perceives its environment using sensors • Acts on environment using effectors • Rational agent • An agent that does the right thing • Basis for action • A measure of degree of success. • Knowledge of what has been perceived so far. • The actions that the agent can perform • Autonomous Agent • Learns from experience • Makes independent decisions

10. Major Topics • Search • Knowledge Representation • Machine Learning

11. Problem-solving agent • A type of goal-based agent • Find sequence of actions that lead to a desirable state • Intelligent agents should make a set of changes in the state of the environment that maximizes the performance measure • Life is simpler if we can set a goal and aim to satisfy it.

12. Components of a problem • Initial state • Set of possible actions • actions can be described as operators • an operator describes an action by specifying the state that can be reached by carrying out an action in a particular state • actions can be described in terms of a successor function S. Given a particular state x, S(x) returns the set of states reachable from x by any single action. Operator a State x State y

13. State Space • The set of all states reachable from the initial state by any sequence of actions • A path in the state space is a sequence of actions leading from one state to another • The agent can apply a goal test to any single state to determine if it is a goal state. • If one path is preferable to another, then we may need to compute path cost (g).

14. 3 5 4 1 2 6 1 8 8 4 7 3 2 7 6 5 Initial State Goal State States Goal Test Operators Path Cost

15. Problem: Find route from Louisville to West Point West Point Pheba Mathiston Mayhew Maben Columbus Starkville Sturgis Artesia Ackerman Crawford Brooksville Louisville

16. A. The initial state Louisville Louisville B. After expanding Louisville Ackerman Starkville Brooksville Louisville C. After expanding Ackerman Ackerman Starkville Brooksville Maben Sturgis Louisville

17. Some terms • New states are generated from old states by operators. • This is called expanding the state. • The choice of which state to expand first is called the search strategy • Result is called a search tree • The set of nodes waiting to be expanded is called the fringe or frontier

18. Search Strategies • Requirements for a good search strategy • causes motion • is systematic • State space can usually be represented as a tree or a graph • Two important parameters of a tree • branching factor (b) • depth (d)

19. Two Types of Searches • Uninformed or blind search • systematically generate states • test states to see if they are goal states • Informed or heuristic search • use knowledge about the problem domain • explore search space more efficiently • may sacrifice accuracy for speed

20. Breadth-first search • All nodes at each depth d are expanded before any nodes at depth d+1

21. Depth-first search • Always expands one of the nodes at the deepest level of the tree • Parameter m is the maximum depth

22. What is a heuristic?(rule of thumb) • A heuristic is a formalized rule for choosing those branches in a state space that are most likely to lead to an acceptable solution (Luger and Stubblefield, 1998). • Used two ways • some problems do not have exact solutions, so we just do the best we can (medical diagnosis) • there may be an exact solution, but it may be very expensive to find

23. Hill Climbing • Use an heuristic function (or objective or evaluation function) to decide which direction to move in the search space. • Always move toward the state that appears to be best (basing all decisions on local information). • Assume that we want to maximize the value of the function. • Can also be used for minimization (called gradient descent)

24. 1 2 3 8 4 7 6 5 1 2 3 7 8 4 6 5 1 2 3 7 8 4 6 6 5 1 2 3 7 8 4 6 5 1 2 3 7 8 4 7 6 5 1 2 3 7 8 4 6 6 5 h= h= Steepest Ascent Hill Climbing Using Manhattan Distance Heuristic Goal 1 2 3 7 4 6 8 5 h= h= h=

25. A* Search • Minimizing the total path cost • Combines uniform-cost search and greedy search. • Evaluation function: f(n) = g(n) + h(n) g(n): cost of path from start to node n h(n): estimate of cost of path from n to goal f(n): estimated cost of the cheapest solution through n

26. Goal: Minimum length path. Is h(n) an admissible heuristic? f(n) = g(n) + h(n) A(22) d = 0 3 5 10 d = 1 B (18) C (21) D (8) 4 6 12 7 8 11 d = 2 E(12) F(7) G (9) H(6) I (13) J(14) 3 11 4 2 7 1 5 12 3 4 d = 3 K (18) L ( 3) M(2) N(9) O(5) P(2) Q(10) R(12) S (18) T(0) U (0) 3 4 14 6 5 d = 4 Numbers in parentheses are h(n) Numbers on edges are operator costs

27. Multiple Sequence Alignment • DNA and protein sequences • Alignment of multiple sequences created by inserting gaps to shift characters to matching positions ATCG-ATCG- TGA--T-GA GATGAT--- • Optimal alignment maximizes the number of matching positions

28. Multiple Sequence Alignment As State-Space Search(Eric Hansen, Rong Zhou) Space Complexity: O (LN) Time Complexity: O (2NLN) Where L is the average length of sequences and N is the number of sequences start ATCG-ATCG- TGA--T-GA GATGAT--- goal

29. 1 9 19 g h f 3 7 16 0 9 18 5 6 17 4 7 18 8 3 14 9 2 13 7 4 15 11 0 11 g h f 6 5 16 4 7 18 3 8 19 2 9 20 g h f 2 8 18 An Illustration of Anytime A* f = g + 2h Nodes pruned by Anytime A* Goal = expanded node Total number of nodes stored = 8 = stored but not expanded node

30. Genetic Algorithms • Search procedure based on a simple model of evolution • Uses a “random” process to explore search space • Has been applied in many domains

31. Terminology • Begin with a population of individuals. Each individual represents a solution to the problem we are trying to solve. • A data structure describes the genetic structure of the individual. (Assume for initial discussion that this is a string of 0’s and 1’s). • In genetics, the strings are called chromosomes and the bits are called genes. • The string associated with each individual is its genotype • Selection is based on fitness of individuals

32. The Genetic Algorithm • Each evolving population of individuals is called a generation. • Given a population of individuals corresponding to one generation, the algorithm simulates natural selection and reproduction in order to obtain the next generation.

33. Three basic operations • Reproduction: • Individuals from one generation are selected for the next generation • Crossover: • Genetic material from one individual is exchanged with genetic material from another individual • Mutation: • Genetic material is altered

34. General GA Procedure Selection, crossover, and mutation operations Initial population Evaluate fitness Parent candidate pool Father and Mother Select parents Crossover and mutate Evaluate fitness and replace Offspring no yes Next generation population Converge?

35. 13 8 15 1 11 13 2 9 1 1 0 1 1 0 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 1 0 1 1 0 1 1 0 0 1 0 0 1 1 1 Example of General GA Procedure Selection, crossover, and mutation operations Generation n 1 1 0 1 1 0 1 1 0 1 0 0 1 0 0 1 Reproduction Crossover Mutation Generation n+ 1

36. Two keys to the success of a GA • Data structures for • Genes • Chromosomes • Population • Fitness evaluation function

37. Knowledge Representation • Semantic networks • Frame based systems • Rule based expert systems • Ontologies • Neural networks

38. Anything AbstractObjects Events Representational Objects Places Processes Sets Numbers Physical Objects Intervals Sentences Measurements Moments Things Stuff Categories Animals Agents Times Weights Humans

39. Expert Systems • Rule based systems • Garnered a great deal of attention in the 1980’s • Most famous examples are in medical domains • Stimulated interest in “logic programming” • Encode knowledge of people as sets of rules • Still widely used • Knowledge acquisition bottleneck

40. Representing Uncertainty • Fuzzy logic • Bayesian reasoning

41. Uncertainty versus Vagueness • Certainty–degree of belief • there is a 50% probability of rain today • I am 30 % sure the patient is suffering from pneumonia • Vagueness–the degree to which an item belongs to a category • the man is tall • move the wheel slightly to the left • the patient’s lungs are highly congested

42. Fuzzy Sets Represent Vagueness • Lotfi Zadeh popularized the idea in the 60’s • Popular concept in Eastern philosophy • Reasoning with fuzzy sets is called fuzzy logic • Fuzzy logic is also called • approximate reasoning • continuous logic

43. Fuzzy Set Definitions • Set membership can be expressed using a characteristic (or descrimination) function • Classic (or crisp) sets If objects x are chosen from some universe X • Fuzzy sets - an element can be a partial member of a set (grade of membership)

44. Examples of Fuzzy Concepts from Natural Language • John is tall • The weather is rainy • Turn the volume up a little • Dr. Bridges’ tests are long • Add water until the dough is the right consistency • There was very little change in the cost • The water bill was somewhat high

45. Representing Fuzzy Sets • Enumeration of membership values of all elements with non-zero membership TALL = {.125/5.5, .5/6, .875/6.5, 1/7, 1/7.5, 1/8} • Represent membership with a function

46. Functional Representations Fuzzy Set Tall Membership 1 Tall 0 Height in feet 4 5 6 7

47. Linguistic (or Fuzzy)Variable • Usually corresponds to a noun • The values of a linguistic variable are fuzzy sets (which correspond to adjectives) • Examples: Linguistic variable Fuzzy sets Height short medium tall Weight light average heavy Temperature cold cool typical warm hot Speed slow medium fast

48. Linguistic Variable Temperature Cold Normal Hot 1 0 30 40 50 60 70 80 90 100

49. Some Fuzzy Set Operations • Set union A  B A  B(x)max(A(x),B(x)) for all x X alternate syntax (join operator) A  B(x)A(x)B(x)) for all x X • Set intersection A B AB(x)min(A(x),B(x)) for all x X alternate syntax (meet operator) A  B(x)A(x) B(x)) for all x X

50. Fuzzy Reasoning • A fuzzy proposition is a statement that asserts a value for a linguistic (or fuzzy) variable • Example: Joe’s height is medium • Linguistic variable (noun) Joe’s height • Fuzzy set (adjective) medium • The fuzzy set “medium” is a value of the linguistic variable “Joe’s height” • A fuzzy rule relates two or more fuzzy propositions • Fuzzy inference techniques are used to draw conclusions using fuzzy rules