74.406 Natural Language Processing - Formal Logic -

1. 74.406 Natural Language Processing- Formal Logic - • Propositional Calculus/Logic (PropLog) • First-Order Predicate Logic/Calculus (FOL or FOPL) • Formal Language (Syntax of formulae; wff) • Inference System • Semantics through Interpretation Function

2. Formal Language A Formal Language is specified as L = (NT, T, P, S) NT Set of Non-Terminal Symbols T Set of Terminal Symbols P Set of Production or Grammar Rules S Start Symbol (top-level node in syntax tree / parse tree) A formal language specifies the syntactically correct or well-formed expressions of a language.

3. Propositional Calculus

4. Propositional Logic (PL) • Propositional Logic: • symbols for facts, statements (propositions) • logical connectives AND, OR, NOT, ,  • "Rules" – condition, consequence; implications • Example:“Dog Mood” • tongue_out  thirsty • growl OR bark  angry • ears_back AND tail_in  anxious

5. Propositional Logic - Syntax • Propositional Logic (Symbols, Terminals): • propositionalsymbolsP, p, Q, q, r, ... • logical connectives,, ,,  • brackets( , ) • Inductive Definition of well-formed formulae (wff): • Propositional symbols P, Q, ... are wffs. • If P is a wff, then also  (P). • If P and Q are formulae then also • (P  Q); (P  Q); (P  Q); (P  Q)

6. Propositional Logic - Semantics • assign truth values to atomic formulae (propositions) • determine truth values for complex formulae (composed from basic propositions using connectives) Truth table

7. Propositional Logic – Example • Example:“Dog Moods” • tongue_out  thirsty • growl OR bark  angry • ears_back AND tail_in  anxious • Exercise:Set-up a truth-table for“Dog Moods” • Write in the left-side columns the observable propositional symbols (growl, bark,...) and in the right columns the derived propositions (anxious,...).

8. Example, Exercise – Truth table • Example: “Dog Moods” • tongue_out  thirsty • growl OR bark  angry • ears_back AND tail_in  anxious

9. Example – Truth tablefor Example: If I win the lottery, every CS420 student gets $1.000. I win the lotteryevery CS420 student gets $1.000

10. First-Order Predicate Logic Syntax and Semantics

11. Syntax of FOPL - Example • PredicateSymbols P, Q, married, ... • Function Symbols f, g, father-of, ... • Variables x, y, z, ... • Constants Sally, John, block-1, c, ... • Connectives, , , ,  • Quantifiers ,  • Terms x, Sally, father-of (Sally) • Sentencesmarried (Sally, John), P (c) • (atomic, complex)x: married (Sally, x), • x y: P (x, y)  Q (x)  R (y)

12. FOPL as Formal Language: Symbols • NT Non-Terminals • Formula, atomic- Formula, complex- Formula, Term, Connective, Quantifier, Predicate, Function, Variable, Constant • T Terminals • Predicate Symbols P, Q, married, ..., T, F • Function Symbols f, g, father-of, ... • Variables x, y, z, ... • Constants Sally, block-1, c • (Binary) Connectives , , ,  • Negation Symbol  • (Unary Connective) • Quantifiers ,  • Equality Symbol = • Other Symbols ( , ) ,:

13. FOPL as Formal Language - Rules 1 Non-terminal Rules Formula ::= complex-Formula | atomic-Formula |T|F atomic-Formula ::= Predicate (Term, ...)| Term = Term complex-Formula ::= Formula Connective Formula | (Quantifier Variable)* ...:Formula | Formula | (Formula) Term ::= Function (Term, ...) | Variable | Constant Terminal Rules Connective ::=  |  |  |  Quantifier ::=  |  Note: The Notation ... in the rules above indicates a list, e.g. a sequence of Quantifier-Variable combinations, or of Terms.

14. FOPL as Formal Language - Rules 2 • General Production Rules • Formula ::= complex-Formula | atomic-Formula |T|F • complex-Formula ::= Formula Connective Formula |Quantifier Variable ...:Formula |Formula | (Formula) • atomic-Formula ::= Predicate (Term, ...)| Term = Term • Term ::= Function (Term, ...) | Variable | Constant • Connective ::=  |  |  |  • Quantifier ::=  |  • Domain Specific Production Rules • Predicate ::= brothers(_,_) |sisters(_,_)|is-mother-of(_,_)| ... • Function ::= gender(_)|age(_)| ... • Variable ::= x|y | • Constant ::= Sally | John | Bill| Mary

15. Notes on FOPL Syntax The term well-formed formula (wff) is often used. equivalent to the term ‘sentence’. wffs are sentences if all their variables are bound by quantifiers. bound variablex: married (Sally, x) open formula: a variable in the formula is not bound, it is free x: married (Sally, x)  happy (y) closed formula: all variables in the formula are bound xy: married (x, y)  happy (x)  happy (y) scope of a quantifier: all occurrences of quantified variables in formulae until over-ruled by new quantifier

16. Equivalence of Formulae x: (x)  x: (x) x: (x)  x: (x) x: (x)  y: (y)

17. Predicate Logic - Semantics An Interpretation function determines the semantics of Predicate Logic formulae. Based on a “Domain” or “Universe” which models “the world”, consists of a set of Individuals (Objects, Constants) with Relations (Roles, Relations, Predicates) among them and Functions (Features, Attributes, Functions). An Interpretation assigns values to terms and formulae: Terms constants, variables, function-expressions Formulae predicate expressions, formulae connected logical connectives, quantified formulae

18. FOPL: Semantics 1 • Define the Semantics of FOPL: • Interpretation – Mapping of symbols of the formal language (predicates, functions, variables, constants) onto the modeled domain (formal: Domain, relational Structure, or Universe) • Valuation - Determine the bindings of variables • Constructive Semantics – Determine the semantics of complex expressions inductively based on the definition of the semantics of basic expressions • Note:Simpler definitions of semantics exist without explicit Valuation function or explicit notation of the interpretation of predicates, functions, constants, and variables in the domain.

19. FOPL: Semantics 2 Interpretation constants I(c)  D (0-ary function) predicates I(P)  Dn for P n-ary predicate functions I(f)  Dn →D for f n-ary function variables I(x)  D(see valuation) ------------------------------------------------------------------- determine constructively based on syntax and above Interpretation: term I(t) D sentence I(α) {T,F}

20. FOPL: Semantics 3 Interpretation term I(f(t1,...,tn)) = I(f)(I(t1),...,I(tn))D atomic sentence I(P(t1,...,tn)) = T if (I(t1),...,I(tn))I(P) complex sentence I(α) = T if I(α)=F  |  |  I(α β) = T if I(α)=T and I(β)=T I(α β) = T if I(α)=T or I(β)=T I(α β) = T if I(α)=F or I(β)=T  |  I(x: α(x)) = T if I(α(x))=T for at least one dI(x) I(x: α(x)) = T if I(α(x))=T for all dI(x)

21. FOPL: Semantics 3b Formulae with multiple / nested quantifiers: Evaluate / Interpret formula from left to right / from outside to inside. I(x: α(x)) = T if I(α(x))=T for at least one dI(x) I(x: α(x)) = T if I(α(x))=T for all dI(x) Easier: Substitute x with constant cC, and later use I(c) instead of I(x). Task: Interpret the following formulae: xy: P(x,y) y x: P(x,y) What is the difference between them?

22. FOPL: Semantics 4 Interpretation of open formulae and Satisfiability Regard complex sentence α with (free) variable x: α(x) choose valuation function and determine satisfiability: valuation function V: V(x) = d D α(x) satisfiable if there is a valuation V with wrt I,V V(x)=d such that I(α(d))=T α(x) true / validif for every valuation V with wrt I V(x)=d, dD I(α(d))=T

23. FOPL: Semantics 5 Model: Given a set of formulae and a structure D with an Interpretation I, and a valuation V, then D is a model of iff I() = Tfor all

24. FOPL: Semantics 6 Semantic-based consequence: Given a set of formulae and a formulaα, and an Interpretation I into a Structure D, we say that αis a logical consequence of iff if I() = T for all then I(α) = T Notation: |=α

25. FOPL: Inference System • Inference in FOPL: • Derive new formulae by syntactic manipulation of existing formulae (through applying inference rules): • given a set of formulae  • apply inference rule (based on  ) • new formula αis derived; αis a Theorem of . • add new formula to . • The set of valid formulae is now α. • Notation:  |– α • α is inferred or derived from .

26. FOPL: Axioms The start-set for inferences in FOPL are the axioms of FOPL. Axioms describe the general features of a logic, and are always assumed to be valid formulae in this logic.

27. FOPL Axioms A1      A2      A3        A4 (  )  ((  )  (  )) A5 x: (x)  (y) A6 (x)  y: (y) based on Frost (1986)

28. Inference Rules – Modus Ponens Modus Ponens:   ,   States that  can be concluded provided we know that the formulae    and  are true in our knowledge base.

29. Inference Rule UGUniversal Generalization Universal Generalization: (x) x: (x) where (x) is a formula containing the free variable x.

30. Inference Rules - Universal Quantifier Introduction Introducing the Universal Quantifier:   (x) x: (x) (x) is a formula containing the free variable x, which is bound in the conclusion by the universal quantifier.

31. Inference Rules - Existential Quantifier Introduction Introducing the Existential Quantifier: (x)   x: (x) (x) is a formula containing the free variable x, which is bound in the conclusion by the existential quantifier.

32. Inference Rules - UI Universal Instantiation: x: (x) (c) where (x) is any formula containing the quantified variable x, and (c) is the same as formula (x) but every occurrence of the variable x is substituted with the arbitrary constant c.

33. Inference Rules - EG Existential Generalization: (c) x: (x) where (c) is a formula containing the arbitrary constant c but not an unbound occurrence of x, and (x) is the same formula as (c) but with every occurrence of the constant c replaced by a variable x. (If x occurs unbound in , use other variable-name.)

34. IR Replacement Rules Replacement Rules                (  ) (  )   

35. FOPL: Semantics and Inference In First-Order Predicate Logic, there is a correspondence(regarding the truth status) between formulae derived through logical Inferenceand their semantic Interpretation. In other words: Any formula derived by inference* is true if and only if it is true in the semantic interpretation. Notation:  |– αiff  |= α * in a sound and complete inference system

36. Inference Systems - Soundness and Completeness Soundness An Inference System is sound iff if  |– αthen |= α Every formula which is derived by formal inference, is semantically true. Completeness An Inference System is complete iff if  |= αthen |– α Every formula which is semantically true can be derived by formal inference.

37. Semantics - Example Predicate Logic Language constants Bill-1, John-3, Sally-1, Mary-1, Mary-2 predicates happy-together, hate-each-other Structure D objects: Uncle-Bill, Uncle-John, Aunt-Sally, The-woman-I-don't-like, Mary relations: Married, Divorced (Uncle-Bill, Aunt-Sally)  Married, (Uncle-John, Mary)  Married (or: {(Uncle-Bill, Aunt-Sally), (Uncle-John, Mary)}=Married (Uncle-John, The-woman-I-don't-like)  Divorced Interpretation I(Bill-1)=Uncle-Bill, I(John-3)=Uncle-John, I(Sally-1)=Aunt-Sally, I(Mary-1)=The-woman-I-don't-like, I(Mary-2)=Mary I(happy-together)=Married, I(hate-each-other)=Divorced True or false? hate-each-other (Bill-1, John-3) happy-together(Bill-1, Sally-1) hate-each-other(John-3, Mary-1) happy-together(John-3, Mary-2)

38. Semantics and Inference -Example Structure D objects: Uncle-Bill, Uncle-John, Aunt-Sally, The-woman-I-don't-like, Mary relations: Married, Divorced (Uncle-John, The-woman-I-don't-like)  Divorced (Uncle-Bill, Aunt-Sally)  Married, (Uncle-John, Mary)  Married (or: {(Uncle-Bill, Aunt-Sally), (Uncle-John, Mary)} = Married) Interpretation I I(Bill-1) = Uncle-Bill, I(John-3) = Uncle-John, I(Sally-1) = Aunt-Sally, I(Mary-1) = The-woman-I-don't-like, I(Mary-2) = Mary I(happy-together) = Married, I(hate-each-other) = Divorced True or false? hate-each-other (Bill-1, John-3) hate-each-other (John-3, Mary-1) happy-together (Bill-1, Sally-1)  happy-together (John-3, Mary-2) x: happy-together(Uncle-Bill, x)) x,y,z: happy-together(x,y)  hate-each-other (x,z) What if you want to add a formula? x,y: happy-together(x,y)  happy-together(y,x)

39. Additional References • Frost, Richard: Introduction to Knowledge Base Systems. Collins Professional and Technical Books, William Collins Sons & Co. Ltd, London, 1986. • Nilsson, Nils J.: Artificial Intelligence - A new synthesis. Morgan Kaufmann Publishers, San Francisco, CA, 1998.