LING 408/508: Computational Techniques for Linguists

LING 408/508: Computational Techniques for Linguists Lecture 11 9/14/2012

Outline • Infinite sequences and induction • Induction: permutations • Recursion: factorial • Recursion: lists • Short assignment #7 • Long assignment #3

Sequences • Sequences are ordered • Each element is in a particular index of the sequence • Finite sequence • 1, 2, 3, 4, 5 • 1, 4, 16, 25, 36, 49, 64, 81, 100 • A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z • Infinite sequences: • 1, 2, 3, 4, 5, 6, … • 1, 3, 9, 27, 81, 243, … • 0, 1, 1, 2, 3, 5, 8, 13, 21, …

Characterizing finite sequences • Explicitly declare elements, or compute a finite range of elements seq1_1 = [1, 2, 3, 4, 5] seq1_2 = list(range(1,6)) seq2_1 = [1, 4, 16, 25, 36, 49, 64, 81, 100] seq2_2 = [x**2 for x in range(1,11)] seq3_1 = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'] import string seq3_2 = [s for s in string.ascii_letters if s.isupper()] • This won’t work for infinite sequences; need inductive definitions

Inductive definitions • Base case(s): the base case(s) are declared to be the initial item(s) of the sequence. • Inductive case: given that previous item(s) are members of the sequence, provide a rule that generates the next item of the sequence • To generate an infinite sequence: beginning with the base case(s), repeatedly apply the inductive rule to generate more and more items

Example: positive integers1, 2, 3, 4, 5, 6, … • How do we get the next number in the sequence? • Given any number in the sequence, we can obtain the next number by adding 1. • What else do we need, to generate the entire sequence? • The first number in the sequence is 1. • Inductive definition: • Base case: S1 = 1 • Inductive case: Sn = Sn-1 + 1

Apply the inductive definition to generate the sequence of positive integers • Inductive definition: • Base case: S1 = 1 • Inductive case: Sn = Sn-1 + 1 • Generate the first member of the sequence, S1. By the base case, S1 = 1. • Generate the next member, S2. • From the inductive case, Sn = Sn-1 + 1. • So, S2 = S2-1 + 1 = S1 + 1 = 1 + 1 = 2. • Generate the rest of the sequence similarly.

Powers of 3 • 1, 3, 9, 27, 81, 243, … • To obtain the next number in the sequence, multiply the previous one by 3. • The first number in the sequence is 1. • Inductive definition: • Base case: S1 = 1 • Inductive case: Sn = Sn-1 * 3

Fibonacci numbers • 0, 1, 1, 2, 3, 5, 8, 13, 21, … • To obtain the next number in the sequence, add the previous two • Next number: 13 + 21 = 34. • The first two numbers must be explicitly stated. • Inductive definition • Base cases: S1 = 0 S2 = 1 • Inductive case: Sn = Sn-2 + Sn-1

Could generate infinite sequence through an infinite loop def powers3(): val = 1 while True: print(val) val *= 3 powers3() • But we don’t want it to go on forever…

Compute nth value in sequence def powers3_inductive(n): val = 1 for i in range(n): val *= 3 return val print(powers3_inductive(15)) # 4782969

nth Fibonacci number # base cases: fib(1) = 0, fib(2) = 1 # inductive case: fib(n) = fib(n-2) + fib(n-1) def fib_inductive(n): a = 0 b = 1 for i in range(1, n): a, b = b, a+b return a fib_inductive(15) # 377

nth Fibonacci number # base cases: # fib(1) = 0, fib(2) = 1 # inductive case: # fib(n) = fib(n-2) + fib(n-1) def fib_inductive(n): a = 0 b = 1 for i in range(1, n): a, b = b, a+b return a • 0, 1, 1, 2, 3, 5, 8, 13, 21, … nth fib a b 0 1 1 0 1 2 1 1 31 2 42 3 53 5 65 8 78 13 813 21

Permutations of a list • Permutations • Input: a list • Output: a list containing all permutations of elements of the input list • Examples • Input: [1, 2, 3] • Output: [[1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 1, 2], [3, 2, 1] ] • Input: [1] • Output: [[1]] • Input: [] • Output: [[]]

Inductive approach to permutations • Input is a list of length n. • Generate all permutations of zero elements • Generate all permutations of element 1, given all permutations of zero elements • Generate all permutations of elements 1 and 2, given all permutations of element 1 • … • Generate all permutations of n elements, given all permutations of the first n-1 elements

Permutations: inductive case • Input list is of length n. • Suppose we have all permutations of first n-1 elements. • Then produce a new list of permutations, where the nth element is inserted into every possible position in each list in the permutations of n-1 elements.

Permutations: inductive case • Example: find permutations of [1,2,3,4] • Previous step gave permutations of [1,2,3]: [[1,2,3], [1,3,2], [2,1,3], [2,3,1], [3,1,2], [3,2,1]] • Make a new list consisting of the results of: insert 4 into [1,2,3] insert 4 into [1,3,2] insert 4 into [2,1,3] insert 4 into [2,3,1] insert 4 into [3,1,2] insert 4 into [3,2,1]

Result: [[4,1,2,3], [1,4,2,3], [1,2,4,3], [1,2,3,4], [4,1,3,2], [1,4,3,2], [1,3,4,2], [1,3,2,4], [4,2,1,3], [2,4,1,3], [2,1,4,3], [2,1,3,4], [4,2,3,1], [2,4,2,1], [2,2,4,1], [2,3,1,4], [4,3,1,2], [3,4,1,2], [3,1,4,2], [3,1,2,4], [4,3,2,1], [3,4,2,1], [3,1,2,1], [3,2,1,4]] • Insert 4 into [1,2,3]: • [4,1,2,3] • [1,4,2,3] • [1,2,4,3] • [1,2,3,4] • Insert 4 into [1,3,2]: • [4,1,3,2] • [1,4,3,2] • [1,3,4,2] • [1,3,2,4] • Insert 4 into [2,1,3]: • [4,2,1,3] • [2,4,1,3] • [2,1,4,3] • [2,1,3,4] • Insert 4 into [2,3,1]: • [4,2,3,1] • [2,4,2,1] • [2,2,4,1] • [2,3,1,4] • Insert 4 into [3,1,2]: • [4,3,1,2] • [3,4,1,2] • [3,1,4,2] • [3,1,2,4] • Insert 4 into [3,2,1]: • [4,3,2,1] • [3,4,2,1] • [3,1,2,1] • [3,2,1,4]

Insert a number into all positions in a list • Insert 4 into [1,2,3]: [[4,1,2,3], [1,4,2,3], [1,2,4,3], [1,2,3,4]] # inner loop: # assume we have a variable 'perm' new_perms = [] for i in range(len(perm)+1): p = perm[:] p.insert(i, val) new_perms.append(p) 0 1 2 3

Full solution def perms_inductive(L): # base case: permutations of zero elements perms = [[]] # inductive case: # given perms of n-1 elements, # generate perms of n elements for i in range(len(L)): val = L[i] new_perms = [] for perm in perms: for i in range(len(perm)+1): p = perm[:] p.insert(i, val) new_perms.append(p) perms = new_perms return perms

Another way to insert a number into all positions in a list • Insert 4 into [1,2,3]: [[4,1,2,3], [1,4,2,3], [1,2,4,3], [1,2,3,4]] [4, 1, 2, 3] = [] + [4] + [1, 2, 3] [1, 4, 2, 3] = [1] + [4] + [2, 3] [1, 2, 4, 3] = [1, 2] + [4] + [3] [1, 2, 3, 4] = [1, 2, 3] + [4] + [] # assume we have a variable 'perm' new_perms = [] for i in range(len(perm)+1): p = perm[:i] + [val] + perm[i:] new_perms.append(p) 0 1 2 3

Base case • What are the permutations of []? • [] ? • [[]] ? • Must be [[]]. If we take [] to be the solution for the base case, then the inductive case doesn’t get anywhere • Inductive case inserts value into all permutations, but [] does not contain anything inside it

We need to do this for all recursively generated permutations • Insert 1 into [2,3,4] • Insert 1 into [2,4,3] • Insert 1 into [3,2,4] • Insert 1 into [3,4,2] • Insert 1 into [4,2,3] • Insert 1 into [4,3,2]

Recursion • A recursive function is a function that calls itself. • Technique is to solve a “smaller” problem (subproblem) first, and use that result to compute a solution for the overall problem. This is the recursive case. • A recursive function must have a base case that does not call itself again, to avoid infinite recursion.

Induction vs. recursion • Suppose the input is n. • Induction begins by computing the element of the base case first. • Recursion computes the solution for n first, where that solution requires a solution to the problem for n-1.

Factorial • The factorial of a positive integer n is defined as the product of all positive integers less than or equal to n. • Example: 5! = 5 * 4 * 3 * 2 * 1 = 120 • The factorial of 0 is defined to be 1: 0! = 1

Factorial computed inductively # assume n >= 0 def factorial_inductive(n): # base case: 0! = 1 fac = 1 # inductive case for i in range(n): fac *= i return fac

Think about factorial recursively • If n = 0: n! = 1 • If n > 0: n! = n * (n-1) * (n-2) * … * 1 • Rewrite this as n! = n * (n-1)! since (n-1)! = (n-1) * (n-2) * … * 1

Factorial: n! = n * (n-1)! • Suppose you want to calculate n! recursively. • A “smaller” version of this problem is (n-1)! • If you have already computed (n-1)!, multiply the result by n to get n! • Minimum problem size: Base case • 0! = 1

Example: factorial • n! = n * (n-1) * (n-2) * … * 1 • n! = n * (n-1)! • 0! = 1 def factorial(n): if n==0: # base case return 1 else: # recursive case return n * factorial(n-1)

Suppose you call factorial(5) def factorial(n): if n==0: # base case: 0! = 1 return 1 else: # recursive case return n * factorial(n-1) • n is 5, so the function returns n * factorial(4) • Take a “leap of faith” that the recursive call to factorial(4)will return the correct result of 24. • Then it returns 5 * 24 = 120.

Base case • Recursion does not happen infinitely because of the base case. • Example: • A call to factorial(5) calls factorial(4) • A call to factorial(4) calls factorial(3) • A call to factorial(3) calls factorial(2) • A call to factorial(2) calls factorial(1) • A call to factorial(1) calls factorial(0) • A call to factorial(0) returns 1

Full evaluation of factorial(5) def factorial(n): if n==0: # base case: 0! = 1 return 1 else: # recursive case return n * factorial(n-1) factorial(5) = 5 * factorial(4) = 5 * (4 * factorial(3)) = 5 * (4 * (3 * factorial(2))) = 5 * (4 * (3 * (2 * factorial(1)))) = 5 * (4 * (3 * (2 * (1 * factorial(0))))) = 5 * (4 * (3 * (2 * (1 * 1)))) base case fac(0) returns 0 = 5 * (4 * (3 * (2 * 1))) factorial(1) returns 1 = 5 * (4 * (3 * 2)) factorial(2) returns 2 = 5 * (4 * 6) factorial(3) returns 6 = 5 * 24 factorial(4) returns 24 = 120 factorial(5) returns 120

Compare recursion and iteration • Algorithms can be coded with either recursion or induction/iteration. For some problems, it seems more natural to use recursion. def factorial_recursive(n): if n==0: # base case: 0! = 1 return 1 else: # recursive case return n * factorial(n-1) def factorial_inductive(n): fac = 1 # base case: 0! = 1 for i in range(n): # inductive case fac *= i return fac

Count number of occurrences in a list • Write a function count_recursive(L, x) that returns the number of occurrences of x in the list L. • Suppose L = [1, 2, 3, 2, 5, 2]. • count_recursive(L, 2) # result is 3 • Of course, in Python we don’t really need to do this, since there is a method L.count(x)

Thinking about a list recursively • A list is either an empty list, or a non-empty list, in which case we can break it up into smaller components: • A single value called the head • The rest of the list, called the tail, which is a list • Suppose a list is non-empty ( >= 1 element). • The head is L[0] • The tail is L[1:] } head tail

Head and tail of a list:head = L[0], tail = L[1:] • L = [1,2,3] • Head = 1 • Tail = [2,3] • L = [1] • Head = 1 • Tail = [] • L = [] • Special case: no head or tail

Count occurrences of x in a list L • Suppose L is the empty list []. • The count of x in L is 0. • Suppose L is not empty. • Break up the list into its head and tail. • Let tail_count be the count of x in the tail. • If head==x, the count of x in entire list is 1 + tail_count. • If head != x, the count of x in entire list is tail_count.

Recursively count number of occurrences of x in a list L def count_recursive(L, x): if L==[]: # base case return 0 else: # recursive case head = L[0] # break up list tail = L[1:] # into head and tail tail_ct = count_recursive(tail, x) if head==x: return 1 + tail_ct else: return tail_ct

Palindromes, recursively 1 2 3 4 3 2 1 • Input: a list • If the list is empty or has only element, it is a palindrome • Suppose the inside portion of a list is a palindrome. • If, in addition, the outer elements are equal, the entire list is a palindrome. pal pal 1 palindrome 2 3 4 3 2 1 palindrome

Palindromes def is_palindrome(L): if len(L) < 2: # empty list or length 1 return True # are both palindromes else: return L[0]==L[-1] and is_palindrome(L[1:-1]) • Recursive calculation of palindromes: • Test if elements on left and right end are the same: L[0]==L[-1] • Recursively test if elements on inside of list constitute a palindrome: • Create a new list containing inside contents: L[1:-1] • Test if this list is a palindrome: is_palindrome(L[1:-1])

Due 9/17 1. Write a function that inductively generates the nth perfect square • Perfect squares are: 1, 4, 9, 16, 25, 36, … • Sequence starts at n = 1 • Example: 1st perfect square is 1 • This is not very hard, but it is also not trivially easy 2. Write a recursive function that returns the length of a list

Due 9/24

LING 408/508: Computational Techniques for Linguists