Introduction to Algorithms

1. Introduction to Algorithms

2. Algorithms • Algorithms are ways of solving problems. There is a technical definition that basically says an algorithm is a clear set of instructions which if followed will lead to a correct problem solution in a finite amount of time • Sorting is a very important problem and has been studied extensively. We begin by looking at a simple sorting algorithm • We build up to the algorithm by starting with finding the smallest element in an array

3. Finding the smallest element in an array min = data(1) 'the smallest so far Fork = 2 TolastIndex 'test each element to see if it is the min so far Ifdata(k) < min Then'new min found min = data(k) End If Next

4. Find the Smallest and Make it the First • What if we want to find the smallest element and put it first in the array? • We’ll do this by switching it with the first element • We need to know the index of the smallest element to do this • A slight modification of our code for finding the smallest element will let us do this

5. Find the smallest element and its index min = data(1) 'the smallest so far minIndex = 1 For k = 2 TolastNdx 'test each element to see if it is the min so far If data(k) < min Then'new min found min = data(k) minIndex = k End If Next k

6. Switching the Values of Variables(PROBLEM!!) • Consider the following code: varA = 1 varB = 4 varA = varB varB = varA • What are the values of varA and varB after I do this?

7. Switching the Values of Variables • Doing it right: varA = 1 varB = 4 temp = varB varB = varA varA = temp • What are the values of varA and varB after I do this?

8. Now Using the Array ‘*** find the smallest element, value and index min = data(1) 'the smallest so far minIndex = 1 For k = 2 TolastNdx 'test each element to see if it is the min so far Ifdata(k) < min Then'new min found min = data(k) minIndex = k End If Next ‘*** exchange the smallest element with the first temp = data(1) data(1) = data(minIndex) data(minIndex) = temp

9. Idea for Sorting • Find the smallest element in the array, and switch it with the first one • Find the smallest element in the rest of the array, and switch it with the second one • Etc. • This is called selection sort

10. Algorithm picture (1) • Here’s an initial array: • The smallest element is in index 3. If we switch it with the element in index 1, we get: • We now know the first element is the smallest.

11. Algorithm picture (2) • Looking at elements 2-5, the smallest is in index 5 • Let’s switch with the element in index 2 • Now we know the first two are smallest, and are in the right order.

12. Algorithm picture (3) • Consider elements 3-5. The smallest is in position 5. • Let’s switch with the element in index 3

13. Algorithm picture (4) • Consider elements 4-5. The smallest is in position 5. • Let’s switch with the element in index 4 • This finishes sorting the array (why?)

14. Algorithm Structure • We want to work on the whole array, then the array without the first element, then the array without the second element, etc. • If we work on a whole array of n elements, that’s a loop from 1 to n. • If we work on a whole array minus the first element, that loop is from 2 to n. • Next we do 3 to n, etc.

15. Loop Setup • A loop from 1 to n looks like: Fork = 1 Ton <code> Next k • Here’s a loop from 2 to n: Fork = 2 Ton <code> Next k

16. In general… • We need a loop that looks like this: Fork = jTon <code> Next k for each j going from 1 to n-1. This we can do by using another loop!

17. The Nested Loop • Here’s what the structure looks like Forj = 1 Ton - 1 Fork = j + 1 Ton <code> Next k Next j

18. Here’s the Code For j = 1 To lastNdx – 1 ‘start with element j min = data(j) 'the largest so far minIndex = j For k = j + 1 TolastNdx‘look at elements that follow j Ifdata(k) < min Then min = data(k) minIndex = k End If Next k temp = data(j) ‘exchange the smallest element with element j data(j) = data(minIndex) data(minIndex) = temp Next j

19. Tricky Bits • Note the -1 and +1 in the loop limits. Getting those right takes some thought • Does the code work on arrays with just one element? With two elements? With no elements? (Nothing to sort, but we want to avoid a runtime error.) What if the data is already sorted?

20. Demo: Simple Sort

21. Other Ways of Sorting • There are actually many ways of sorting items • Sorting is very important so people have put a lot of thought into it • Some well-known methods: • Bubble sort • Quicksort • Heapsort • Mergesort • Bucket sort

22. Which Method is Best? • With small data sets, the best method is usually the easiest one to program • With large data sets, speed becomes an issue • We could measure the time with a stopwatch, but the essential factor is the functional form of the time: if n is the length of the list of data, is the time proportional to n? n log n? n2?

23. Comparison of n, n log n, n2

24. Time for Selection Sort • The number of comparisons of data elements in a sorting algorithm is usually proportional to the time • On the first loop in Selection Sort, we do n-1 comparisons. The second loop does n-2, etc; the last loop does 1 • So the time is roughly proportional to (n-1) + (n-2) + … + 1 = (n^2 – n)/2 • The largest power of n is n^2 which dominates the time for this algorithm • This means Selection Sort is actually too slow to use on large amounts of data

25. Importance of Algorithms • You now know the basics for programming: assignment statements, conditionals, procedures and functions, loops, and arrays • This is like knowing the rules for chess or go • What you have only started to learn are the tactics and strategies to use these tools effectively • Algorithms are the tactics for how to accomplish tasks quickly and correctly

26. Software Engineering • Software engineering is about the strategies to control the complexity of designing large programs • We’ve been learning a few of these strategies (e.g. naming conventions, principles of program structure, requirements and specifications) • Good software engineering allows one person or a large group to produce a complex program that is correct and cost-effective