What’s an Algorithm?

1. What’s an Algorithm? Yuri Gurevich Bristol, March 2013 1

2. Computability and algorithms

3. The point Even though computability has played a central role in logic, its utility is questionable, while that of algorithms is obvious.

4. Computable vs. tractable 1 • Logician’s computability is computability in principle, with unbounded resources. • But resources are bounded, in principle. The early optimism that computable implies feasible has not panned out. Nevertheless we, the logicians, “hijacked” the terms like computable, decidable, solvable. Hence the rise of terms like feasible or tractable.

5. Computable vs. tractable 2 • Solvable problems may be intractable. • Real arithmetic • Unsolvable problems may be tractablefor engineering purposes • Byron Cook’s Terminator

6. Algorithms • Algorithms perform tasks. • Computing functions just one class of tasks, a rather narrow class. • What function does an OS compute? • To me, the right question is not “what computable functions are” but “what algorithms are.”

7. The intuitive notion of algorithm

8. Consensus: algorithms are executable • “In mathematics and computer science, an algorithm is a step-by-step procedure for calculations,” Wikipedia. • The executor is an abstract machine, andthe execution may be idealized. • Gauss elimination procedure • Euclid’s algorithms for line segments

9. Dissent: algorithms are recursors • The only dissent that I know is by Yiannis Moschovakis. • His point of view reflects a genuine concern that the analysis of algorithms may be bogged down with irrelevant details.

10. Moschovakis shared the consensus in 1984 “we might be expected ... to simply identify the Euclidean algorithm with the functional gcd. We will not go quite that far, because the time-honored intuitive concept of algorithm carries many linguistic and intensional connotations (some of them tied up with implementations) with which we have not concerned ourselves. Instead we will make the weaker (and almost trivial) claim that the functional gcd embodies all the essential mathematical properties of the Euclidean algorithm.”

11. 2001 Moschovakis “When algorithms are defined rigorously in Computer Science literature ..., they are generally identified with abstract machines . . . My aims here are to argue that this does not square with our intuitions about algorithms ... to promote the problem of defining algorithms correctly; and to describe briefly a plausible solution, by which algorithms are recursive definitions while machines model implementations, a special kind of algorithms.”

12. 2008 Moschovakis (with a student of his) “Moschovakis has proposed a mathematical modeling of the notion of algorithm --- a set-theoretic ‘definition’ of algorithms, much like the ‘definition’ of real numbers as Dedekind cuts on the rationals or that of random variables as measurable functions on a probability space.”

13. My take • Algorithms are executable. • Recursive analysis of algorithms is useful, and recursion plays a big role in computer science. • CS recursions go beyond YM recursors! • There may be no algorithms behind an YM recursor. • In cases like YM’s favorite mergesort,recursors are partial specifications

14. Is it possible to define algorithms?

15. No and yes • No, not in full generality; the notion of algorithm keeps expanding. • Yes, some strata of algorithms have matured and can be formalized. Restricting to computations by means of devices (rather than humans) does not alter at all the situation at all.

16. The algorithm to numbers analogy • The notion of number has been expanding. • Integer, rationals, reals, complex • quaternions and relatives • finite and infinite cardinals • finite and infinite ordinals • Similarly the notion of algorithm keeps expanding. • Classical (a.k.a. sequential) algorithms • Parallel, quantum • interactive, distributed • Real-time, analog, hybrid algorithms

17. Why bother to define algorithms? Here are some reasons: • An intellectual challenge • Proving lower bounds of computational complexity • Software specifications • Algorithmic completeness • Better computation models

18. Classical algorithms

19. Foundational analyses The history is not as rich as it may seem to be. • Turing • Why not Church, Post, Markov, etc.? • Kolmogorov

20. Examples of non-Turing classical algorithms • Ruler and compass • Algorithms working with abstract entities • genuine reals • continues functions • qubits • finite structures

21. D s q p l r C s‘ Example: A ruler-and-compass algorithm Given: A circle C with center p, and a point q outside C. The problem: Construct a tangent at C through q The solution:

22. Bisection program Given a continuous function f with f(a) < 0 < f(b) and given  > 0,find m with |f(m)| < . m:=(a+b)/2while |f(m)|  doif f(m) > 0 then b := m else a := m m:=(a+b)/2output := m

23. An axiomatic definition of classical algorithms

24. Personal motivation • What is 1982 computer science about? • Algorithms: programs, programming languages, compilers, operating systems, etc. • Hence a foundational questionwhat’s an algorithm? • And practical: software specification • Hence (eventually) Michigan  Microsoft

25. Informal definition • Kolmogorov 1953: Algorithms compute in steps of bounded complexity. • An algorithm is a deterministic transition system. • There is a bound, independent from the initial state, on the amount of work performed at each step.

26. Postulate 1: Sequential Time • An algorithm is a transition system determined by states and the transition function. • Two questions remain: • What are the states? • What are the transitions?

27. States, informally The state is information that, given the program, fully determines the ensuing computation(s).

28. Postulate 2: Abstract State States are structuresin the sense of mathematical logic.(Plus some technical details)

29. Postulate 3: Bounded Exploration There are expressions e1,..,en , called critical terms, that determine the state change during any step. • For the bisection program, the critical expressions are

30. Definition of seq algorithmsand a representation theorem • Definition. A seq algorithm is an abstract-state bounded-exploration transition system. • Representation Theorem. For any sequential algorithm, there is a sequential abstract state machine with the same states and the same transition function.

31. Beyond classical algorithms

32. Extensions of the axiomatic definition • Parallel algorithms • Interactive algorithms

33. Some immature strata of algorithms • Distributed • Analog • Learning

34. Thank you