Interaction: Conjectures, Results, and Myths

Interaction: Conjectures, Results, and Myths Dina GoldinUniversity of Connecticut Tufts University

Some Philosophical Questions(to justify the “Ph” in PhD) Fundamental questions underlying Theory of Computation: What is computation? How do we model it? Tufts University

computation: finite transformation of input to output input: finite size (e.g. string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Turing thesis: Turing Machines capture this notion of computation Shared Wisdom(from our undergraduate Theory of Computation courses) Mathematical worldview: All computable problems are function-based. Tufts University

The Mathematical Worldview “The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development." Martin Davis, Computability & Unsolvability, 1958 “Our basic ideas about what a computer is, what is does, and how it does it, have hardly changed for decades.” Paul Dourish, Where the Action is, 2001 “Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.” SIGACT News, March 2004 “A TM can do anything that a computer can do.” Michael Sipser, Introduction to the Theory of Computation, 1997 Tufts University

computation: finite transformation of input to output input: finite-size (string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Turing thesis: Turing Machines capture this notion of computation computation: ongoing process which performs a task or delivers a service dynamically generated stream of input tokens (requests, percepts, messages) open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence) behavior: objects, processes, components, control devices, reactive systems, intelligent agents Wegner’s conjecture: Interaction is more powerful than algorithms Rethinking Shared Wisdom:(what do computers do?) Tufts University

Example: Driving home from work Algorithmic input: a description of the world (a static “map”) Output: a sequence of pairs of #s (time-series data)- for turning the wheel- for pressing gas/break Similar to classic AI search/planning problems. Tufts University

? Driving home from work (cont.) But… the output depends on every grain of sand in the road (chaotic behavior). Can we possibly have a map that’s detailed enough? Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians. We can’t possibly be expected to predict that in advance! Nevertheless the problem is solvable! Google “autonomous vehicle research” Tufts University

Driving home from work (cont.) • The problem is solvable interactively. • Interactive input: stream of video camera images, gathered as we are driving • Output: a sequence of pairs of #s (time-series data), • generated as we are driving • similar to control systems, or online computation Tufts University

Outline • Rethinking the mathematical worldview • Persistent Turing Machines (PTMs) • interactive extension of the Turing Machine model • PTM expressiveness • Sequential Interaction Thesis • Future work Tufts University

input work S output Three-tape Turing Machines (N3TM) s - current state w1- contents of input tape w2- contents of work tape w3- contents of output tape n1 , n2 , n3- tape head posns • N3TM configurations: • Computation = a sequence of transitions between configurations, from initial to halting. Tufts University

Extending N3TM Computations in1 in1 in2 in2 w1 w1 w2 e ... out1 e out2 e S0 Sh S0 Sh • Dynamic stream semantics • - Inputs are streams of dynamically generated tokens (strings). • - For each input token, there is an N3TM computation generating the corresponding output token. • Persistence (memory) • - The contents w of the work tape at the beginning of each N3TM computation is the same as at the end of the previous one. Tufts University

Persistent Turing Machine (PTM) PTM: N3TM with persistent stream-based computational semantics • Persistent Stream Language (PSL) of a PTM: set of streams Tufts University

PTM Example: Answering Machine (AM) fAM(record X, Y) = (ok, XY) fAM(erase, X) = (done, e) fAM(playback, X) = (X, X) • PSL(AM) contains: (record hello, ok), (erase, done), (record Brown, ok), (record University, ok), (playback, Brown University), … • but not (record hello, ok), (erase, done), (playback, Brown University), … PTM as a sequential object Tufts University

PTM Example: Driving Home from Work Tufts University

1 (1*,1) (1*,1) # (1*,0) (0*,1) (0*,1) 0 (0*,0) PTM Example: Latch • At each step, output first bit of previous step. • inputs in1; outputs 1 • inputs in2; outputs 1st bit of in1 • inputs in3; outputs 1st bit of in2 • ... • PTM with only 3 states • “state” means contents of worktape • PSL(Latch) contains: PTM as a Labeled Transition System Tufts University

Outline • Rethinking the mathematical worldview • Persistent Turing Machines (PTMs) • PTM expressiveness • interactive transition systems • Infinite equivalence hierarchy • Amnesic PTMs • Sequential Interaction Thesis • Future work Tufts University

1 (1*,1) (1*,1) # (1*,0) (0*,1) (0*,1) 0 (0*,0) Interactive Transition Systemsover S < S, m, r > • S is set of states • ris initial state (root) • m is transition relation Required to be recursively enumerable Tufts University

ξ(M) = < reach(M), m, e > > Þ < > < >Îm s , w , s ' , w s ' , w < w , s i o o M i From PTMs to ITSs Reachable memories of a PTM M: Set of words (work-tape contents) w encountered after zero or more macrosteps. where iff Tufts University

Interactive Stream Equivalence • Infinite sequences of input/output token-pairs emanating from a particular ITS state • For an ITS T and state s, ISL(T(s)) [and ISL(T)] are defined similarly to PSL(M(s)) [and PSL(M)] T1 =ISLT2 if ISL(T1) = ISL(T2) Tufts University

ITS Isomorphism Let be ITSs, i=1,2 1. 2. Tufts University

T1 =bisim T2 if $ an interactive bisim. between them ITS Bisimulation Let be ITSs, i=1,2 is a (strong) interactive bisimulation if: 1. 2. 3. Clause 2. with roles of s and t reversed Tufts University

Bisimulation Example S1 S2 a a a S1bisim S2 b c c b Tufts University

=PSL =ms PTMs ITSs =ISL =bisim =iso Equivalence Relationsfor PTMs vs. ITSs Tufts University

Infinite Equivalence Hierarchy • Lk(M) = stream prefix language of PTM Mset of prefixes of length  k for streams in PSL(M) represents finite observations of M • Corresponding notion of equivalence: M1 =k M2 : Lk(M1) = Lk ( M2 ) • Theorem: for any k, there exist PTMs M, M’such that M =kM’ but Mk+1M’ • Variation: for any k and any PTM M, there exists PTM M’such that M =kM’ but Mk+1M’ Tufts University

=∞ =PSL =1 =2 ... Infinite Equivalence Hierarchy (cont) =1 , =2 , =3 … are equivalence relations that partition theset of PTMs into progressively finer equivalence classes =PSLis Persistent Stream Equivalence; it is at the top of this infinite equivalence hierarchy =1 is Turing Machine equivalence Tufts University

=PSL =1 =2 = ... Equivalence Hierarchy Gap • Proof: construct PTMs M1 and M2 where L(M1) = L (M2 )but PSL (M1 ) = PSL (M2 ) Tufts University

sdiv (S*, t) (S*, t) e ... (S*, 1) (S*, 1) (S*, 1) (S*, 1) (S*, 1) (S*, 0) ... n = 0 n = 1 n = 2 n = 3 (S*, 1) (S*, 1) (S*, 1) (S*, 1) ITS for M1 • M1 produces output streams of the form 1*0+ • M2 is the same, but also produces the stream 1* • M1 , M2 exhibit unbounded non-determinism • Theorem: If a PTM M has unbounded nondeterminism, then M diverges. Tufts University

Amnesic PTMs:“half-way” between TMs and PTMs in1 in1 in2 in2 e w1 e w2 ... e e out2 out1 S0 Sh S0 Sh • Amnesic PTMs extend TMs with stream-based semantics. • -- At least as expressive as TMs • Unlike PTMs, they lack persistence. Example: squaring machine (outi = ini2) [Prasse & Rittgen] Tufts University

It Pays to be Persistent =1 =PSL = ASL PSL Two ways to show that PTMs are more expressive than Amnesic PTMs (and, by extension, TMs): • Collapse of the equivalence hierarchy. • Smaller set of stream languages. ASL = {PSL(M): M is an amnesic PTM} PSL = {PSL(M): M is a PTM} Tufts University

Summary of Results [I&C’04] =ASL = =PSL =1 =2 = ... =ms PTMs ITSs =ISL =bisim =iso Tufts University

Outline • Rethinking the mathematical worldview • Persistent Turing Machines (PTMs) • PTM expressiveness • Sequential Interaction Thesis • the origins of the Turing Thesis myth • interaction as a paradigm shift in CS • Future work Tufts University

Sequential Interaction • Sequential interactive computation • system continuously interacts with its environment by alternately accepting an input string and computing a corresponding output string. • Examples of sequential interactive computations: • method invocations of an object instance in an OO language • a C function with static variables • queries and updates to a single-user database • control systems • online computation • recurrent neural networks • transducers • dynamic algorithms • embedded systems Tufts University

Sequential Interaction Thesis • Whenever there is an effective method for performing sequential interactive computation, this computation can be performed by a Persistent Turing Machine • (PTMs can simulate any interactive computing device) • analogue of the Turing Thesis for the computation of TMs: • Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine Tufts University

Strong Turing Thesis • Turing Thesis: Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine • Common Reinterpretation (Strong Turing Thesis) A TM can do (compute) anything that a computer can do • Strong Turing Thesis is a myth • dogmatically accepted by the computer science community • the function-based behavior of algorithms does not capture all forms of computation • Turing himself would have denied it • in the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, a model of computation distinct from Turing Machines and not reducible to it. • choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation. Tufts University

Origins of the Turing Thesis Myth A TM can do anything that a computer can do. Based on several claims: • A problem is solvable if there exists a Turing Machine for computing it. • A problem is solvable if it can be specified by an algorithm. • Algorithms are what computers do. Each claim is correct in isolation (provided we understand the underlying assumptions) Together, they induce an incorrect conclusion Tufts University

Deconstructing the Turing Thesis Myth (1) Claim 1: A problem is solvable if there existsa Turing Machine for computing it. • Assumes: All computable problems are function-based. • Reasons for this assumption: • Adoption of mathematical principles for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines. • Theory of Computation was a field of mathematics. • The batch-based modus operandus of original computers did not lend itself to other conceptualizations of computation. Tufts University

Deconstructing the Turing Thesis Myth (2) Claim 2:A problem is solvable if it can be specified by an algorithm. • Assumes: - A problem is solvable if there exists a Turing Machine for computing it. - Algorithmic computation is function based (the computational role of an algorithm is to transform input data to output data). • Reasons for this assumption: • Original (mathematical) meaning of “algorithms” • E.g. Euclid’s greatest common divisor algorithm • Knuth’s definition of algorithms specified this explicitly: • “An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68] Tufts University

Deconstructing the Turing Thesis Myth (3) Claim 3: Algorithms are what computers do. • Reasons for this assumption: • The ACM Curriculum (1968): Adopted algorithms as the central concept of CS without explicit agreement on the meaning of this term. • When defining algorithms, most textbooks left implicit the assumption of their closed function-based nature; some explicitly violated it. “An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.” [Rice&Rice’69] “An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes...” [Sipser] Tufts University

The Shift to Interaction in CS Algorithmic Interactive Tufts University

Modeling Interactive Computation: PTMs in Perspective • Many other interactive models • Concurrency theory, process algebras • Reactive and embedded systems • Dataflow, I/O automata, synchronous languages, finite/pushdown automata over infinite words • Interaction games, online algorithms • What makes PTMs unique? • First model of interaction to bridge the gap between concurrency theory (labeled transition systems) and traditional TOC. • Models of concurrency are orthogonal to traditional models of computation • Other interactive models were used as tools for proving traditional complexity results (e.g. interactive TMs in cryptography) rather than studied for their own sake Tufts University

Outline • Rethinking the mathematical worldview • Persistent Turing Machines (PTMs) • PTM expressiveness • Sequential Interaction Thesis • Future work • Interactive complexity theory • Multiagent systems • Formalizing indirect interaction Tufts University

Future Work: 3 conjectures • Interactive complexity theoryconjecture: useful notions of complexity can be developed for sequential interaction computation • Multi-stream interaction conjecture: multi-agent interaction is more powerful than sequential interaction [Wegner’97] • Formalizing indirect interaction conjecture: direct interaction does not capture all forms of multi-agent behaviors Tufts University

Referenceshttp://www.cse.uconn.edu/~dqg/papers/ [Wegner’97] Peter WegnerWhy Interaction is more Powerful than AlgorithmsCommunications of the ACM, May 1997 [EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computationbook chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004 [I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine SondereggerTuring Machines, Transition Systems, and InteractionInformation & Computation Journal, 2004 [GW’04] Dina Goldin, Peter WegnerThe Origins of the Turing Thesis MythBrown University Technical Report Tufts University

Interaction: Conjectures, Results, and Myths