The Effect of Coding on Computation

1. The Effect of Coding on Computation Adina Lederhendler Topics in Biological Physics 23/12/08 Shannon: A universal Turing machine with two internal states. Automata Studies, 1956 Miller: The magical number seven, plus or minus two: some limits on our capacity for processing information, Psychological Review, 1956

2. Outline What is a code? Abstract computation Different codes for universal Turing machines Real-world computation The use of coding in working memory

3. What is a code? Code – Set of rules for translation of input into output. Output 1 Input 1 Output 2 Input 2 Output 3 Input 3 Input NI Output NO

4. What is a code? Examples of codes: ASCII code Binary digits English alphabet Genetic code Amino acids Nucleotide sequences

5. Configuration: Symbol + Internal state. Information encoded as symbols on tape. Turing machine Reading/ writing head Si

6. Si Turing machine Code Configuration Step symbol + state symbol + shift + state complexity of machine (size of code)

7. Universal Turing Machine Si UTM Input tape and code of a Turing machine T Same output as T. UTM not related to any specific computation.

8. Universal Turing machine Shannon, 1956 – what are the limitations on the code of a UTM? Minimal number of states for a UTM? Minimal number of symbols for a UTM? No 1-symbol UTM Cannot carry any information No 1-state UTM Each step depends only on the letter currently being read. Cannot carry enough information Shannon, Automata Studies, 1956

9. Universal Turing machine Universal Turing machine A Symbols States Universal Turing machine B Universal Turing machine C States States ? ? Symbols Symbols ? ? How do alternative codes affect the computation?

10. Two-state UTM Universal Turing machine B States Symbols Symbols that represent input/output Symbols that represent intermediate states during computation Information about states of A carried by symbols of B (symbol-state tradeoff)

11. Two-symbol UTM Universal Turing machine C States Symbols Information about symbols of A carried by states of C (symbol-state tradeoff)

12. Two-symbol UTM Example– Machine A: m = 5, n C tape: A tape:

13. Two-symbol UTM Machine A Machine C

14. Two-symbol UTM Use information about machine A

15. Two-symbol UTM

16. Two-symbol UTM

17. Two-symbol UTM Move to beginning of the next input

18. Two-symbol UTM Ready to begin next step.

19. Symbol-state product Shannon: What is the minimum symbol-state product required to construct a universal Turing machine?

20. Minimal Turing machines Minsky, 1962 7-state, 4-symbol UTM. Later efforts Find additional minimal (size) UTMs. Find more efficient minimal UTMs. Woods, Theoretical Computer Science, 2008

21. Real world systems Best code for specific task Real-world considerations such as: Efficiency: How much time, what resources can we devote to the computation? Accuracy: How much noise is present in the system? How accurate does the computation need to be? Evolution/adaptation to specific task Biological systems

22. AMOUNT OF INFORMATION PER ITEM Working memory Miller, 1956 Attempt to quantify working memory capacity How many items can we recall immediately after being presented with a sequence? Miller,The Psychological Review, 1956

23. Working memory Recoding into “chunks” and storage in short-term memory Sequences of items Retrieval Adaptation through learning New codes = More kinds of “chunks”

24. Working memory Baddeley & Hitch, 1974 P T O B K V What relation must there be between A and C? All of B is included in A Some of C is included in B

25. Summary • A code is a set of rules for carrying out a computation. • Different codes can be used to perform the same computation. • The choice of code may affect the efficiency and accuracy of a computation. • Biological systems can adapt codes to specific tasks.