1 / 11

Dynamics of Learning: Single-agent learning theory

Dynamics of Learning & Distributed Adaptation PI: James P. Crutchfield, Santa Fe Institute Agent-Based Computing Grantee Meeting 3-5 October 2000. Dynamics of Learning: Single-agent learning theory Emergence of Distributed Adaptation:

chaney
Download Presentation

Dynamics of Learning: Single-agent learning theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamics of Learning & Distributed AdaptationPI: James P. Crutchfield, Santa Fe InstituteAgent-Based Computing Grantee Meeting3-5 October 2000 Dynamics of Learning: Single-agent learning theory Emergence of Distributed Adaptation: Agent-collective learning theory Related Projects: Computational Mechanics www.santafe.edu/projects/CompMech Evolutionary Dynamics www.santafe.edu/~evca Network Dynamics Program discuss.santafe.edu/dynamics Coyote: Multiprocessor for Agent-based Computing

  2. Computational Mechanics:The Learning Channel TLC: Adaptation of Communication Channel What are fundamental constraints on learning? How to measure environmental structure? How to measure “cognitive” capacity of learning agents? How much data for a given complexity of inferred model?

  3. Computational Mechanics:Preliminaries Observations: s = ss Past  Future: … s-Ls-L+1…s-1s0|s1…s L-1sL … Probabilities: Pr(s), Pr(s), Pr(s) Uncertainty: Entropy H[P] = -i pi log pi [bits] Prediction error: Entropy Rate h = H[Pr(si|si-1si-2si-3…)] Information transmitted to future: Excess Entropy E = H[Pr(s)/ (Pr(s)Pr(s))] Measure of independence: Is Pr(s)=Pr(s)Pr(s)? Describes information in “raw” sequence blocks

  4. Computational Mechanics:Mathematical Foundations Casual state = Condition of knowledge about future -Machines = {Causal states, Transitions} Optimality Theorem: -Machines are optimal predictors of environment. Minimality Theorem: Of the optimal predictors, -Machines are smallest. Uniqueness Theorem: Up to isomorphism, an -Machine is unique. The Point: Discovering an -Machine is the goal for any learning process. Practicalities may preclude this, but this is the goal. (w/ DP Feldman/CR Shalizi)

  5. Computational Mechanics: Why Model? Structural Complexity of Information Source C = H[Pr(S)], S = {Casual states} Uses: Environ’l complexity: Amount/kind of relevant structure Agent’s inferential capacity: Sophistication of models? Theorem: E C Conclusion: Build models vs. storing only E bits of history. Raw sequence blocks do not allow optimal prediction, only E bits of mutual information in blocks. Optimal prediction requires larger model: 2C, not 2E. Explicit: 1D Range-R Ising spin system: C =E+Rh.

  6. Dynamics of Learning: The Aha! Effect Learning complex environments (w/ C Douglas) Learning paradigm Three phases Memorization Aha! Refinement

  7. Dynamics of Learning:Hierarchical Modeling Computation at the Onset of Chaos Onset of chaos leads to infinite -machine Learn the higher level representation Go from series of DFAs to Stack Automaton

  8. Dynamics of Learning: Some Open Questions Learning agents Dynamical systems view of learning as a process whose behavior is predictive model building Define and measure agent “cognitive” abilities Development math’lly analyzable and simulatable models What state-space structures are responsible for learning? E.g., Basins = robust memories; bifurcations = adaptation; models = attractor-basin portrait in subspace; … Robot collectives Group versus individual function Define and measure degree of cooperation Agent collective functioning versus communication topologies

  9. Evolutionary Dynamics Research Mathematical Analysis Epochal evolution Fitness barrier crossing: neutral paths v. fitness valleys? Optimal evolutionary search w/ E van Nimwegen (Dissn@SFI, Fall ‘99) Evolving Cellular Automata Population dynamics Embedded particle computation in CAs w/ W Hordijk (Dissn@SFI, Fall ‘99), M Mitchell, L Pagie, C Shalizi

  10. Structure and DynamicsinComplex Interactive Networks Research Areas Network Structure Network Dynamics Hierarchical and Heterarchical Networks Components: Annual Workshop SFI-Intel Post-Doctoral Fellow Visitor Program Multiprocessor JPC-DW Individual Research Intel (BusNet) 3 years (JPC & DW)

  11. Coyote: SFI’s Beowulf A Supercomputer for Complex Adaptive Systems • Cheap Off-the-Shelf Technology (“Piles of PCs”) • 64 Compute Nodes (128 CPUs), expandable • Fast Network Interconnect (Cisco Gigabit switch) • Physical: Gatehouse room (power/cooling retrofit) • General Availability: Summer Y2K • Team: • Lolly: Cluster Administration/Maintenance • JPC: Coordination, System Architecture • Tim: Node and Network Architecture • Alex: Parallel, Distributed Code Development

More Related