1 / 14

Learning Automata

Learning Automata. Learns the unknown nature of an environment Variable structure stochastic learning automaton is a quintuple { j,a,b, A , G} where: j (n) , state of automaton; j ={ j 1 ,…, j s } a (n) , output of automaton; a ={ a 1 ,…, a r }

gilbertcarter
Download Presentation

Learning Automata

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning Automata • Learns the unknown nature of an environment • Variable structure stochastic learning automaton is a quintuple {j,a,b,A,G} where: • j(n), state of automaton; j={j1,…,js} • a(n), output of automaton; a={a1,…,ar} • b(n), input to automaton; b={b1,…,bm} • A, is the learning algorithm; • G[.], is the output function; a(n)=G[j(n)] • n indicates the iteration number.

  2. Learning Automaton Schematic Action Environment Response b(n) a(n) Automaton <f,a,b,A,G> <states, actions, input, learning algorithm, output function> Input Output

  3. Probability Vector • pj(n), action probability; the probability that automaton is in state j at iteration n. • Reinforcement scheme • If a(n) = ai and for j <> i; (j=1 to r) • pj(n+1) = pj(n) - g[pj(n)]when b(n)= 0. • pj(n+1) = pj(n) + h[pj(n)]when b(n)= 1. • In order to preserve probability measure, • S pj (n) = 1, for j = 1 to r.

  4. contd ... • If a(n) = ai • r • pi(n+1) = pi(n) + S g(pj(n)) when b(n) = 0 • j=1, j<>i • r • pi(n+1) = pi(n) - S h(pj(n)) when b(n) = 1 • j=1, j<>i • g(.) is the reward function • h(.) is the penalty function

  5. Schematic of Proposed Automata Model for Mapping/Scheduling Machine Eresp HC System Model bsi(n) asi(n) Automaton for task si <asi,bsi,Asi> <machines, environment response, learning algorithm > Input Output

  6. Model Construction • Every task si associated with an S-model automaton (VSSA). • VSSA represented as {asi, bsi,Asi}, since r = s • asi is set of actions asi = m0, m1, …, m|M|-1 • bsi is input to the automaton, bsi [0, 1] • closer to 0 – action favorable to system; • closer to 1 – action unfavorable to system • Asi is reinforcement scheme • pij(n) - action probability vector • probability of assigning task si to machine mj

  7. contd … • Automata model for Mapping/Scheduling • S-model VSSA is used • Each automaton is represented as a tuple {asi,bsi,Asi} • asi = m0, m1, …, m|M|-1 • bsi [0, 1] (closer to 0 - favorable, 1 - unfavorable) • If ck(n) is better than ck(n-1) Ekresp = 0elseEkresp = 1 • Translating Ekresp to bsi (n) requires two steps

  8. Translating Ekresp to bsi E1resp E2resp Ekresp m|S|-11 m0k m1k m01 m|S|-1k m11 Task s0 Task s1 Task s|S|-1 bs0 bs1 bs|s|-1

  9. contd … • Step 1: Translate Ekresp to msik(n), where • msik(n) - input to automaton si with respect to cost metric ck • achieved by the heuristics • Step 2: Achieved be means of Lagrange's multiplier |C| |C| bsi (n) = S lk * msik(n), i=1 to |S|-1; S lk = 1, lk > 0 j=1 j=1 where lk is the weight of metric ck

  10. Schematic of Proposed Automata Model for Architecture Trades Machine Peval System Solution bsi(n) asi(n) Automaton for component si <asi,bsi,Asi> <components, performance evaluation, learning algorithm > Input Output

  11. Model Construction • Every component of the HW system si associated with a P-model automaton (VSSA). • VSSA represented as {asi, bsi,Asi}, since r = s • asi is set of component types asi = c0, c1, …, c|M|-1 • bsi is input to the automaton, bsi= 0, 1 • 0 – performance favorable to system; 1 – unfavorable to system • Asi is reinforcement scheme • pij(n) - action probability vector • probability of choosing component si from component type cj

  12. contd … • Automata model for Architecture Trades • P-model VSSA is used • Each automaton is represented as a tuple {asi,bsi,Asi} • asi = c0, c1, …, c|M|-1 • bsi0, 1 (0 - favorable, 1 - unfavorable) • If ck(n) is better than ck(n-1) Peval = 0elsePeval = 1

  13. Conclusions • Adaptive Framework for Mapping and Architecture trades • Automata models allow optimization of multiple criteria • Efficient / gracefully degradable solutions • Framework construction suitable for tool integration • Mapping algorithm integrated with SAGETM • Provides a basis for systems design from application to the embedded HW

More Related