Acquisition of Control Knowledge of Nonholonomic System by Active Learning method

1. 能動学習法を用いた非ホロノミック系の知識制御の獲得能動学習法を用いた非ホロノミック系の知識制御の獲得 Acquisition of Control Knowledge ofNonholonomic System by Active Learning method Yoshitaka Sakurai Nakaji Honda Junji Nishino Presented by: Pujan Ziaie

2. Paper Information • Journal of Advanced Computational Intelligence Intelligent Informatics • Received August 28,2002 ; accepted December 13,2002 • Proc. of 2003 IEEE International Conference on Systems • pp.2400--2405 (2003.10)

3. About author • Yoshitaka Sakurai (P.H.D. Student) • University of Electro-Communications • Department of systems Engineering • Honda Lab.

4. Introduction • ALM (Active Learning Method) • IDS (Ink Drop Spread) • Simulation for Gymnastic Bar Action • Mathematical Model & Equations • Active Learning Approach • Conclusion

5. Active Learning Method • Why ALM? • No need to now the System Inner Structure • Improving performance by its own • Characteristics • Construction • Modeling

6. ALM Characteristics • Using SiSO systems • Choosing most effective data • Accumulation of knowledge by Experience • Reinforcement Learning (reward or punishment) • Estimation of overall information By fragmentary information

7. ALM Construction • Similar to human learning Knowledge Acquisition Part Controller Trial & Error IDS Storage Of I/O data Sampling Rule Modeling Database ControlRule Data collection Evaluation SystemUnderControl

8. ALM Modeling (1) • Dividing MIMO System to SISO Systems • Dividing input Domains to fuzzy regions • Extracting the continues narrow path • Calculating the output by Sum of the (Adaptability of each region * region-output) Combination Rule MIMO System SISO SISO Combination Rule SISO

9. ALM Modeling (2) • Example : 2-Input > 1-Output yM yL y ßL ZL ßM ZM X2 X2 b b VS S M L VL y = ßvs* Zvs+ ßs* Zs+ ßM* ZM+ ßL* ZL+ ßVL* ZvL X1=a & X2=b X1 y = ßM* ZM+ ßL* ZL a

10. InkDropSpread method • What is IDS? • Extract narrow path by using fuzzy process on input-output data • Why using IDS? • Create a continuous narrow path • Measure the data distribution amount (extracting the most effective input)

11. IDS Algorithm (1) • Using irradiation pyramid on data plane

12. IDS Algorithm (2) Data plan Projected plan Narrow path Combining the lights

13. IDS Algorithm (3) • Sample of IDS Gathering more Data ( through feedback ) Gathering more Data

14. Control process (1) • Defining the control structure • Dividing inputs into regions according to their range • Selecting most efficient input for the required output ( by human or controller) • Defining evaluation rule for selecting suitable data

15. Control process (2) • Control cycle • Gathering data by using control rules • First time using random numbers • After first time, using the developed controller • Evaluate the gathered data • Improve the partial knowledge function (in case of proper data) • Repeat from step 1

16. Control process (3) • Output calculation method • Remove the most efficient input from inputs • Build input states tree according to valid fuzzy regions • Extract a narrow path of the most efficient input and output for each leaf of the tree • Calculate the final output value by sum of output of each node multiplied by the adaptability of that node. From narrow path By multiplying the Membership values Of nodes from root To the leaf

17. Gymnastic Bar Action • Model of Bar Gymnast • 4 joints & 5 links • Link 0 is not driven • θ0is dependent of the position of center of gravity of the model and shape of posture. • The mass of the head is assumed to be 0. GOAL: achieve the largest swing angel

18. Equations: • θi: relative angle between link i-1 and link i at each joint i. (i=0..4) • T: kinetic energy • V: potential Energy (gravity) • L: T-V > Lagrangian equation • Ii: moment of inertia • xi, yi: coordinates of center of gravity of the ith link • Ni: torque applied on each joint i

19. Acquisition of knowledge • Does a little Kid learn the gymnastic Bar, by Solving lagrangian equation?! NO! • Trying to Learn from the environment by trial and error

20. ALM against Model of Bar gymnast IDS Diagrams Knowledge Acquisition Part Controller Probability based on distribution IO Model Sampling Rule IDS Modeling ControlRule Database Evaluation Data collection Simulator Sequential Database After some specified time Comparing with last most Swing angles

21. Simulation properties • Sampling rate: each 1/1000 Sec • Evaluation: each 2 minutes • Angle range & division: • θ0 : -180 to 180 > 8 MFs • θ1: 0 to 130 > 5 MFs • θ2: -180 to 0 > 5 MFs • θ3: -130 to 30 > 5 MFs • θ4: 0 to 130 > 5 MFs • Most Effective input of each output (joint Torque) > the angle of the same joint

22. Simulation result (1)

23. Simulation result (2)

24. Conclusion • ALM is a Strong flexible method against some complicate control problems • Mathematics is completely useless for many control problems • Advantages of this approach • flexibility • easiness • disadvantages • imperfect information collecting rule • still too crisp

25. Why did I choose this paper? • I liked it. • It was quite a challenge • It was brand new • I have some ideas to improve it • using fuzzy approaches for output • correcting membership functions instead of adding new data

26. acknowledgment • Special thanks to • Sakurai-san for giving me his time and answering my question • Yamazaki-san who helped me to write the Japanese translation of technical words • Serata-san who set me an appointment with Sakurai-san

27. Thank you all for listening • Any easy questions?!

28. Sampling rule • Probability based on distribution function probability function y y Xn

29. input tree • i.e. : • y – four inputs (x0..x3) • – x1 is the most efficient x0 ßob ßoa x2 x2 y ß_L1_S1*f_L1_S1(x1)+ ß_L1_S2*f_L1_S2(x1)+ ß_L1_S3*f_L1_S3(x1)+ ß_L1_S4*f_L1_S4(x1) ß2c ß2d ß2c ß2d x3 x3 x3 ß3e x3 ß3f adaptability of this state(1): ß_L1_S1: ß0b * ß2c * ß3e y using partial knowledge function x1 output for this state(1): f_L1_S1(x1)