Self-* Programming: Run-Time Parallel Control Search for Reflection Box

1. Self-* Programming: Run-Time Parallel Control Search for Reflection Box Olga Brukman and Shlomi Dolev Ben-Gurion University of the Negev Israel

2. Problem • Ideally • Systems should anticipate every possible scenario • Reality • Engineers fail to create such systems despite the effort • The number of possibilities of different interactions with dynamic environment is enormous • Relying on accumulated knowledge of human operator to deal with unexpected situations

3. Example 1 • Airplane flying into ash cloud • Engines stopped • Pilots managed to fly the plane out of the cloud, waited till the engines cooled down, and were able to restart them

4. Example 2 • Airplane crossing the speed of sound • Airplane control handles behave opposite to the expected behavior • Pilots increase the plane speed so it becomes higher than speed of sound, plane control is back to normal

5. Our Contribution:Automatic Control Search Engine for Dynamic Environment • No assumptions on possible environment changes • Experimentation on replicas • Parallelization of experiments • Polynomial search time • Parallelization • Exposing system state • Observing system state (e.g., with Java reflection) • Setting system state to a certain state

6. System Settings: Environment • Environment is large, sophisticated, dynamic • Non deterministic infinite automaton • At every given time slot environment is • deterministic automaton • probabilistic automaton with a transition function Fi • Environment • Non-deterministic infinite automaton • Environment • DA/PA(F1) • Environment • DA/PA(F2) • Environment • DA/PA(F3) t t

7. System Settings: Environment vs. Plant System Environment Environment in’1…...., in’m out’1,……,out’n Plant Control in1,.., ink Plant out1, …,outl Our program (control) interacts with some machinery in environment – plant

8. System Settings: Environment and Plant System • Environment is • Reentrant : no mutual replicas interference • History oblivious: deterministic repetition of behavior for the plants in the same initial state and with the same control Environment in’’1...., in’’m in’1...., in’m in’1….., in’m out’1,……,out’n out’’1,……,out’’n out’1,……,out’n Plant Plant Plant Control1 Control1 Control2

9. System Environment in’1…...., in’m out’1,……,out’n Plant |AP|=N ≤Nmax Control in1,.., ink out1, …,outl Recording plant-environment interaction Control Search Engine Observer bhv={io1,…,ioj} Goals= Behaviors Control Generator plant-environment interaction

10. Settings:PeriodicControl bhv={io2, io3 , io4}, io2, io3 , io4 C={in1(in2, in3, in4)*} + Plant automaton = P=3 in2 io2 in3 in1 io3 io1 in4 io4

11. Control Search Algorithms

12. Control Search Algorithms for Deterministic Environment

13. Algorithm I: black-box • Complexity • Total number steps in experiments: • Longest experiment: O(PNmax) C1 s C2 x∙P sstart s s … P(Nmax +1) CM scurr s sP_1 sP_2 sP_3 sP_N_max+1 scurr … P 0<i≤Nmax P

14. Algorithm IV: reflection-set box sN s1 …. σΣ σΣ σ1 σ1 ... ... σ2 σ2 Ap • Off line search of the constructed plant automaton • Try all controls of form from every state • Complexity • Total number of steps in experiments: • Longest experiment: O(1)

15. Control Search Algorithms for Probabilistic Environment

16. Probabilistic Environment • Plant is unaware of the entire state of the environment • Environment can be considered to be probabilistic automaton • Plant transition function is probabilistic • Control search algorithm executes all the time • Due to probabilistic transition function of plant automaton • Monitoring • Recognize changes in the plant probabilistic transition function

17. Probabilistic Plant Automaton • 0≤pr(s, s’, σ, io) ≤ 1 • prmin– minimal probability σ1, io1,1, pr1,1 sj si … σ1, io1,2 ,pr1,2 σm, iom,1 ,prm,1 σm, iom,2 ,prm,2

18. Computing Probabilistic Plant Automata Graph (PPAG) σ sj? si io? SF times PPAG[si,sj,σ,io]=pr • SF=1/prmin: number of experiments required to discover the edges with the smallest probability

19. Preprocessing: Behavior Suffix Probability (BSP) Table • For every state s and j=1,…,|bhv|: • BSP[si ,j] = [prmax, σ] • prmax is the maximal probability to obtain suffix (bhv,j) starting from plant in state si ; σis the first entry in this control • BSP computed from PPAG pr1 prmax si σ pr2 pr3 j

20. Preprocessing: Behavior Suffix Probability (BSP) Table Cont. • Base step: for every si compute BSP[si,1] • For every si find input σ that produces bhv[k] with the highest probability : max{PPAG[si, *, σ,bhv[k]]} pr1 σ1 (io1) σ(bhv[k]) prmax si σ2 (io2) pr2 σ3(io3) pr3 1

21. Preprocessing: Behavior Suffix Probability (BSP) Table Cont. • Assume we computed all entries BSP[si ,j] for every si and j≤m • BSP[si , (m+1)]? • Let σbe the one that yields the largest value of • Σs_j (PPAG[si,sj,σ,bhv[k-m-1]]∙BSP[s,m].pr) • BSP[si, m+1]=[Σs_j (PPAG[si,s, σ,bhv[k-m-1]]∙BSP[s,m].pr), σ] s,s’,s’’,… pr1= Σs_j (PPAG[si,sj,σ1,bhv[k-m-1]].pr∙BSP[sj,m].pr) σ1 (bhv[k-m-1]) s,s’,s’’,.. σ(bhv[k-m-1]) prmax=Σs_j (PPAG[si,sj, σ,bhv[k-m-1]]∙BSP[sj ,m].pr) si σ2 (bhv[k-m-1]) s,s’,s’’,.. pr2= Σs_j (PPAG[si,sj,σ2,bhv[k-m-1]].pr∙BSP[sj,m].pr) σ3 (bhv[k-m-1]) s,s’,s’’,… pr3= Σs_j (PPAG[si,sj,σ3,bhv[k-m-1]].pr∙BSP[sj,m].pr) 1

22. Algorithm V: reflection-set box sstart=max{BSP[s,|bhv|].pr} BSP[sstart,k].σ BSP[snext,k-1]>BSP[sstart,k] snext pr=0.4 sstart pr=0.6 snext BSP[snext,k-1]<BSP[sstart,k]

23. Algorithm VI: reflection box sbest=max{BSP[s,k].pr} BSP[sstart,k].σ BSP[snext,k-1]>BSP[sstart,k] snext pr=0.7 sbest sstart pr=0.3 snext BSP[snext,k-1]<BSP[sstart,k] s’best=max{BSP[s,k].pr}

24. Conclusions • Framework for automatic control search • Control, plant, environment • Deterministic plant • Probabilistic plant • Polynomial time • Parallelization • Exposing plant state