Bayesian Metanetworks for Context-Sensitive Feature Relevance

1. Bayesian Metanetworksfor Context-Sensitive Feature Relevance Vagan Terziyan vagan@it.jyu.fi Industrial Ontologies Group, University of Jyväskylä, Finland SETN-2006, Heraclion, Crete, Greece 24 May 2006

2. Contents • Bayesian Metanetworks • Metanetworks for managing conditional dependencies • Metanetworks for managing feature relevance • Example • Conclusions Vagan Terziyan Industrial Ontologies Group Department of Mathematical Information Technologies University of Jyvaskyla (Finland) http://www.cs.jyu.fi/ai/vagan This presentation: http://www.cs.jyu.fi/ai/SETN-2006.ppt

3. Bayesian Metanetworks

4. Conditional dependence between variables X and Y P(Y) = X (P(X) · P(Y|X))

5. Bayesian Metanetwork • Definition.The Bayesian Metanetwork is a set of Bayesian networks, which are put on each other in such a way that the elements (nodes or conditional dependencies) of every previous probabilistic network depend on the local probability distributions associated with the nodes of the next level network.

6. Two-levelBayesian C-Metanetworkfor Managing Conditional Dependencies

7. Contextual Effect on Conditional Probability (1) X x5 x6 x7 x2 x3 x4 x1 contextual attributes predictive attributes Assume conditional dependence between predictive attributes (causal relation between physical quantities)… xt … some contextual attribute may effect directly the conditional dependence between predictive attributes but not the attributes itself xk xr

8. Contextual Effect on Conditional Probability (3) Xt1 : I am in Paris Xt2 : I am in Moscow xt Xr1 : visit football match Xr2 : visit girlfriend Xk1 : order flowers Xk2 : order wine xr xk Xr:Make a visit Xk:Order present

9. Contextual Effect on Conditional Probability (4) Xt1 : I am in Paris Xt2 : I am in Moscow xt xr xk

10. Contextual Effect on Unconditional Probability (1) X x5 x6 x7 x2 x3 x4 x1 contextual attributes predictive attributes Assume some predictive attribute is a random variable with appropriate probability distribution for its values… xt P(X) … some contextual attribute may effect directly the probability distribution of the predictive attribute X x1 x4 x2 x3 xk

11. Contextual Effect on Unconditional Probability (3) Xt1 : I am in Paris Xt2 : I am in Moscow xt P1(Xk) P2(Xk) 0.7 0.5 0.3 Xk Xk 0.2 Xk1 Xk2 Xk1 Xk2 Xk1 : order flowers Xk2 : order wine xk Xk:Order present

12. Two-level Bayesian C-Metanetwork for managing conditional dependencies

13. Two-level Bayesian R-Metanetworkfor Modelling Relevant Features’ Selection

14. Feature relevance modelling (1) We consider relevance as a probability of importance of the variable to the inference of target attribute in the given context. In such definition relevance inherits all properties of a probability.

15. Feature relevance modelling (2) X: {x1, x2, …, xnx }

16. Example (1) • Let attribute X will be “state of weather” and attribute Y, which is influenced by X, will be “state of mood”. • X (“state of weather”) ={“sunny”, “overcast”, “rain”}; • P(X=”sunny”) = 0.4; • P(X=”overcast”) = 0.5; • P(X=”rain”) = 0.1; • Y (“state of mood”) ={“good”, “bad”}; • P(Y=”good”|X=”sunny”)=0.7; • P(Y=”good”|X=”overcast”)=0.5; • P(Y=”good”|X=”rain”)=0.2; • P(Y=”bad”|X=”sunny”)=0.3; • P(Y=”bad”|X=”overcast”)=0.5; • P(Y=”bad”|X=”rain”)=0.8; P(X) Let: X=0.6 P(Y|X)

17. Example (2) • Now we have: • One can also notice that these values belong to the intervals created by the two extreme cases, when parameter X is not relevant at all or it is fully relevant: !

18. General Case of Managing Relevance (1) Predictive attributes: X1 with values {x11,x12,…,x1nx1}; X2 with values {x21,x22,…,x2nx2}; … XN with values {xn1,xn2,…,xnnxn}; Target attribute: Y with values {y1,y2,…,yny}. Probabilities: P(X1), P(X2),…, P(XN); P(Y|X1,X2,…,XN). Relevancies: X1 = P((X1) = “yes”); X2 = P((X2) = “yes”); … XN = P((XN) = “yes”); Goal: to estimate P(Y).

19. General Case of Managing Relevance (2) Probability P(XN)

20. Example of Relevance Bayesian Metanetwork (1) Conditional relevance !!!

21. Example of Relevance Bayesian Metanetwork (2)

22. Example of Relevance Bayesian Metanetwork (3)

23. When Bayesian Metanetworks ? • Bayesian Metanetwork can be considered as very powerful tool in cases where structure (or strengths) of causal relationships between observed parameters of an object essentially depends on context (e.g. external environment parameters); • Also it can be considered as a useful model for such an object, which diagnosis depends on different set of observed parameters depending on the context.

24. Conclusion • We are considering a context as a set of contextual attributes, which are not directly effect probability distribution of the target attributes, but they effect on a “relevance” of the predictive attributes towards target attributes. • In this paper we use the Bayesian Metanetwork vision to model such context-sensitive feature relevance. Such model assumes that the relevance of predictive attributes in a Bayesian network might be a random attribute itself and it provides a tool to reason based not only on probabilities of predictive attributes but also on their relevancies.

25. Read more about Bayesian Metanetworks in: Terziyan V., A Bayesian Metanetwork, In:International Journal on Artificial Intelligence Tools, Vol. 14, No. 3, 2005, World Scientific, pp. 371-384. http://www.cs.jyu.fi/ai/papers/IJAIT-2005.pdf Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In: German Conference on Artificial Intelligence (KI-2003), LNAI, Vol. 2821, 2003, pp.370-384. http://www.cs.jyu.fi/ai/papers/KI-2003.pdf Terziyan V., Vitko O., Learning Bayesian Metanetworks from Data with Multilevel Uncertainty, In: M. Bramer and V. Devedzic (eds.), Proceedings of the First International Conference on Artificial Intelligence and Innovations, Toulouse, France, August 22-27, 2004, Kluwer Academic Publishers, pp. 187-196 . http://www.cs.jyu.fi/ai/papers/AIAI-2004.ps