1 / 31

Causal Models for Performance Analysis of Computer Systems

Causal Models for Performance Analysis of Computer Systems. Jan Lemeire TELE lab May 24 th 2006. Statistics/Causality. Philosophy. Machine Learning. Performance Modeling. What can be learnt about the world from observations?. We have to look for regularities & model them.

emmy
Download Presentation

Causal Models for Performance Analysis of Computer Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Causal Models for Performance Analysis of Computer Systems Jan Lemeire TELE lab May 24th 2006 Causal Performance Models

  2. Statistics/Causality • Philosophy • Machine Learning • Performance Modeling Causal Performance Models

  3. What can be learnt about the world from observations? • We have to look for regularities • & model them Causal Performance Models

  4. MDL-approach to Learning • Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causal Performance Models

  5. Randomness vs. Regularity • 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causal Performance Models

  6. Ex.: Numberplate Recognition Noise fiercely hinders recognition algorithms Shortest program? + Separation! Two-part code: ‘MWV735’ + letter style + + + drop size variance + drop frequency + random information Causal Performance Models

  7. Conclusions Part I • Extensions to Shannon (information content of a message): • Algorithmic Information Theory • & Kolmogorov Complexity • Fundamental! • But not practical… • No algorithm can exist that outputs the shortest program and Kolmogorov Complexity of an object. Causal Performance Models

  8. II Model of Multivariate Systems • Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causal Performance Models

  9. 1 variable • Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(xi|x1…xi-1) (CPD) • Factorization • Mutual information decreases entropy of variable Causal Performance Models

  10. Conditional Independence • Two variables A and B are independent if: • P(A|B)=P(A) • Qualitative property: • Quality of my speech is independent of chance of rain today • P(rain|speech)=P(rain) ? Causal Performance Models

  11. A.Conditional independencies • Reduction of factorization complexity • Bayesian Network • Minimal factorization = MDL • B.Faithfulness Joint Distribution Directed Acyclic Graph Conditional independencies  d-separation Theorem: if faithful graph exists, it is the minimal factorization. Causal Performance Models

  12. C.Causal Interpretation • Definition through interventions, otherwise only correlation • V-structure <> Markov Chain • Motivation: Causal models describe all relational regularities in a canonical form Causal Performance Models

  13. Reductionism • Causality = reductionism • Building block = P(Xi|parentsi) • Unique, minimal, independent • Whole theory based on it, like asymmetry of causality • Intervention • = change of block Causal Performance Models

  14. But… Engineers use causal models all the time! Causal Performance Models

  15. Causal model is MDL of joint distribution if Incompressible Incompressible (random distribution) Contribution 1: MDL interpretation of causal models Causal Performance Models

  16. Learning Algorithms • Construct causal model from experimental data • Directly related variables cannot become independent by conditioning on other variables • Undirected graph • V-structures determine orientation • Directed graph Causal Performance Models

  17. Part III: When do causal models become incorrect? • By other regularities! Causal Performance Models

  18. A. Lower-level regularities • Compression of the distributions Causal Performance Models

  19. B. Better description form • Pattern • in figure Causal model? • Other models are better • Why? Graph is compressible & blocks (CPDs) are related Causal Performance Models

  20. C. Interfere with independencies X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causal Performance Models

  21. Deterministic relations • Y=f(X1, X2) • Y becomes unexpectedly independent from Z conditioned on X1 and X2 Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information • Choose simplest relation Causal Performance Models

  22. Moral • Occam’s Razor works • Describe all regularities Contribution 2: Faithful representation of deterministic relations Causal Performance Models

  23. Part IV: Performance Analysis • High-Performance computing parallel system 1 processor Performance Questions: • Performance prediction • Parameter-dependency? • Reasons of bad performance? • System-dependency? • Effect of Optimizations? Causal Performance Models

  24. Causal models (cf. COMO lab) • Representation form • Close to reality • Learning algorithms • TETRAD tool Causal Performance Models

  25. No magic bullet!! Complexity of real data • Mix of continuous and discrete variables • Non-linear relations • Deterministic relations • Context-specific variables and relations Frederik Verbist Joris Borms Causal Performance Models

  26. Causal Performance Model • Computation time of a quicksort algorithm Contribution 3: Formal definition of causal performance models Causal Performance Models

  27. Integrated in statistical analysis • Statistical characteristics • Regression analysis Iterative process • Perform additional experiments • Extract additional characteristics • Indicate exceptions • Analyze the divergences of the data points with the current hypotheses Contribution 4: Performance modeling tool (EPDA) Causal Performance Models

  28. Results so far 1. Learning of non-trivial models Iterative algorithm for solving differential equation in parallel (Aztec benchmark Library) Now: expert can input background knowledge Causal Performance Models

  29. 2. Point-to-point communications flight time = latency + message size/bandwidth?? Causal Performance Models

  30. 3. Explanations for outliers 4. Effects of optimizations … Causal Performance Models

  31. Conclusions • Theoretical foundations for performance models • Practical use: a lot of tuning • integration, tests, extensions, … • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causal Performance Models

More Related