310 likes | 497 Views
Causal Models for Performance Analysis of Computer Systems. Jan Lemeire TELE lab May 24 th 2006. Statistics/Causality. Philosophy. Machine Learning. Performance Modeling. What can be learnt about the world from observations?. We have to look for regularities & model them.
E N D
Causal Models for Performance Analysis of Computer Systems Jan Lemeire TELE lab May 24th 2006 Causal Performance Models
Statistics/Causality • Philosophy • Machine Learning • Performance Modeling Causal Performance Models
What can be learnt about the world from observations? • We have to look for regularities • & model them Causal Performance Models
MDL-approach to Learning • Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causal Performance Models
Randomness vs. Regularity • 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causal Performance Models
Ex.: Numberplate Recognition Noise fiercely hinders recognition algorithms Shortest program? + Separation! Two-part code: ‘MWV735’ + letter style + + + drop size variance + drop frequency + random information Causal Performance Models
Conclusions Part I • Extensions to Shannon (information content of a message): • Algorithmic Information Theory • & Kolmogorov Complexity • Fundamental! • But not practical… • No algorithm can exist that outputs the shortest program and Kolmogorov Complexity of an object. Causal Performance Models
II Model of Multivariate Systems • Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causal Performance Models
1 variable • Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(xi|x1…xi-1) (CPD) • Factorization • Mutual information decreases entropy of variable Causal Performance Models
Conditional Independence • Two variables A and B are independent if: • P(A|B)=P(A) • Qualitative property: • Quality of my speech is independent of chance of rain today • P(rain|speech)=P(rain) ? Causal Performance Models
A.Conditional independencies • Reduction of factorization complexity • Bayesian Network • Minimal factorization = MDL • B.Faithfulness Joint Distribution Directed Acyclic Graph Conditional independencies d-separation Theorem: if faithful graph exists, it is the minimal factorization. Causal Performance Models
C.Causal Interpretation • Definition through interventions, otherwise only correlation • V-structure <> Markov Chain • Motivation: Causal models describe all relational regularities in a canonical form Causal Performance Models
Reductionism • Causality = reductionism • Building block = P(Xi|parentsi) • Unique, minimal, independent • Whole theory based on it, like asymmetry of causality • Intervention • = change of block Causal Performance Models
But… Engineers use causal models all the time! Causal Performance Models
Causal model is MDL of joint distribution if Incompressible Incompressible (random distribution) Contribution 1: MDL interpretation of causal models Causal Performance Models
Learning Algorithms • Construct causal model from experimental data • Directly related variables cannot become independent by conditioning on other variables • Undirected graph • V-structures determine orientation • Directed graph Causal Performance Models
Part III: When do causal models become incorrect? • By other regularities! Causal Performance Models
A. Lower-level regularities • Compression of the distributions Causal Performance Models
B. Better description form • Pattern • in figure Causal model? • Other models are better • Why? Graph is compressible & blocks (CPDs) are related Causal Performance Models
C. Interfere with independencies X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causal Performance Models
Deterministic relations • Y=f(X1, X2) • Y becomes unexpectedly independent from Z conditioned on X1 and X2 Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information • Choose simplest relation Causal Performance Models
Moral • Occam’s Razor works • Describe all regularities Contribution 2: Faithful representation of deterministic relations Causal Performance Models
Part IV: Performance Analysis • High-Performance computing parallel system 1 processor Performance Questions: • Performance prediction • Parameter-dependency? • Reasons of bad performance? • System-dependency? • Effect of Optimizations? Causal Performance Models
Causal models (cf. COMO lab) • Representation form • Close to reality • Learning algorithms • TETRAD tool Causal Performance Models
No magic bullet!! Complexity of real data • Mix of continuous and discrete variables • Non-linear relations • Deterministic relations • Context-specific variables and relations Frederik Verbist Joris Borms Causal Performance Models
Causal Performance Model • Computation time of a quicksort algorithm Contribution 3: Formal definition of causal performance models Causal Performance Models
Integrated in statistical analysis • Statistical characteristics • Regression analysis Iterative process • Perform additional experiments • Extract additional characteristics • Indicate exceptions • Analyze the divergences of the data points with the current hypotheses Contribution 4: Performance modeling tool (EPDA) Causal Performance Models
Results so far 1. Learning of non-trivial models Iterative algorithm for solving differential equation in parallel (Aztec benchmark Library) Now: expert can input background knowledge Causal Performance Models
2. Point-to-point communications flight time = latency + message size/bandwidth?? Causal Performance Models
3. Explanations for outliers 4. Effects of optimizations … Causal Performance Models
Conclusions • Theoretical foundations for performance models • Practical use: a lot of tuning • integration, tests, extensions, … • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causal Performance Models