1 / 34

Benchmarking for Graph Transformation

Benchmarking for Graph Transformation. Gergely Varró gervarro @cs.bme.hu Andy Schürr andy.schuerr@es.tu-darmstadt.de Dániel Varró varro@mit.bme.hu. Talk o verview. Paradigm & tool features. GT overview. Introduction. Benchmarking terminology. Measurements. Model transformation in MDD.

tacy
Download Presentation

Benchmarking for Graph Transformation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Benchmarking for Graph Transformation Gergely Varrógervarro@cs.bme.huAndy Schürrandy.schuerr@es.tu-darmstadt.deDániel Varróvarro@mit.bme.hu

  2. Talk overview Paradigm & tool features GT overview Introduction Benchmarking terminology Measurements

  3. Model transformation in MDD • Model Driven Development (MDD) • thorough system modeling before implementation • precise and visualSW models • on a high abstraction level • from different viewpoints • automatic code generation • Model transformation • automated translations within and between visual modeling languages • MDD requirements on model transformation: • Specification: high-level specification language • Execution: efficient algorithms and techniques • Industrialization: extensive tool support • Upcoming standard: QVT

  4. QVT Main features: object based patterns precondition unidirectional bidirectional Problems: immature tool support lack of experience GT Main features: similar to QVT wide tool support Problems: non-bidirectional(except for TGGs) Pattern matching phase: most critical issuefrom performance viewpoint same as in QVT Specification techniques for MT

  5. Benchmarking • Aim: • systematic and reproducable measurements • on performance • under different and precisely defined circumstances • Overall goal: • help system engineers in selecting tools • Popular approach in different fields • AI • relational databases • rule-based expert systems

  6. Benchmarking in model transformation • Specification examples for MT • Goal: assessing expressiveness • UML-to-XMI, object-relational mapping, UML-to-EJB, etc. • No performance benchmarks forMT • In the paper:Benchmarks for graph transformation tools • terminology for benchmarking • quantitative comparison of tools • Ongoing research: • Benchmarks for general model transformation tools

  7. Talk overview Paradigm & tool features GT overview Introduction Benchmarking terminology Measurements

  8. Metamodeling Class Attribute 1 next 1 * At most one Process * Association 1 1 1 Metamodel Multiplicity constraint held_by token release Inheritance Arbitrary * * * request blocked * * Resource Slot Instance model r4:Resource p4:Process Object r3:Resource h4:held_by h3:held_by n4:next n3:next Link p1:Process p3:Process n1:next n2:next h1:held_by h2:held_by r1:Resource p2:Process r2:Resource

  9. Graph transformation rule p:Process rn:Resource p:Process reqn:request rel:release hb:held_by LHS RHS r:Resource r:Resource Negative applicationcondition ReleaseRule Left-HandSide Right-HandSide p1:Process p2:Process p1:Process p2:Process n:next n:next rel:release t:token LHS RHS r:Resource r:Resource GiveRule

  10. Pattern matching phase p:Process rn:Resource p:Process reqn:request rel:release hb:held_by LHS RHS r:Resource r:Resource ReleaseRule r4:Resource p4:Process r3:Resource h4:held_by h3:held_by n4:next n3:next p1:Process p3:Process n1:next n2:next h1:held_by h2:held_by r1:Resource p2:Process r2:Resource

  11. Updating phase p:Process rn:Resource p:Process reqn:request rel:release hb:held_by LHS RHS r:Resource r:Resource ReleaseRule r4:Resource p4:Process r3:Resource rl:release h4:held_by h3:held_by n4:next n3:next p1:Process p3:Process n1:next n2:next h1:held_by h2:held_by r1:Resource p2:Process r2:Resource

  12. Talk overview Paradigm & tool features GT overview Introduction Benchmarking terminology Measurements

  13. Paradigm features • Definition: problem characteristics • Model size • number of nodes and edges in the model graph • Pattern size • number of nodes and edges in the LHS graph • Fan-out (maximum degree of model nodes) • number of edges adjacent to a certain node • Number of matchings • Transformation sequence length

  14. Tool features • Definition: typical optimization strategies supported by different tools • Multiplicity based optimization • More powerful strategy when traversing edges with ‘at most one’ (0..1) multiplicity • Parameter passing • Parallel rule execution • ‘As long as possible’ (ALAP) rule application • Indexes (for key/index attributes) • Incremental pattern matching • ...

  15. Parameter passing I. p:Process rn:Resource p:Process reqn:request rel:release hb:held_by • Between consecutive rule applications • No recalculation in later steps LHS RHS r:Resource r:Resource ReleaseRule r4:Resource p4:Process r3:Resource h4:held_by h3:held_by n4:next n3:next p1:Process p3:Process n1:next n2:next rl1:release h1:held_by h2:held_by r1:Resource p2:Process r2:Resource

  16. Parameter passing II. p1:Process p2:Process p1:Process p2:Process n:next n:next rel:release t:token LHS RHS r:Resource r:Resource GiveRule r4:Resource p4:Process r3:Resource h4:held_by h3:held_by n4:next n3:next p1:Process p3:Process n1:next n2:next rl1:release h2:held_by t1:token r1:Resource p2:Process r2:Resource

  17. Parallel rule execution (PM) p:Process rn:Resource p:Process reqn:request rel:release hb:held_by • Calculation of all matchings • Independent matchings LHS RHS r:Resource r:Resource ReleaseRule r4:Resource p4:Process r3:Resource h4:held_by 4 3 4 4 h3:held_by 3 n4:next n3:next p1:Process p3:Process 1 3 n1:next n2:next h1:held_by 1 h2:held_by 2 r1:Resource p2:Process r2:Resource 1 2 2

  18. Parallel rule execution (Update) p:Process rn:Resource p:Process reqn:request rel:release hb:held_by • No re-evaluation of valid matchings • Transaction block LHS RHS r:Resource r:Resource ReleaseRule r4:Resource p4:Process r3:Resource rl4:release h4:held_by 4 3 4 4 rl3:release h3:held_by 3 n4:next n3:next p1:Process p3:Process 1 3 n1:next n2:next rl1:release h1:held_by 1 h2:held_by rl2:release 2 r1:Resource p2:Process r2:Resource 1 2 2

  19. Talk overview Paradigm & tool features GT overview Introduction Benchmarking terminology Measurements

  20. Terminology I. • Broad application field of graph transformation • model analysis • model transformation • visual languages with dynamic operational semantics • Informal characteristics

  21. Terminology II. • Incarnation of a scenario • fulfills the informal characteristics • e.g. Mutex for the simulation of visual languages 1 next LHS RHS 1 * Process ReleaseRule * 1 1 1 GiveRule held_by token release RequestRule * * * Resource TakeRule blocked request * *

  22. rN:Resource pN:Process hN:held_by nN:next p1:Process Instance model n1:next h1:held_by r1:Resource p2:Process Terminology III. • N ReleaseRule • N GiveRule • N RequestRule • N TakeRule • Parametric and deterministicspecification

  23. Terminology IV. • Multiplicity based optimization: OFF • Parameter passing: OFF • Parallel rule execution: ON

  24. Terminology V. • Complete characterization • Reproducable measurements N = 10

  25. Talk overview Paradigm & tool features GT overview Introduction Benchmarking terminology Measurements

  26. Categorization • Style • Interpreted: AGG, GTDB • Compiled: PROGRES, Fujaba • Base algorithm • Constraint satisfaction: AGG, GTDB • variables + constraints • Local searches: Fujaba, PROGRES • step-by-step extension of the matching

  27. Environment • Hardware and OS • 1500 MHz Pentium • 768 MB RAM • Linux kernel of version 2.6.7 • Tool related • Runs without GUI • Rule applications guided by Java programs • PROGRES: guided by C program • Standard services of the default distribution • fine-tuning based on developers’ suggestions

  28. Tool specific issues • AGG • Manual implementation of parameter passing • Progres • GRAS (default underlying graph-oriented database) • Prolog-style cuts (determinism) • User undo/redo mechanism being switched off • Fujaba • Additional node for storing processes and resources • Slightly altered models to provide better performance • GTDB • MySQL (version 4.1.7) • No additional optimization

  29. Tool feature support

  30. Quantitative comparison • Constant overhead distributed over more rule applications • Compilation step from graphsto categories in each GT step • Result set clean-up procedure

  31. Effect of multiplicity based optimization GiveRule AGG PROGRES FUJABA GTDB ReleaseRule

  32. Effect of parameter passing AGG PROGRES FUJABA GTDB STS test set, Pattern matching phase, ReleaseRule

  33. Effect of parallel rule application AGG PROGRES FUJABA GTDB ALAP test set, Pattern matching phase, ReleaseRule

  34. Thanks for your kind attention • Contributions to model transformation benchmarking are welcome • additional benchmark examples, test sets, test cases • participation of further tools • new optimization strategies • Benchmark example descriptions, specifications, and measurement results • http://www.cs.bme.hu/~gervarro/benchmark/

More Related