Trace-Directed Modelling State-of-the-Art, Ideas and Plans

1. Trace-Directed ModellingState-of-the-Art, Ideas and Plans Timothy C. Lethbridge CRuiSE (Complexity Reduction in Software Engineering) Research Group SITE (School of Information Technology and Engineering) University of Ottawa December 2009 tcl@site.uottawa.cahttp://www.site.uottawa.ca/~tcl

2. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

3. Current Status • Project considerably delayed due to difficulty recruiting students • However I finally have three new students who are supposed to be starting in January • Two for this project and one for basic science that will help with this project • Hamoud Aljamaan • Ahmed and Mahmoud Orabi • More later on plans

4. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

5. Quick Overview of Subproject Intent • Find ways to improve models so tracing can be more effectively performed • Models that simplify understanding and design for multi-core architectures • With adjustable levels of abstraction • Generating an optimal set of tracepoints • Find better ways to extract models from traces • Match traces to models they were generated from to increase understanding and find discrepancies • It was previously agreed that the focus of modeling will be using large-scale concurrent state machines • All the above is subject to adjustment according to the needs of the sponsors

6. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

7. State of the Art - Verification Using State Machine Models - Li et. Al. • Li, X., Qiu, X., Wang, L., Lei, B., and Wong, W. E. 2008. “UML state machine diagram driven runtime verification of Java programs for message interaction consistency.” Proc ACM SAC '08. pp 384-389. • DOI= http://doi.acm.org/10.1145/1363686.1363781 • Steps in their method • Design the system with method call sequences specified using a state machine • Instrument method calls to allow tracing • Drive program with random test cases to generate traces • Verify sequences match the original design

8. Sample State Machine Used in the Experiments of Li et. al.

9. State of the art - Bottleneck Detection and Elimination - Saidi et al. • Saidi, A. G., Binkert, N. L., Reinhardt, S. K., & Mudge, T. “End-to-end performance forecasting: finding bottlenecks before they happen”. Proc 36th Annual International Symposium on Computer Architecture, 2009. ISCA '09. ACM, pp.361-370. • DOI= http://doi.acm.org/10.1145/1555754.1555800 • Overall approach aims to analyse hardware+software in multi-core systems • Identify end-to-end critical paths • Modelling performed as collections of state machines interacting via queues • Case study: Linux TCP/IP stack & ethernet controller • Identifies bottlenecks, then change model to remove

10. Saidi et al cont’d: Method • Start with state machines if possible • But if the system is not modelled with state machines then: • Reverse engineer a state machine • Little detail in this paper - I will discuss later • Manually refine the state machine so it has the required level of abstraction • Convert multiple interacting state machines into a dependency graph • Transitions become the nodes • States become arc with timing while in state

11. Saidi et al cont’d:Sample Original State Machines

12. Saidi et al cont’d:Sample Derived Dependency Graph

13. Saidi et al cont’d:Modifying Dependencies

14. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

15. State of the art - Inferring Models from Traces - Walkinshaw et al • Walkinshaw, N. and Bogdanov, K. “Inferring Finite-State Models with Temporal Constraints”. Proc 23rd IEEE/ACM international Conference on Automated Software Engineering 2008 pp. 248-257. • DOI= http://dx.doi.org/10.1109/ASE.2008.35 • Key ideas • State machine models have wide potential use • They are ‘unpopular’ due to weaknesses in tools to keep them up to date to generate them • Reverse engineering complete state machines from traces is hard • To generate a complete machine, complete path coverage needed • A large number of test cases must be traced • Temporal logic can help

16. Walkinshaw et al - cont’d:Literature Review • Provides an excellent overview of the state of the art in inferring state machines from traces • Traditional approach • Passive merging of similar sequences in an augmented prefix tree acceptor (APTA) • Exemplified in these papers • A. Biermann and J. Feldman. “On the synthesis of finite-state machines from samples of their behavior”. IEEE Trans. Computers, 21:592–597, 1972. • G. Ammons, R. Bodík, and J. Larus. “Mining specifications”. POPL’02, 4–16, 2002. • J. E. Cook and A. L. Wolf. “Discovering models of software processes from event-based data”. ACM TOSEM, 7(3):215–249, 1998. • D. Lo and S. Khoo. “SMArTIC: towards building an accurate, robust and scalable specification miner”. In SIGSOFT FSE, 265–275, 2006

17. Walkinshaw et al - cont’d:The Evolution of State Merging • Passive merging leads to overgeneralization if the traces don’t cover the full space of possibilities • A problem for most large systems without complete testcases • More recent work: Active merging • Asking questions of the developer when information is needed • Exemplified in: • P. Dupont, B. Lambeau, C. Damas, and A. van Lamsweerde. “The QSM algorithm and its application to software behavior model induction”. Applied Artificial Intelligence, 22:77–115, 2008. • N. Walkinshaw, K. Bogdanov, M. Holcombe, and S. Salahuddin. “Reverse engineering state machines by interactive grammar inference”. WCRE’07, 2007.

18. Walkinshaw et al - cont’d:Passive then Active Inference

19. Walkinshaw et al - cont’d:New approach uses LTL • The developer suggests some constraints in Linear Temporal Logic • An iterative process

20. LTL review • Next x • x has to hold in the next state • Global x • x has to hold for every future state • Eventually x • x has to hold eventually • Until x U y • x has to hold until y • Release x R y • y is true until x becomes true • x releases y • You can’t load again unless you close first

21. Walkinshaw et al: Sample Sequence of Reverse Engineering Steps

22. Walkinshaw et al: Results • Having the user be able to specify LTL constraints reduces the number of questions asked by 50-70% • But: • Reverse engineers must be fluent in temporal logic • Maybe a better interaction method can be defined to allow LTL entry in a simpler form? • Graphical? Patterns? • System knowledge is still needed

23. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

24. State of the art: State Machine Modeling Tools and Code Generation

25. State of the art: URLs of KeyState-Machine Generating Tools • http://fsmgenerator.sourceforge.net/ • http://www.smartstatestudio.com/ • http://www.mentor.com/products/sm/model_development/bridgepoint/ • http://www.telelogic.se

26. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

27. State of the Art:Umple, a Modeling Infrastructure • Sponsored by IBM CAS Ottawa • Project has one more year of current sponsorship • A textual language in which • UML concepts appear like programming language concepts • Modelling can be performed • Fully generated code that needs no round-tripping • Programming can be done as normal • Normal Java code, just incremented with UML abstractions • Still under development, but robust enough for real development • Current grammar is at: http://cruise.site.uottawa.ca/umple/UmpleGrammar.html

28. Umple continued • Associations: • class X { • } • class Y { • 1 -- * X; • } • Constraints maintained: • Referential integrity • Multiplicity • Umple is written in itself • Online demo http://cruise.site.uottawa.ca/umpleonline/

29. State Machines in Umple • class X { • SM1 { • S1 { • e1 / {callMethod();} -> S2;} • S2 { • e1 -> S1;} • } • } • class Y { • Boolean b = true; • SM2 { • S3 { • e2 [b] -> S4;} • S4 { • e3 -> / {setB(false);} S3;} • } • }

30. A larger sample of state machine code pedestrianFlow { FourWayStop { entry / {P1Light = DoNotWalk; P2Light = DoNotWalk;} initiateRegularOperation -> P2Go; } P1Go { entry / {P1Light = Walk;} afterSeconds(P1GoTime) -> P1Halting; detectMalfunction -> FourWayStop; } P1halting { entry / {P1Light = Flashing;} afterSeconds(P1GoTime) -> P2Go; detectMalfunction -> FourWayStop; } P2Go { entry / {P2Light = Walk;} afterSeconds(P2GoTime) -> P2Halting; detectMalfunction -> FourWayStop; } P2Halting { entry / {P2Light = Flashing;} afterSeconds(flashingTime) -> P1Go; detectMalfunction -> FourWayStop; } } directionOneLight { FlashingRed, Red, Yellow, Green } directionTwoLight { FlashingRed, Red, Yellow, Green } P1Light { DoNotWalk, Flashing, Walk } P2Light { DoNotWalk. Flashing, Walk } } • class IntersectionTimerDriven { • Integer directionOneGoTime; • Integer directionTwoGoTime; • Integer yellowTime; • Integer P1GoTime; • Integer P2GoTime; • flow { • FourWayStop { • entry / {directionOneLight = FlashingRed; • directionTwoLight = FlashingRed; } • initiateRegularOperation -> DirectionOneGo; • } • DirectionOneGo { • entry / {directionOneLight = Green; • directionTwoLight = Red; } • afterSeconds(directionOneGoTime) -> • DirectionOneHalting; • detectMalfunction -> FourWayStop; • } • DirectionOneHalting { • entry / {directionOneLight = Yellow; } • afterSeconds(yellowTime) -> DirectionTwoGo; • detectMalfunction -> FourWayStop; • } • DirectionTwoGo { • entry / {directionOneLight = Red; • directionTwoLight = Green; } • afterSeconds(directionTwoGoTime) -> • DirectionTwoHalting; • detectMalfunction -> FourWayStop; • } • DirectionTwoHalting { • entry / {directionTwoLight = Yellow; } • afterSeconds(yellowTime) -> DirectionOneGo; • detectMalfunction -> FourWayStop; • } • }

31. Variability Modeling in Umple • Allows • Creating a product family • Putting together a product from sets of features • http://cruise.site.uottawa.ca/umpleonline/vml.html

32. Why use Umple for this Project 1 • Allows for • Arbitrarily nested and concurrent states • Interruptible do activities • Multiple state machines per class • Reusable state machines • Product families of state machines • Definition of model patterns • Separation of diagrams from model • Aspects and mixins

33. Why use Umple for this Project 2 • Integrated with Eclipse • Works like any other compiler • Generates multiple languages • Under our control so we can experiment • Textual • So we can create, generate and edit sophisticated models easily • So we can apply sophisticated tools to analyse and transform them • But text is not XML, so merging and analysis can be easier • Graphical so we can visualize state machines • Integrate state and class diagrams tightly

34. Agenda • Current status • Quick overview of subproject intent • State of the art • Verification and analysis using state machines • Reverse engineering from traces • State machine generation and modeling • Umple: A modeling infrastructure • Plans and potential directions

35. Students to be Involved • Directly on project (starting in January) • Hamoud Aljamaan, PhD - Scholarship student • Enhacing modeling tool Umple to facilitate tracing • Mahmoud Orabi, PhD - Funded by project • Reverse engineering traces to state machines • Working on supporting technologies • Ahmed Orabi, PhD student starting in January, NSERC funds • Metamodeling for tracing • Omar Badreldin, PhD student 30% completed, IBM funds • State machine modeling and code generation • Andrew Forward, PhD student 90% completed, IBM funds • Overall Umple architecture • One more masters student is expected to start in May • Also scholarships supported • Making Umple generate for with C++ / Erlang, etc. • Undergrad students working on IDE

36. Near-Term Plans:Develop and Enhance Tracable Models • Manually model some existing systems of interest to Ericsson or DND • Most effective do this in Umple • If we have source code we can try re-factor to Umple • Refactor to associations and state machines • Umple can be made to generate C++, Erlang, etc. • We have a lot of experience refactoring to Umple with associations • Learning to do it with state machines would be an interesting exercise

37. Near-term Plans: Reverse Enginering from Traces to State Machines • Attempt to replicate (by borrowing software?) existing semi-automated reverse engineering techniques • Target language would be Umple • But easy to translate to and from other targets • New research questions not addressed by current research: • Reverse engineering to concurrent state machines and nested state machines • Full reverse engineering of correct guards, actions, and activities • Can we make it work with truly huge traces?

38. Near-term plans:Deepen our literature review • The new students who will be starting will be tasked with this

39. Task for a student • Work with Debrief software • As provided by Mario Couture • One of several case studies we will need to develop • Instrument and trace • Manually model prior to attempting to automatically model • Generate code from the model for further experimentation

40. Other general project tasks • Meet more with Ericsson and DND to better understand • Perspectives • Requirements • Target examples • Investigate synergies with IBM • They are interested in Umple, but would like to see a customer interested in it before investing