Effective Abstractions for Defining and Verifying Processes

1. Effective Abstractions forDefining and Verifying Processes Leon J. Osterweil (ljo@cs.umass.edu) Univ. of Massachusetts Amherst, MA 01003 USA URL: laser.cs.umass.edu UC Irvine 9 November 2001

2. What do we mean by “process” • Activities related to the development, verification, evolution, evaluation, management, etc. of software products • Examples of high level processes: • Develop requirements • Do Object Oriented Design • Formally verify • Examples of low level processes • Archive result of running a test case • Verify a lemma • Tend to be concurrent (coordination of people, systems) • Usually (regrettably) informal or undefined

3. Why the interest in process? • Superior quality from superior processes • Build quality in, don’t “test in” quality (manufacturing) • Many observed “process errors” • Real processes are intricate, have intricate interconnections • Machines can help people with process complexities • Process effectiveness from human/machine synergy

4. Approaches to Process Definition • Process Definition may have different goals • Communication • Coordination • Intuitive understanding • Verification • Training • Automation • Deep understanding • Etc. • Different definition formalisms support different of these goals to different degrees • Formalisms vary in rigor, precision (semantic detail), semantic scope, clarity

5. Our Premise: Processes are software

6. Consist of Requirements, Specification, Design, Code, etc. Process Requirements --Key to designing suitable process --Basis for evaluation and improvement of process Process Specification/Modeling/Design --Helps conceptualization, communication, visualization --Can be management aid Process Code --Provides rigor and complete details --Basis for execution/tool support and integration Process Measurements and Evaluation --Results of Static Analysis and Dynamic Measurement --Basis for.... Process Maintenace (aka Process Improvement) Software Processes as Software Software processes should be developed using a (Software development process) development process

7. Process Representation Approaches • Natural and Structured Natural Language • Pictorial representations often used --DFD’s, FSA’s, etc. • Programming language approaches can be used too • Process aspects often modelled --Behavior modelling --Procedure modelling --Dataflow modelling --Product modelling --Role/actor/agent modelling ----some integrate, interrelate several Directly analogous to product definition formalism approaches

8. Pictorial Approaches • Support intuition about processes • Tend to have weak semantics • Weak support for reasoning • Examples • Data flow diagrams • Petri Nets • Finite state machines • Control flow diagrams • Combinations of the above

9. Petri Net Requirements specification process Developers Users Real World JSD Interview RW_Desc Develop Spec Sys_Spec_Diag Develop Implementation Sys_Impl_Diag + Sys_Spec_Diag Design_Spec

10. Decomposition of a part of the Rqts. process RW_Model Model_System RW_Model Identify_Model_Process Model_Process_ Name_List Connect Connection_List + Model_Process_Name_List Specify_Model_Process Model_Process_List RW_Model Init_Sys_Spec_Diagram

11. Req_Spec Design Process Petri net BOOD Req_Spec Identify_Object Objects States Identify_Operations Objects Operations States Establish_Visibility Objects States Operations Visibility Establish_Interface Interface Create_Implementation Implementation Interface Create_Design_Spec Design_Spec Design_Spec

12. Req_Spec More Identify_Object Req_Spec Identify_Nouns Nouns + Req_Spec Identify Actor Identify Concrete Object Identify Abstract Object Identify Agent Identify Server concrete object Agent Actor server Abstract Object Identify_Class Objects Classes Identify_Attributes Objects States Objects States

13. Problems with Petri Nets • Hard to define artifact flow • Hard to specify complex exception management • Petri Nets are not inherently hierarchical • Hard to specify agents, resources

14. Data Flow Diagram ofTraditional Waterfall Model Requirements High-LevelDesign Low-LevelDesign Code Test

15. With Rework Requirements High-LevelDesign Low-LevelDesign Code Test

16. With More Rework Requirements High-LevelDesign Low-LevelDesign Code Test

17. The Trouble with Dataflow Diagrams Where does output go? Requirements What to do when reviews fail? What causes this rework? High-LevelDesign What portion of activity should bedone? Low-LevelDesign Code Test How do we break this cycle?

18. Representing Processes with Code • Uses languages with execution semantics • Provides details and elaborations upon process design • Forces grappling with details omitted from model/design • Vehicle for meshing process control with product data at arbitrarily low levels of detail • Provides superior visibility enabling better control • Basis for better predictability • Basis for process execution • Blueprint for tool integration

19. Variety of Approaches to CodeFormalisms • More traditional coding languages: --Procedural (Sutton's Appl/A) --Rule-based (Kaiser's Marvel) --Functional Hierarchy (Katayama’s HFSP) --Law based (Minsky) --Object Oriented (schema definition languages) • Key issue: developing abstractions to facilitate process definition

20. (a) JSD(Real_World | Design_Spec) => (1)Develop_Spec(Real_World_Desc |System_Spec_Diagram) (2)Develop_Impl(System_Spec_Diagram |System_Impl_Diagram) (3)WhereReal_World_Desc = Interview(Users, Developers,Real_World) (4) Design_Spec = union(System_Spec_Diagram, System_Impl_Diagram) Second_level (b) Develop_Spec(Real_World_Desc |System_Spec_Diagram) => (1)Develop_System_Model(Real_World_Desc |Real_World_Model, Init_System_Spec_Diagram) (2)Develop_System_Func(Real_World_Model, Init_System_Spec_Diagram |System_Spec_Diagram) Third_level (c) Develop_System_Model(Real_World_Desc |Real_World_Model, Init_System_Spec_Diagram) => (1)Model_Reality(Real_World_Desc |Real_World_Model) (2)Model_System(Real_World_Model |Init_System_Spec_Diagram) (d) Develop_System_Func(Real_World_Model, Init_System_Spec_Diagram |System_Spec_Diagram) (1)Define_Func(Real_World_Model, Init_System_Spec_Diagram |System_Function, Function_Process) (2)Define_Timing(Init_System_Spec_Diagram, System_Function |Timing) (3)Where System_Spec_Diagram = is_composed_of(Init_System_Spec_Diagram, System_Function, Function_Process, Timing) Fourth_level (e)Model_Reality(Real_World_Desc | Real_World_Model) => (1)Identify_Entity_Action(Real_World_Desc | Entity_Action_List) (2)Draw_Entity_Structure(Entity_Action_List | Entity_Structure_List) Where Real_World_Model = is(Entity_Structure_List) Real_World_Process = is(Entity_Structure) (f) Model_System(Real_World_Model | Init_System_Spec_Diagram) => (1)Identify_Model_Process(Real_World_Model | M_Proc_Name_List) (2)Connect(Real_World_Model, M_Proc_Name_List | Connection_List) (3)Specify_Model_Process(Connection_List, Real_World_Model, M_Proc_Name_List |Model_Process_List) (4)Where Init_System_Spec_Diagram = is(Model_Process_List) (5)Connection = is(State_Vector) or is(Data_Stream) HFSP design model

21. HFSP design model (a) JSD(Real_World | Design_Spec) => (1)Develop_Spec(Real_World_Desc |System_Spec_Diagram) (2)Develop_Impl(System_Spec_Diagram |System_Impl_Diagram) (3)WhereReal_World_Desc = Interview(Users, Developers, Real_World) (4) Design_Spec = union(System_Spec_Diagram, Syst em_Impl_Diagram) (b) Develop_Spec(Real_World_Desc | System_Spec_Diagram) => (1)Develop_System_Model(Real_World_Desc | Real_World_Model, Init_System_Spec_Diagram) (2)Develop_System_Func(Real_World_Model, Init_System_Spec_Diagram |System_Spec_Diagram)

22. More Elaboration (b) Develop_Spec(Real_World_Desc |System_Spec_Diagram) => (1)Develop_System_Model(Real_World_Desc | Real_World_Model, Init_System_Spec_Diagram) (2)Develop_System_Func(Real_World_Model, Init_System_Spec_Diagram |System_Spec_Diagram) (d) Develop_System_Func(Real_World_Model, Init_System_Spec_Diagram |System_Spec_Diagram) (1)Define_Func(Real_World_Model, Init_System_Spec_Diagram |System_Function, Function_Process) (2)Define_Timing(Init_System_Spec_Diagram, System_Function | Timing) (3)Where System_Spec_Diagram = is_composed_of(Init_System_Spec_Diagram, System_Function, Function_Process, Timing)

23. (a) BOOD(Req_Spec | Design_Spec) => (1) Identify_Object(Req_Spec | Objects, States) (2) Identify_Operations(Req_Spec, Objects, States | Operation) (3) Establish_Visibility(Req_Spec, Objects, States, Operation | Visibilty) (4) Establish_Interface(Visibility, Objects, States, Operation | Interface) (5) Establish_Implementation(Interface | Implementation) (6) Where Design_Spec = is_composed_of(Interface, Implementation); Second Level (b) Identify_Object(Req_Spec | Objects, States) => (1) Identify_Nouns(Req_Spec | Nouns) (2) Identify_Concrete_Object(Req_Spec, Nouns | Concrete_Object) (3) Identify_Abstract_Object(Req_Spec, Nouns | Abstract_Object) (4) Identify_Server(Req_Spec, Nouns | Server) (5) Identify_Agent(Req_Spec, Nouns | Agent) (6) Identify_Actor(Req_Spec, Nouns | Actor) (7) Identify_Class(Req_Spec, Agent, Server, Actor, Concrete_Object, Abstract_Object | Class) (8) Identify_Attributes(Objects | States) (9) Where Objects = union(Concrete_Object, Abstract_Object, Class, Agent, Actor, Server) (c) Identify_Operation(Req_Spec, Object, States | Operation) => (1) Identify_Suffered(Req_Spec, Object, States | Operation_Suffered) (2) Identify_Required(Req_Spec, Object, States | Operation_Required) (3) Defining_Time_Order(Req_Spec, Operation | Time_Order) (4) Defining_Space(Req_Spec, Operations | Space) (5) Where Operation = union(Operation_Suffered, Operation_Required) (d) Establish_Visibility(Req_Spec, States, Operation | Visibility) => (1) Specify_Object_See(Objects | Objects_See) (2) Specify_Object_Seen(Objects | Object_Seen) (3) Where Visibility = union(Objects_See, Object_Seen) (e) Establish_Interface(Visibility, Object, States, Operations | Interface) => (1) Derive_Module(Object | Module) (2) Specify_Attr(States, Module | Attributes) (3) Specify_Proc(Operations, Module | Procedures) (4) Specify_Visibility(Visibility, Module | Visibility_Spec) (5) Where Subsystem = is_in_terms_of(Module), (6) Interface = is_composed_of(Attributes, Procedure, Visibility_Spec); HFSP OO Design Model

24. Process definition strains programming languages severely • Some issues • More pervasive dynamism • Complex reactive control • State reification • Novel transaction notions • Need for new abstractions • Specification of resources • Real time specification • More central role for humans • Need to mix paradigms • Language design grows from new challenges • better languages, and not only for processes

25. Questions Where does output go? Requirements What to do when reviews fail? What causes this rework? High-LevelDesign What portion of activity should bedone? Low-LevelDesign Code Test How do we break this cycle?

26. Little JIL • Exploring effective process abstractions • Clear, precise definition of realistic processes • Graphical language with execution semantics • Expresses process coordination • Designed to be used interchangeably with other product, resource, scheduling factors • Visual JIL is the graphical interface to Little JIL

27. Little-JIL Step Notation Interface Badge Prerequisite Badge Postrequisite Badge TheStepName Z X Sequencing Handlers Reactions

28. Little-JIL Example:“Smart” Regression Test Process RegressionTest GetArtifacts ReportResults Stop PerformTest SelectTests Stop PerformTest Report Failure ExecuteTest GetExecutable GetTest Cases NoteFailure Compare Results Get Input Data Run Test Get Expected Output Data

29. The BOOD (Booch OO-Design) Process as an example What BOOD is: • Notation • Class diagrams, state transition diagrams, interaction diagrams, ... • Method: • Identify classes and objects • Identify semantics of classes and objects • Identify relationships between classes and objects • Specify the interface and implementation of the classes and objects

30. What BOOD Doesn’t Address • Only describes the “nominal” process • Doesn’t address how to handle non-nominal conditions (exceptions) • Only describes “design-in-the-small” • Design “in the large” is not addressed • General lack of precision • Does not address collaboration • Much design is done by teams • Does not address configuration management • How to handle design evolution

31. Sequential In order, left to right Parallel Choice Choose from Agenda Try Proactive Flow Sequencing Kinds

32. The Nominal Process • Using parallel and sequential step kinds Booch Booch Identify_Classes_and_Objects Implement Identify_Relationships Classical Use Case Behavioral Identify_Semantics

33. More Control Constructs Implement Reuse_Implementation Custom_Implementation Look_for_Inheritance Look_for_Parameterized_Class Look_for_Objects_to_Delegate_to

34. Scoped Exception Handing • All steps may contain one or more exception handlers • React to exceptions thrown in descendent steps • Handlers are steps themselves InterfaceFilesDontCompile DevelopInterfaceFiles InterfaceFilesCompile

35. Four different continuations on exception handlers • Complete • Handler was a “fixup” and now it is OK to go back • Continue • Handler brought step to an acceptable postcondition state and it is OK to go on • Restart • SNAFU. Handler cleaned up mess, now OK to redo • Rethrow • Go up to parent and hope the parent knows what to do

36. Prerequisites and Postrequisites • Test before/after execution of a step • Requisites are themselves entire steps • Program what to look for, in what order, and distinguish among responses for various contingencies • Uses exception handling to respond to failures

37. Assessing Milestones Using Postrequisites • In BOOD, defined as conditions on the product • In Little-JIL, represented with postrequisites on steps InterfaceFilesDontCompile DevelopInterfaceFiles InterfaceFilesCompile

38. Real Time Specification • Each step may have a deadline specification • Exception thrown when step execution exceeds deadline • Scheduling algorithms can detect when a step’s substep structure is unschedulable • Exception is thrown

39. Examples of Resources • Input artifacts: requirements document, locks on key artifacts • People: designers with varying skills • Tools: ROSE • Agents: Each step has a distinctly identified unique resource responsible for execution of the step (and all of its substeps)

40. Bob Resource Human Design Team Carol Hardware Ted Software Alice Data Manager PC Class Sparc Designer Schedulable Class Instance Resource Model:Is-A Relationships

41. Bob Resource Human Design Team Carol Hardware Ted Software Alice Data Manager PC Sparc Designer Resource Model:Requires Relationships

42. Bob Resource Human Design Team Carol Hardware Ted Software Alice Data Manager PC Sparc Designer Resource Model:Whole-Part Relationships

43. An Example Resource Model

44. Resource Request Example Agent: BOODDesigner;expert tool: ClassDiagramEditor artifact: DiagramReposLock IdentifyRelationships SpecifyRelationships RefineRelationships Resource request is a query on the Resource specification repository

45. Artifact Flow Booch new_ident new_concept Identify_Classes_and_Objects Implement new_ident new_concept new_concept Identify_Relationships Identify_Semantics

46. Remember These Questions Where does output go? Requirements What to do when reviews fail? What causes this rework? High-LevelDesign What portion of activity should bedone? Low-LevelDesign Code Test How do we break this cycle?

47. High-Level Process

48. Requirements Details

49. Design Details

50. Requirements Rework