Systematic research of Combinatorial Effects at Requirements Engineering Level

1. Systematic research of Combinatorial Effects at Requirements Engineering Level Jan Verelst Alberto Rodrigues Silva Herwig Mannaert David Almeida Ferreira Philip Huysmans

2. Overview • Introduction • Normalized Systems Theory • IdentifyingCombinatorialEffects • BPMN • UML Use Cases • “Real World” • UML Domain Models • DEMO/EO • Conclusions • Research agenda

3. Introduction • The origin: Sabbatical Alberto Silva, specialized in Requirements Engineering (RE) • The idea: toapplyNormalized Systems Theory (NS) toRE. CanRE beconsidered in terms of modularstructures ? Or is thistootechnicalandthereforeinappropriate ? • Jorge Sanz’ talk: bring EE to mainstream management ! • “Together with communications theory-based approaches, such as DEMO, this would suggest that the real world is first and foremost an area of human behavior, which should therefore not predominantly be studied by theories based on computer science and/or automation. We agree with this point of view. Nevertheless, in modern society, human behavior increasingly takes place in highly structured, process- based contexts. Therefore, we argue that it is relevant to study these aspects of reality based on concepts such as modularity, while at the same time making an abstraction from purely human and communication aspects.”

4. Software RequirementsSpecs • Software RequirementsSpecification (SRS) • A requirements specification is used throughout different stages of the project life-cycle, namely to help sharing the system vision among the main stakeholders, as well as to facilitate their communication, the overall project management, and system development processes. • Benefits • Establishes the basis for agreement between the customers and the suppliers on what the system is expected to do; • reduces development efforts; • provides a basis for estimating costs and schedules; • provides a baseline for verification and validation; facilitates the system deployment;and • serves as a basis for future maintenance activities

5. Business and System Level • Business level • Constructs • Terminology, goals, stakeholders, business processes, business use cases • Languages/Models • Goal-orientedlanguages (i*, KAOS), UML, BPMN, RUP Business Modeling, DEMO/EO, Archimate • System level • Context models • Constructs • System, subsystem, componenents, nodes, external actors • Languages • SysML Block Diagrams, UML Deployment Diagrams, Data Flow Diagrams • Domain Models • Constructs • Entities, classes, … • Languages • UML Class Diagrams, EntityRelationshipDiagrams • Functionalrequirementsmodels • Constructs • Actors, functionalrequirements, use cases, scenario’s, user stories • Languages • Natural languageenrichedwithmetadata (priority, risk levels…), UML Use Case diagrams, SysMLRequirementsDiagrams • Qualityattributemodels • Constructs • Qualities, metrics, utilityvalues, • Languages/Models • lists of qualityattributes, quality-attributes scenario’s

6. Whystudy CE at RE-level ? • In theory • The importance of evolvabilityin RE is sometimesoverlooked • OO: anthropomorphism •  Simsion et al.: analysis = creative design activity • In practice • Inabilitytoevolvemay lead tomisalignmentsbetweenrequirementsand information systems • Requirementsoftenconstitutedocumentation => out-of-date • RE requiresconsiderable resources => inefficient

7. About NS Theory • A theoreticalframeworkinvestigatingModular StructuresunderChange • Based on Systems Theoreticconcepts •  Completely independent of anyframework, programminglanguage, package, … • Has showntobeableto deal with the challenge of increasingcomplexity • E.g. hardware, Internet, spaceindustry… • Initial scope: Modular Structures in Software Architectures • Publications: book, >50 conference proceedings, (invited) lectures at different universities… • Education: undergraduate, postgraduate…

8. NS @ software level Struct Customer - Name - Address - VATnr - … Struct Invoice - Nr - Date - … Struct IMPACT Func Func sendInvoice FunccomputeInvoice N Func inviteCustomer IMPACT IMPACT IMPACT N

9. NS Principles • Modularity x Change  CombinatorialEffects (CE) ! • CE = (hidden) couplingor dependencies, increasingwithsize of the system ! • NS Principlesidentify CE at seeminglyorthogonal levels • SoC: Whichtasks do youcombine in a single module ? • “An action entitycanonlycontain a single task.” • DVT: How do youcombine a data and action module ? • “Data entitiesthat are received as input or produced as output by action entities, needtoexhibitversiontransparency.” • AVT: How do youcombine 2 modules ? • “Action entitiesthat are calledbyother action entities, needtoexhibitversiontransparency.” • SoS: How do youcombine modules in a workflow ? • “The calling of an action entitybyanother action entityneedstoexhibit state keeping.” • CE explainLehman’sLaw of IncreasingComplexity !

10. NS Summary Assumption: Modular Structures: Complexity ↑ X Change ↑ • NS-Determinism • 1. Modularity • - No Combinatorial Eff. • - Black box reuse • - Lehman controlled • 2. Standardization • - Expansion • Current Practice • 1. Modularity • - Combinatorial Eff. • - White-box reuse • - Lehman • 2. Standardization • - No expansion Fine-grained/ No Combinatorial Eff. Expansion Black-box reuse/ Lehman controlled Lehman

11. NS as a simpletransformationover F/C gap Tasks computeInvoice inviteCustomer sendInvoice F Customer -Name -Address -VATnr … Invoice -Nr -Date -… Data Change: addAttribute Struct Customer - Name - Address - VATnr - … Struct Invoice - Nr - Date - … Struct IMPACT C Func Func sendInvoice FunccomputeInvoice N Func inviteCustomer IMPACT IMPACT N IMPACT

12. The concern trapezoid • Examples of concerns: • “Want innovativeinvoicing” • High-level business • Strategy (innovationvscost) • Internationalisation • High-level ICT • Stick withcurrent package • Commodity ICT • Human • Stakeholder issues (political…) • Communication, negotiation Business F Concerns are additive (?)  #concerns ↑↑ C • Examples of concerns: • Low level ICT: • Performance of analgorithm or call to package • Interface stability • Internationalisationlibraries present • Technical security details • … ICT

13. Bridging a F/C gap • An ill-structured (or wicked) design problem • Incomplete andambiguousspecification of the problem; • No deterministicpathto solution; • Knowledge of severaldomainsneedstobeintegrated in order tofind a solution; • No clear criteria te compareandevaluatepossiblesolutions. • Characteristics • M:N, not 1:1 • Integration = Fragile/Non-lineair behavior: 5% extra reliability totally different architecture • Integration = sometimesall-or-nothing • ALL concerns needtobeseparated at compile/deploy/runtime, or the remainingcoupling • May make the artifacttotallyuseless in practice ! • Solvingthisrequires a whitebox-perspective without complexityreduction !

14. NS as a complex transformationover F/C gap IMPACT Tasks computeInvoice inviteCustomer sendInvoice IMPACT F Invoice Element Customer -Name -Address -VATnr … Invoice -Nr -Date -… IMPACT Data Change: addAttribute Customer Element Data Elements C invite Customer Element compute Invoice Element send Invoice Element Action Elements

15. Enterprise = n F/C gaps F F F F F C C C C C Strategy NS @ Enterprise= Normalized Transformation Over the F/C gaps RW PPM/EA DEMO/EO PM RE/Analysis (Alberto Silva’s group) Use Cases Domain Models NS@ Software Design BPMN Implementation Increasing modular structure Maintenance

16. Enterprise = n F/C gaps F C F F Strategy NS @ Enterprise= Normalized Transformation Over the F/C gaps RW C C PPM/EA DEMO/EO PM RE/Analysis (Alberto Silva’s group) Use Cases Domain Models NS@ Software Design BPMN Implementation Increasing modular structure Maintenance

17. BPMN models

18. BPMN-createExpenseReimbursement

19. BPMN Real World createExpenseReimbursement Real World createBonusPayment N F Change: checkAccountExistencev2 checkAccountExistence is shared createBonusPayment createExpenseReimbursement N C BPMN Task IMPACT N IMPACT IMPACT

20. BPMN-withseparatedTask Real World createExpenseReimbursement Real World createBonusPayment N F Change: checkAccountExistencev2 checkAccountExistence is shared N createBonusPayment createExpenseReimbursement C checkAccountExistence Remark:this is NOT an NS element ! BPMN Task IMPACT

21. BPMN • PhD Dieter Van Nuffel containsexamples of CE regardingSoCand 25 guidelinestoeliminatethem • Modular structure? • “Easy, obvious!” • Constructs? • All BPMN constructs… • CE ? • Violationsof SoC, SoSmayoccur • applicationof AvTandDvt is lessclear • Implications • Evolvability of BP is limited •  popular claim of BPM-engines, even thoughthey do addevolvability at the software-level !

22. UML Use Cases

23. UML Use Cases Use Case createExpenseReimbursement 3. checkAccountExistence 4. createAccount … 7. reimburse Use Case createBonusPayment 3. checkAccountExistence 4. createAccount 5. executePayment

24. UML Use Cases Real World Interviews (oral) Interview transcripts F Change: checkAccountExistencev2 createExpenseReimbursement createBonusPayment N C Use Cases IMPACT IMPACT N IMPACT

25. UML Use Cases-with hypertext construct Real World Interviews (oral) Interview transcripts F Change: checkAccountExistencev2 N createExpenseReimbursement createBonusPayment checkAccountExistence C Remark:this is NOT an NS element ! Use Cases IMPACT

26. Use Cases • Modular structure ? • Constructs ? • YES • Name of the use case => primitive interface of the module • Pre- and post-conditions => delineatefunctionality of the module • Workflow (tekst) => functionality details of the module • Other: A scenario, anexception, a trigger, a stakeholder, … • NO • Text-baseduse cases allowforGOTO’s, ambiguities… • Hypertext construct notalwaysavailable in tooling ! • CE ? • Use cases are usuallytoounderspecifiedtoallowdetailed analysis of CE • Violations of SoCmayoccur • application of AvTandDvt is lessclear: do use cases have interfaces ? • Implications • Evolvability of Use Cases is limited • This is a well-known issue, particularly in large projects, • Maintainingdocumentationbecomesexpensive • Use Cases does notnecessarily document the code (when the code itself is changed)

27. Real World

28. Example 1: createExpenseReimbursement Real World Interviews (oral) Interview transcripts F Change: checkAccountExistencev2: “Use NL bank onlyfromnow !” Actor 2: createBonusPayment Actor 1: createExpenseReimbursement N C Manually Executed BP & IS (=paper) IMPACT IMPACT N IMPACT

29. Example 1: createExpenseReimbursement • Thisexamplecanbe 100% manual ! • Modular structure ? • Constructs? • Human actors executingformal/informal procedures • CE ? • Visible at runtime (resources): Violation of SoC ?

30. Example2: Exam Marks Real World Procedure: ‘ouruniversityapplies … rounding’ F Change: Procedure v2 Professor 2 Professor 1 N C Decentralized Execution of BP & IS IMPACT IMPACT N IMPACT

31. Example2: Exam Marks • Modular structure ? • Constructs? • Human actors executingformal/informal procedures • CE ? • Visible at runtime (resources): Violation of SoC ?

32. Example2: Exam Marks – Compile Time Model Real World Procedure: ‘ouruniversityapplies … rounding’ F Change: Procedure v2 ProcedureExecutedbyall Professors N C Decentralized Execution of BP & IS INVISIBLE IMPACT !!!

33. Example2: Exam Marks Real World Procedure: ‘ouruniversityapplies … rounding’ F Change: Procedure v2 Student Office C Centralized Execution of BP & IS IMPACT

34. Example 3: Mail distribution Real World Procedure: ‘ouruniversityallowsphysical mail’ F Change: Procedure v2 Secretary2 Secretary 1 N C Decentralized Execution of BP & IS IMPACT IMPACT N IMPACT

35. Example 3: Mail distribution • Modular structure ? • Constructs ? • Human actors executingformal/informal procedures • CE ? • Visible at runtime (resources): Violation of SoC ?

36. Example 3: Mail distribution Real World Procedure: ‘ouruniversityapplies … rounding’ F Change: Procedure v2 Centralized & virtual e-mail office N C Centralized Execution of BP & IS IMPACT

37. Real World • Modular structure ? • Constructs ? • Manual: actors, departments, manual databases, manual procedures, … • (IT-basedexecution): • CE ? • Violations of SoCmayoccur • Violations of SoC are close todiscussions in management literature on • ‘decentralizationvscentralization’ • ‘the needfor matrix organizations on top of departments’ • ‘the needfor business processes on top of departments’ • => EE andOrganizational Sciences meet ! • Remark: Organizational Sciences have swingingpreferences over time formany of these topics… • Implications • Enterprises, even manual ones, have limitedevolvability

38. UML OO Domain Models

39. UML OO Domain Models

40. OO Domain Models • Modular structures • See OO programming… YES ! • CE • Data: • Codd’snormalization… but is thissufficient ? • Functions: • Violation of DvT: use of atomic data types • Violation of SoS: Use of syncpipelines • Coupling • Is high coupling the reasonthat domain models are not in widespreaduse in practice ?

41. DEMO/EO

42. Are DEMO/EO modelsmodularstructures ? • A few indications, but we didnot focus on DEMO/EO specifically in this paper ! • Similaritiesbetween DEMO and NS • Separation of State in NS => P- and C-facts ! • Workflows in NS => structuredaggregations of actions into transactions • Expansion in NS => instantiating transaction pattern in DEMO • Do DEMO/EO-modelscontain CE ? • Possibly… • In production acts… • In implementation, but is this DEMO/EO ? • In runtimebehavior, but is this DEMO/EO ? • Nevertheless, we shouldfind out… seeconclusion !

43. The production act of a transaction seemstobe a module consisting of a number of tasks, detailed in the action models. However, foreachproduction act, there are 4 coordination acts  transactions are aimed at coordination-intensive problems, likeenterprisesconsisting of human actors. Actually, such transactions define the interfaces of the modules ! Reminds of negotiation at operational level, but also project level (~IS acceptanceproblems) Thisis why DEMO/EO worksso well: it is human modularity, which is usedto control complexity, and… DEMO transactions

44. Reducecomplexityby 70-90 % Byusing the transaction pattern, with the sameinternalstructure, forall transactions = similar approach toNS expansion ! DEMO transactions

45. Conclusion

46. CE exist at all these levels ! F C F F Strategy NS @ Enterprise= Normalized Transformation Over the F/C gaps RW C C PPM/EA DEMO/EO PM RE/Analysis (Alberto Silva’s group) Use Cases Domain Models NS@ Software Design BPMN Implementation Increasing modular structure Maintenance

47. Conclusions • Examples of CE’s are relativelystraightforwardbut • they are sufficient to illustrate the omnipresence of instabilities in a domain that is sometimes considered to be about "identification of objects in the real world”. • at the software, heuristicshave shown to be insufficient to control the large number of highly complex CEs that are responsible for the symptoms of Lehman’s law. • Some examples of CE’s correspond to technical advances • Eg. The shift from physical to e-mail • => this CE is no marginal detail ! • Implications • Initially, when the system is small, the CE’s would probably not be problematic, but over time their effects would grow and slowly but surely increase the rigidity of requirements models and specifications (which are sometimes used as the technical documentation of the information system, or a component in a legal contract concerning the system).

48. Conclusions - Research Agenda • Identification of CE at eachenterprise level at compile-, deployment- andruntime, as well as entropy-related issues • Mechanismsto control CE • Expansion/tooling (hypertext support in RE-tool?) • (semi-)manual mechanisms at E-level ? • Appropriate levels of control at eachenterprise level • The examples show that these CE exist, andmanyemployees will feel thattheyshouldberemoved => NS perspectiveon Enterprises is not ‘tootechnical’, but • at the same time: Enterprises remainhuman entities, andit is extremelyunlikelythattheyshouldbenormalizedto the sameextent as software !

49. Conclusions - Research Agenda • Approach: domain-dependentartifacts, such as in classical engineering ! • “loosely coupled artifacts need to be developed in areas such as finance, accounting, transport, human resources, or in subareas such as invoicing, staffing, project management, mail distribution, payments, etc.” • Then: “Whenthese artifactsare developed using a modular structure which exhibits control of coupling issues (such as a low number of CE), they can be aggregated into higher-order structures such as aninvoice.” • Ex: PhD Els Vanhoof: simpleproblem, no solution in 2013 !

50. Link to Business Meeting… This paper illustrateshowAntwerpand Lisbon wereabletocollaborate, in the context of Alberto Silva’s sabbatical ! Let’s continue this, and do more, because… “In this way, we support the call by Dietz et al. for the area of Enterprise Engineering to be developed [33]. The amount and complexity of issues that need to be solved to achieve the next generation of truly agile enterprises both in the service and industrial sector, both in the for-profit and not-for-profit sector, is such that a scientific basis focusing on structural issues (including coupling) will be required.” Thank you for your attention !