Design science methodology for a PhD project

1. Design science methodology for a PhD project Roel Wieringa IS Seminar 31st August

2. Goal • Discuss one of our research projects • Subject • Methodological structure • Relate this to your project IS Seminar 31st August

3. 1. Subject • Project goal: • Extending enterprise architecture design with goal-oriented requirements engineering • Partners: • BizzDesign (Wilco Engelsman) • Novay (Dick Quartel) • UT (Roel Wieringa) IS Seminar 31st August

4. Work done so far KAOS and i* and Tropos and causal loop diagrams and .... Fusion: reasons to add elements ARMOR Reduction: reasons to delete elements Light Armor IS Seminar 31st August

5. Result (roughly) influence decomposition Whatever is observable to stakeholders Phenomenon Desire is phenomenon ideal for a stakeholder Is-a Stakeholder Desire conflict Is-a Is-a Goal is desire for which budget is made available Goal Is-a If the goal is a system property, it is called a requirement Requirement System property Requirements must be satisfied by observed system property IS Seminar 31st August

6. Status • Concepts are stable for now ... Fixed point in improvement iterations • Still to look at action goals and state goals • Measure of preferences? • Relationships require a lot more work • Influence: state→state, action →state, action →action. • Use + and – of causal loop diagram for state → state? • Decomposition: analytic and synthetic • Conflict: logically impossible, physically impossible, technically impossible, violation of standards, socially impossible, humanly impossible, too expensive ... • We will do some empirical work before we return to this IS Seminar 31st August

7. 2. Methodological structure Engineering cycle (a.k.a. regulative cycle): • Problem investigation • Treatment design • Treatment validation • Implementation (transfer to Bizzdesign) • Implementation evaluation “Solution” Artefact Technique Method .... IS Seminar 31st August

8. Methodological structure of PhD thesis • Problem investigation • Treatment design • Treatment validation • Implementation (transfer to Bizzdesign) • Implementation evaluation IS Seminar 31st August

9. What did we do so far • Problem investigation: TBD • Treatment design: light Armor • Treatment validation: TBD • Implementation (transfer to Bizzdesign) • Implementation evaluation IS Seminar 31st August

10. Methodological options • Problem investigation • What we know now: Mging large # of reqs, tracing EA to reqs, determining impact of req change or EA change are all problematic • To do: survey • Among BizzDesign consultants? BizzDesign customers? • In-depth interviews? • Questionnaire • Treatment design: light Armor • Treatment validation: TBD IS Seminar 31st August

11. Initial problem understanding Initial treatment design Improved problem understanding Improved treatment design Treatment validation IS Seminar 31st August

12. Methodological options • Problem investigation: Some survey • Treatment design: light Armor • Treatment validation: TBD IS Seminar 31st August

13. Methodological options • Problem investigation: Some survey • Treatment design: light Armor • Treatment validation questions • Internal validity: LArmor in BizzDesign consultancy mitigates identified problems? • Trade-offs: All of LArmor? • Sensitivity: All of BizzDesign concultancy? Only Bizzdesign consultancy? IS Seminar 31st August

14. Methodological options • Problem investigation • Treatment design: light Armor • Treatment validation methods • Simulation: Wilco uses LArmor with toy example • Simulation: Wilco uses LArmor with past project data • Action research: Wilco uses Larmor in a project • Opinion research: Wilco elicits opinion of Bizzdesign consultants about Larmor concepts & method • Experiment: Wilco studies BizzDesign consultants using Larmor on an artificial problem, compares to control group using current BizzDesign method • Case study: Wilco studies BizzDesign consultants using Larmor in a project IS Seminar 31st August

15. Methodological issues • Generalization • Applicable to similar GORE methods? • Applicable to similar companies/projects? • Theory • A theory is a dictionary plus a statement with scope of application; mechanism counterfactually supported • Preferably, statement indicates mechanism • Scope = generalization; always uncertain, possibly limited • Mechanism • Counterfactual • Theories applied? • Theory that will explain case study observations? IS Seminar 31st August

16. Organizational design science (Van Aken) • Elaborate problem investigation (diagnosis) • Like a physician • Artefact design science (Wieringa) • Elaborate treatment design and validation • Like a medicine manufacturer IS Seminar 31st August

17. Discussion • What about your project? • Written presentation structure? • Problem – treatment – validation, with varying emphases • Oral presentation structure? • Table of contents of two items • Context: the stage • Goals: what are you going to do • Why interesting for audience • What is new wrt related work • Substance 1: the version that everyone can understand • Substance 2: brief indication of some technical detail • Please keep your slides readable: minimum point size is 18 IS Seminar 31st August

18. Discussion • What about your project? • Written presentation structure? • Problem – treatment – validation, with varying emphases • Oral presentation structure? • Table of contents of two items • Context: the stage • Goals: what are you going to do • Why interesting for audience • What is new wrt related work • Substance 1: the version that everyone can understand • Substance 2: brief indication of some technical detail • Please keep your slides readable: minimum point size is 18 IS Seminar 31st August

19. Discussion • What about your project? • Written presentation structure? • Problem – treatment – validation, with varying emphases • Oral presentation structure? • Table of contents of two items • Context: the stage • Goals: what are you going to do • Why interesting for audience • What is new wrt related work • Substance 1: the version that everyone can understand • Substance 2: brief indication of some technical detail • Please keep your slides readable: minimum point size is 18 IS Seminar 31st August