1 / 19

Sundar Gopalakrishnan, Guttorm Sindre, and John Krogstie:

Sundar Gopalakrishnan, Guttorm Sindre, and John Krogstie: Adapting UML activity diagrams for mobile work process modelling: Experimental comparison of two notation alternatives. PoEM 2010 Delft, 9.-10. Sept. Agenda. Motivation Background Research method Results

zorion
Download Presentation

Sundar Gopalakrishnan, Guttorm Sindre, and John Krogstie:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sundar Gopalakrishnan, Guttorm Sindre, and John Krogstie: Adapting UML activity diagrams for mobile work process modelling: Experimental comparison of two notation alternatives PoEM 2010 Delft, 9.-10. Sept.

  2. Agenda • Motivation • Background • Research method • Results • Discussion of threats to validity • Summary and further work

  3. Traditionalperspectives to modeling • Structural • Behavioral • Functional • Goal and rule-oriented • Object-oriented • Social communication • Actor/role-oriented Whataboutlocation/place/space... ?

  4. Zachman Framework

  5. Motivation • ’Where’ is increasingly relevant • Outsourcing • SupplyChainManagement/Logistics • Virtualorganization • Mobile applications and information systems • And it is possible to utilize ’where’ to a largerdegree (also real time) to know whereusers, equipment and goodsshould be, are or where at a certain time • Tracking (RFID, UWB, GPS, GSM, WiFi, Ultrasound…) • Internet ofThings (IoT) • This paper: • Presents somedifferentnotation alternatives basedon UML activity diagrams vs. Modeling mobile information systems • Summarizes an analyticalevaluation from a previouspaper (I-ESA’10) • Makes an experimentalcomparisonofthetwo most promisingones

  6. Possible notations looked at(I-ESA’10) • Standard UML, using annotation boxes to indicate context or location • Redefining swimlanes to indicate context / location • Using colour for context / location • These three were compared analytically, using a home care case supported by a mobile IS as an example

  7. Alt 1: Using annotation boxes • Advantages • Smallest deviation from standard UML AD • Enhanced understandability for those who already know AD • Disadvantages • Greatly increases # nodes in diagram (poor expressive economy) • May be confusing if you also need to use notes for something else in addition to context / location

  8. Alt 2: Swimlanes for where • Advantages • Shifting ”who” to stick figures: AD more uniform with UCD • Swimlanes intuitively indicate location • Disadvantages • Many lines from stick figures to activities, poor readability (would be even worse with bigger example) • New usage of swimlanes may confuse those already familiar with AD

  9. Alt 3.: using colour • Advantages • No increase in # nodes or lines, better readability • With two process design alternatives beside each other, it is easy to spot differences in location • Disadvantages • Larger deviation from standard UML • Must add legend • Possible challenge for colour blind users

  10. Analytical comparison Analytical comparison (I-ESA’10): • Alt 1. (trad. UML with annotations) and Alt 3. (Colours) came out as the two most promising • Proposed further work: Make experimental comparison

  11. Experimental design • Compare two diagram alternatives, annotation and colour • Controlled experiment looking at the participants’ • Performance using the notation • Opinion about the notation • Within-subjects design (Latin squares) • Controls better for selection bias • ”Doubles” the N • Measured variables: • Performance: • Understanding: score on 12 True/False questions about a case after reading textual description and seeing diagram • Error_detection: score on identifying 5 deliberately seeded errors in a diagram relative to textual description • Opinion: • Perceived Ease of Use, Perceived Usefulness, and Intention to Use, measured by answers to a TAM-inspired questionnaire w 14 questions

  12. Hypotheses • Since the colour notation was best in the analytical comparison, this was hypothesized to have advantages, i.e.: • H1: understanding scores will be better for the colour notation than for the annotated notation • H2: error detection scores will be better for the colour notation than for the annotated notation • H3: participants’ opinion about the colour notation will be more positive than for the annotated notation

  13. Experimental tasks • Answering a pre-experiment questionnaire investigating relevant competence • Reading tutorial about first diagram notation (annotatated or colour depending on group) • Reading text and diagram for case (home care or traffic control), answering 12 T/F questions • Answering post-task questionnaire giving opinion about the notation • Repeating steps 2-4 with the opposite case and notation • Repeating 2-6 with the error detection task

  14. Latin squares design

  15. Results • 46 students participated, randomly assigned into the four Latin squares groups • Clear advantage for colour notation in performance • 3 students performed very poorly in error detection (not doing a serious job); might be considered outliers? • Slight, but not significant advantage for colour when it comes to opinion

  16. Conclusion on hypotheses • H1: coloured notation would be better than annotated for understanding (answering T/F questions): CONFIRMED • H2: coloured notation would be better than annotated for detecting errors: CONFIRMED • H3: participants’ opinion about the coloured notation would be more positive: REJECTED

  17. Threats to validity • Conclusion validity: significant results but small to moderate effects, should have had larger N to make a strong claim about results • Construct validity: many other ways of understanding a model than answering T/F questions, and many other work tasks than identifying errors. But at least, this is a relevant task, and the ability to answer questions correctly should indicate to some extent whether a model has been understood

  18. Threats to validity, cont. • Internal validity: Latin squares design and pre-exp. questionnaire should control very well for any selection bias. Notations were presented in equal detail and style in tutorials, and no preferred or hypothesized outcome was signalled to the students • External validity: The biggest challenge: • Students are not practitioners and motivation may be limited in an experiment which has no impact on their job or a delivered product. But the comparative nature of the experiment should mean that performance with both notations are equally hurt by low competence or motivation. • Small experimental tasks are not representative of the more complex tasks in ”real” mobile IS development.

  19. Further work • Also experiment with different pattern fills instead of colour (e.g. better for colour blind users) • More experiments, possibly including practitioners and/or including collaboration among several persons instead of just individuals answering questions • Larger industrial case studies, to try out alternative notations with larger and more realistic work tasks

More Related