1 / 20

Construction and Pre-test of a Semantic Expressiveness Measure for Conceptual Models

Construction and Pre-test of a Semantic Expressiveness Measure for Conceptual Models. Ann Maes Frederik Gailly Geert Poels Roland Paemeleire (Ghent University). outline. Research objectives and questions Theoretical foundations Research methodology Item generation Item Refinement

jakeem-reid
Download Presentation

Construction and Pre-test of a Semantic Expressiveness Measure for Conceptual Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Construction and Pre-test of a Semantic Expressiveness Measure for Conceptual Models Ann MaesFrederik GaillyGeert PoelsRoland Paemeleire(Ghent University)

  2. outline • Research objectives and questions • Theoretical foundations • Research methodology • Item generation • Item Refinement • Reliability and Validity analysis • Discussion

  3. Research objectives and questions • REA accounting model  one quality: semantic expressiveness • SE: “how well a model reflects the underlying reality the model represents”

  4. Alleged benefits of SE • Easier integration of representations of accounting and non-accounting phenomena • Better user understanding of accounting systems  Empirical test (Dunn & Grabski, 2000)

  5. Empirical test of SE • REA <> DCA • Measure: a seven-point Likert scale assertion • “the documentation I received provided me with a realistic representation of the accounting information flows of the business” • Results: • Users perceive REA-model based accounting system as more semantically expressive • SE associated with higher accuracy in information retrieval tasks

  6. Our research objective • Development of better measure for PSE • multi-item measurement instrument  yield a more pure indicant of the conceptual variable • flexible measurement instrument  evaluate the REA model but also other conceptual modeling approaches

  7. Theoretical foundations • Conceptual definition of the construct to be measured ( content validity) • Dunn & Grabski : SE = property of REA model “how well a model reflects the underlying reality the model represents” • McCarthy: SE = property of a schema “the degree to which elements in the final enterprise schema correspond to or capture the meaning of elements in the modelled corporate reality”

  8. Lindland et al. framework for quality in conceptual modeling • Language – domain – representation – audience interpretation • Semantic quality: correspondence between domain and representation (cf. McCarthy) • Language-domain appropriateness: a measure of how the language fits the domain, the extent to which the language makes the kind of statements needed in the domain (cf. Dunn & Grabski)

  9. Wand & Weber research agenda for IS and conceptual modelling research (2002) • Grammar quality  script quality • Evaluation needs  SE = semantic quality • Lastly: semantic quality of a schema is impossible to evaluate empirically as it requires observing and interpreting the problem domain  perceived semantic quality

  10. Perceived semantic expressiveness • Definition: “the correspondence between the user understanding of a representation developed according to a language and the user understanding of the reality that needs to be represented”

  11. Research methodology Item Generation Step 1 – Literature Review and Scale Creation • Examine literature for existing scales • Assess applicability of existing scales and revise if appropriate • Develop new items as necessary based on conceptual definition/ theoretical framework Item Refinement Step 2 – Pre-Test • Test scale and items using convenience sample • Calculate reliability and validity scores and modify scales as necessary • Purify measurement instrument Confirmatory Analysis Step 3 – Gather Field Dataset • Collect response data from representative random sample Step 4 – Exploratory Factor Analysis • Perform factor analysis on initial measurement model • Assess reliability and validity of measurement model • Remove “problem” indicators, if any from measurement model Step 5 – Confirmatory Factor Analysis (CFA) • Examine overall fit and inspect item-level fit for multi-dimensionality • Assess modified scales for acceptable reliability

  12. Item generation *taken from Lindland et al. (1994)

  13. Item Refinement • Pre-test • Experiment with convenience sample • Aim = other quality aspects • Post experiment questionnaire with also two other measures for perception based variables: - perceived ease of use - user information satisfaction

  14. PEOU instrument * Adapted from (Davis, 1989)

  15. UIS instrument * Adapted from Seddon and Yip (1992)

  16. Reliability and Validity analysis *** Correlation is significant at the 0,01 level (2-tailed) ** Correlation is significant at the 0,05 level (2-tailed) * Correlation is significant at the 0,1 level (2-tailed)

  17. Reliability analysis on initial PSE instrument

  18. Remaining items… *** Correlation is significant at the 0,01 level (2-tailed) ** Correlation is significant at the 0,05 level (2-tailed) * Correlation is significant at the 0,1 level (2-tailed)

  19. Reliability analysis on refined PSE instrument

  20. Discussion • “consistent item” • Compared to Dunn & Grabski single item measurement • Confirmatory analysis • Larger sample • Different set of participants Reliable and valid flexible multi-item measurement instrument for PSE

More Related