1 / 29

Empirical Evaluation and Comparison of Enterprise Models: A Framework and Its Application

Empirical Evaluation and Comparison of Enterprise Models: A Framework and Its Application. Jean-Paul Van Belle Information System Dept jvbelle@commerce.uct.ac.za. Why research "models"?. You get interesting results!. Research Objective.

modesta-curtis
Download Presentation

Empirical Evaluation and Comparison of Enterprise Models: A Framework and Its Application

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Empirical Evaluation and Comparison of Enterprise Models: A Framework and Its Application Jean-Paul Van Belle Information System Dept jvbelle@commerce.uct.ac.za

  2. Why research "models"? You get interesting results! Framework for Model Evaluation ECITE'04

  3. Research Objective • “The development and validation of a comprehensive framework for the (semi-automatic) analysis and evaluation of (enterprise) models” • Why? • Model-driven development paradigm • Evaluation of method(ologie)s but not of output (actual model ‘quality’) • A good Ph D topic  Framework for Model Evaluation ECITE'04

  4. Comparing Enterprise Models Framework for Model Evaluation ECITE'04

  5. A Framework Was Needed … Framework for Model Evaluation ECITE'04

  6. Framework – 1st dimension Framework for Model Evaluation ECITE'04

  7. Framework- 2nd dimension Framework for Model Evaluation ECITE'04

  8. The Populated Framework Framework for Model Evaluation ECITE'04

  9. Theoretical Validation • Construct efficiency (“Occam’s razor”) and simplicity • Perspicuity • Coverage and completeness • Orthogonality • Extensibility, customisability, robustness and flexibility. • Genericity, universal applicability, portability and reusability • Formality, objectivity, absoluteness. • Theoretical foundation Framework for Model Evaluation ECITE'04

  10. Empirical Validation:The Model Database • Purpose: to serve as a validating test bank for the analysis framework. • Model Selection Criteria: • Domain = “The Generic Enterprise” • Sufficiently large size • ~200 entities; ~300 relationships • Publicly available • From very different reference disciplines Framework for Model Evaluation ECITE'04

  11. The Models: Systems Engineering • Reference Frameworks • Purdue • Nippon • ARRI • OO • BOMA • Fowler (patterns!) • San Francisco • ERD Libraries • Silverston • Hay Framework for Model Evaluation ECITE'04

  12. The Models: Practitioners • ERPs • BAAN • SAP R/3 • Real organisations • AKMA • NHS • Data warehousing • Inmon Framework for Model Evaluation ECITE'04

  13. The Models: Various • Ontologies • TOVE • CYC (subset) • AIAI • Miscellaneous • Miller (systems theory) • Ottawa (linguistic) • Random • Finance • Belgian Accounting • USB Growth Model Framework for Model Evaluation ECITE'04

  14. Validated SyntacticMetrics and Measures Framework for Model Evaluation ECITE'04

  15. Clustering (Fruchterman-Reingold) Framework for Model Evaluation ECITE'04

  16. ACIS 2003 Plotting the Fan-out Frequency Distribution Framework for Model Evaluation ECITE'04

  17. Validated Semantic Metrics and Measures Framework for Model Evaluation ECITE'04

  18. Simple Semantic Analysis: Common core concepts Framework for Model Evaluation ECITE'04

  19. Semantic analysis metrics: automated procedures • Flesch Reading EaseScore = 206.835 – (1.015 x ASL) – (84.6 x ASW) • ASL = average sentence length (the number of words divided by the number of sentences) • ASW = average number of syllables per word (the number of syllables divided by the number of words) • Noun synonyms generated by WordNet for the AIAI model word "action".Synonyms/Hypernyms (Ordered by frequency) of noun action: 9 senses • Sense 1: action => act, human action, human activity • Sense 2: action, activity, activeness => state … • Rank Adjusted Weighted Perspicuity Count Framework for Model Evaluation ECITE'04

  20. Semantic Overlap Framework for Model Evaluation ECITE'04

  21. Semantic DistanceDendogram Framework for Model Evaluation ECITE'04

  22. Validated PragmaticMetrics and Measures Framework for Model Evaluation ECITE'04

  23. Framework for Model Evaluation ECITE'04

  24. Extending the Framework to Other Areas • models of other domains • web site analysis • user interface • algorithms • documentation & training materials • frameworks & methodologies • software applications • programming languages • software architectures • "any" intellectual work of a conceptual nature Framework for Model Evaluation ECITE'04

  25. Conclusion • Framework was necessary • Validation • theoretically • empirically • independent parallel research (Chris Taylor @ QUT) • Some novel metrics – esp. semantic • Use in other domains • Useful for ‘composite metrics’ e.g. quality Framework for Model Evaluation ECITE'04

  26. Selected References • Böhm, B. et al. (1978). Characteristics of Software Quality. Elsevier North-Holland: New York. • Courtot, T. (2000). What to Look for in Packaged Data Models. Proceedings of the Meta Data Conference, Arlington. • Edmonds, B. (1999) Syntactic Measures of Complexity. Doctoral Thesis, University of Manchester, 1999. • Fowler, M. (1997) Analysis Patterns. Addison-Wesley: Reading (MA). • Claxton, J.C. and McDougall, P.A. (2000). Measuring the Quality of Models. The Data Administration Newsletter, 2000:14. • Gillies, A. (1997). Software Quality: Theory and Management. Thomson: London. • Hay, D.C. (1996) Data Model Patterns. Dorset House: New York. • McGabe, T.J. (1976) “A Software Complexity Measure” IEEE Trans. Software Engineering, 2 (Dec 1976), pp. 308-320. • Miller, J.G. (1978) Living Systems. McGraw-Hill: New York. • Ngo, D.; Chek L., Teo, L. et al.(2000) A Mathematical Theory of Interface Aesthetics. Unpublished working paper. • Perreault, Y. & Vlasic, T. (1998) Implementing Baan IV. Que: Indianapolis, Indiana. • Reyns, C., Jorissen, A., & Vanneste, J. (1994) Inleiding tot Accountancy. UFSIA: Antwerp. • Scheer, A.-W. (1998) Business Process Engineering. Reference Models for Industrial Enterprises. Springer-Verlag: Berlin • Shepperd, M. (1995) Foundations of Software Measurement. Prentice-Hall, London. • Silverston, L., et al. (2001) The Data Model Resource Book:Library of Universal Data Models For Enterprises. Wiley: NY. • Someya, Y. (1999). A Corpus-based Study of Lexical and Grammatical Features of Written Business English. Masters Dissertation, Dept of Language and Information Sciences, University of Tokyo. • Uschold, M. et al..(1998) The Enterprise Ontology. The Knowledge Engineering Review, Vol. 13. • Valente, A. (1996). Towards Principled Core Ontologies. In Proceedings of the 10th WKNAKBS, Banff, Canada, 1996. • Van Belle, J.P. (2002a) A Survey of Generic Enterprise Models. Proceedings of 32nd SACLA Conference, Port Elizabeth. Van Belle, J.P. (2002b) Towards a Syntactic Signature for Domain Models: Proposed Descriptive Metrics for Visualizing the Entity Fan-out Frequency Distribution. In Proceedings of the SAICSIT Conference, Sep 2002, Port-Elizabeth (South Africa). • Williams, TJ (ed). (1991) A Reference Model For Computer Integrated Manufacturing (CIM). CIM Reference Model Committee, International Purdue Workshop on Industrial Computer Systems. Framework for Model Evaluation ECITE'04

  27. Questions? Framework for Model Evaluation ECITE'04

  28. Framework for Model Evaluation ECITE'04

  29. Framework for Model Evaluation ECITE'04

More Related