Lifelong Competence Development: On the Advantages of Formal Competence-Performance Modeling

1. Learning Networks for Lifelong Competence Development March 30 – 31, 2006, Sofia, Bulgaria Lifelong Competence Development: On the Advantages of Formal Competence-Performance Modeling Michael D. Kickmeier-Rust, Dietrich Albert, & Christina SteinerCognitive Science Section, Department of PsychologyUniversity of Graz, Austria

2. Lifelong Competence Development • Lifelong competence development is undoubtedly an important and ambitious aim for the information and knowledge society • This presentation intends to introduce and motivate Knowledge Space Theory and the Competence-Performance Approach as tools to facilitate the development and assessment of competencies over time

3. Introduction:Competence vs. Competency • Competence • vague • broad and intangible • complex vs. simple • hard to be modeled on a formal basis • Competency • small and unique • still intangible • easy to be modeled on a formal basis

4. Introduction:Competence vs. Performance • Often the terms competence/y and performance are mixed-up • Often it is assumed that competencies could directly be observed or assessed • Inflation of competencies because different assessment methods measure different competencies or sets of competencies • e.g., maintain an aircraft, write a scientific article, pass a certain school exam • Incomparability of different assessment methods • e.g., school exam vs. assessment on the job

5. Introduction:Competence vs. Performance • The American Heritage Dictionary of the English Language states: “Competence means the state or quality of being adequately or well qualified; a specific range of skill, knowledge or ability” • This and many other definitions have in common that they describe competence as • abstract • latent • not directly observable

6. Introduction:Competence vs. Performance • Chomsky (1965) distinguished latent competence and observable performance in linguistic theory • Today, this distinction between competence and performance has a much wider application e.g., in the field of knowledge and learning psychology • competence is an unobservable quality or ability • performance is the observable behavior in specific situations (e.g., an exam), which is determined by one specific competency or by a set of competencies

7. Why distinguish competence and performance? • Example 1: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator • (1) If two students master a certain task of the exam, can we conclude that these students do have the same competencies with regard to the task? • Student 1 might have the necessary competencies to master the task without using the formulary • Student 2 maybe mastered the task only by chance, incidentally choosing the right formula from the formulary

8. Why distinguish competence and performance? • Example 2: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator • (2) If three students fail in a certain task, can we conclude that these students lack the same competencies? • Student 1 might lack the competence to fully understand the task and its formulation • Student 2 might fully understand the task and also might be able to choose the right formula, but maybe this student is not able to use a required function of the calculator • Student 3 might have the necessary competencies to master the entire task but might have problems to concentrate on the tasks during an exam

9. Lifelong learning? • In terms of tracking and assessing lifelong competence development • we should make sure to measure competencies independent from assessment methods • refer to probably standardized competencies • refer to defined developmental / learning paths

10. Knowledge Space Theory • A well-elaborated theory that may help to achieve these goals isKnowledge Space Theory by Doignon & Falmagne (1985, 1999) and its extensions • KST provides a set-theoretic framework to organize and model the knowledge / competencies in a given domain of knowledge by utilizing Surmise Relations, which establish Knowledge Spaces • KST in its initial form is only a behavoristic approach focusing on problems (e.g., test tasks), which can be mastered or not • From mastering a certain problem, KST allows to assume the mastering of other problems and from failing in a certain problem, KST allows to assume a failing in other problems

11. Knowledge Space Theory • Example: Five problems from the domain “basic algebra”: • a: addition • b: subtraction • c: multiplication • d: division • e: simple linear equations •  Q = {a, b, c, d, e}

12. Knowledge Space Theory • Example:Prerequisite Relation for the domain Q = {a, b, c, d, e}

13. Knowledge Space Theory • Example: We can establish a Knowledge Space, which does not contain all of the 25 possible Knowledge States but only 8

14. Knowledge Space Theory • Advantages: • Reduction of the number of possible Knowledge States and definition of meaningful learning paths • Mathematical properties: • reflexive • transitive • anti-symmetric

15. Competence-Performance Approach • KST is purely behavioristic focusing on observable performance • CPA (Korrosy, 1997, 1999) is an extension of KST, which distinguishes latent competencies and observable performance • We have a set E of abstract competencies that are relevant for a domain • The Competence State is the collection of a person’s competencies • As in KST, Prerequisite Relations are described on the set of competencies establishing a competence structureC, which contains all possible competence states

16. Competence-Performance Approach • Example: Four competencies from the domain “basic algebra”: • A: addition • B: subtraction • C: multiplication • D: division •  E = {A, B, C, D}

17. Competence-Performance Approach • Example:Prerequisite Relation for the competencies in the domain E = {A,B,C,D}

18. Competence-Performance Approach • Example: We can establish a Competence Space, which does not contain all of the 24 possible Competence States but only 7

19. Competence-Performance Approach • Unfortunately, we cannot observe this… • Representation and Interpretation Functions enable to map test items / tasks to the competencies  We can determine a person’s Competence State We can determine the required competencies to master a specific task  No 1-to-1 mapping required

20. Competence-Performance Approach • Example: Two tasks from the domain “basic algebra”: • a: multiplication problem • b: solving linear equations •  Q = {a, b} •  Representation Function

21. Competence-Performance Approach • Example: • Solved a but not b • Solved a and b • Solved not a and not b • Solved b and not a

22. Advantages • Modeling a domain of knowledge on a formal basis • Referring to clearly defined and unique competencies • Mapping different assessment methods to the same set of competencies • Efficient adaptive testing • Efficient adaptive teaching • Modeling of individual learning path • Computable

23. So what about errors? • Careless errors • Lucky Guesses - Besides the deterministic approach there are also probabilistic approaches

24. Applications • Adaptive, personalized eLearning (RATH, APeLS, EASEL, iCLASS, ELeGI, ELEKTRA) • Organizational competencies / knowledge (Know-Center) • Research tool (e.g., in child development)

25. Applications • Commercial eLearning platform ALEKS (www.aleks.com) Image courtesy of ALEKS CorporationSanta Ana, CA, USA

26. THANK YOU