Chapter 11 Phase 5 : Worker Competencies Analysis

1. Chapter 11Phase 5 : Worker Competencies Analysis 홍 승 권

2. Purpose • The fifth phase of CWA • To address the traditional core concerns of the HF and HCI community • To identify the competencies that an ideal worker should exhibit • Two steps to consider • To consolidate the requirements imposed by the previous phases • To determine how those requirements can be met in a way that is consistent with human limitations and capabilities • Modeling tool : SRK taxonomy • A framework within which domain requirements and existing knowledge of human cognition can be integrated • An example using the DURESS II process control microworld

3. The ecological approach revisited

4. Ecological Work Domain Control Tasks Strategies Social-Organizational Worker Competencies Cognitive • Funneling in with each successive phase • Reducing the degree of freedom • From environment constraints to cognitive constrains

5. From cognitive constraints to system design • There are much knowledge of human capabilities and limitations • Highly fragmented and applicable only to narrow psychological phenomena • Potentially relevant to systems design • A challenge in worker competencies analysis • To pull together requirements of the application domain and the relevant subset of existing knowledge about human cognition in a integrated way • To derive practical implications for system design

6. The origin of Skills, Rules, Knowledge taxonomy • Rasmussen (1983) 개발 • To help organize relevant knowledge into a form that is more useful for system design • As Sanderson and Harwood(1988) : It is easy to misinterpret SRK taxonomy

7. A taxonomy, not a model • The taxonomy provides a set of distinctions, not a detailed model of psychological processes • One of the primary criteria in the development of the taxonomy is usefulness, not necessarily “truth”. • Each level in the taxonomy corresponds to a category of human performance • Respecting these distinctions should make it easier to develop models for each category of human performance

8. Criteria to distinguish categories of human performance • Dependent on it’s purpose • SRK taxonomy is defined by • Distinguishing categories of human behavior according to fundamentally different ways of representing the constraints in the environments • Unpack these premises into a series of logical steps • Goal-directed interaction between a worker and an environment depends on constraints • Constraints that need to be taken into account can be represented by workers in fundamentally different ways • Each of these ways taking into account action-relevant constraints is linked or defines, a different categories of human performance

9. Logical steps to unpack the premises • Goal-directed interaction between a worker and an environment depends on constraints • Ex) gravity : 공 잡기 • Different types of constraints that are relevant to the worker’s current goals : work domain constraints, control task constraints • Constraints that need to be taken into account can be represented by workers in fundamentally different ways • One : by reasoning analytically using a symbolic representation of Newton’s law • Two : by a set of instructions that would specify what actions should be taken • Three : by an internal embodiment (or implicit model) of the relevant dynamics to continuously guide actions • Each of these ways taking into account action-relevant constraints is linked or defines, a different categories of human performance (후속)

10. Action-relevant constraints & human performance • KBB is defined by serial, analytical reasoning based on a symbolic representation of the relevant constraints in the environment. • KBB guides action by representing the goal-relevant constraints in the environment as a mental model • RBB is defined by an if-then mapping between a familiar perceptual cue in the environment and an appropriate action • No reasoning is required and there is a direct association between a cue and an action • RBB guides action by representing the goal-relevant constraints in the environment in terms of perceptually grounded rules • SBB is defined by real-time, direct coupling to the environment via dynamic world model • Dynamic world model is only an implicit model of the environment • If skilled in motor control, actions are successful without having to represent gravity explicitly internally. • Just as the representation of information in an analog computer corresponds to its physical structure • By providing a basis for direct coupling and parallel, continuous interaction with the world

11. Relation between levels of cognitive control in SRK taxonomy and the way in which constraints in the environment are represented and processed internally

12. Internal representation & Skill-based behavior • Smooth, automated, and highly integrated patterns of action that are performed without conscious attention • Typical example : automated psychomotor activity (e.g. walking) that is driven by a continuous perception-action loop • Anticipatory actions, direct coupling to the environment, prototypical temporal-spatial patterns, non-verbalized.

13. Internal representation & Rule-based behavior • Stored rules derived from procedures, experience, instruction, or previous problem-solving activities • Action is goal-oriented but goals are not explicitly represented. Goals can only implicitly be found in the structure of the rules. • The rule will reflect the functional properties of the environment that constrain action • People are not reasoning; they are merely using familiar perceptual cues in the environment to trigger actions directly • Workers are usually aware of their cognitive activities at the RBB and thus verbalize their thought

14. Internal representation & Knowledge-based behavior • Deliberate serial search based on an explicit representation of the goal and a mental of the functional properties of the environment • Goals are considered explicitly rather than implicitly • Slow serial and effortful because it requires conscious, focal attention • Frequently used unfamiliar situations where previous experience is no longer valid and a solution must be improvised by reasoning

15. Goals Symbols Knowledge-Based Behavior Identification Decision Planning Rule-Based Behavior Signs Association state/task Stored rules for activities Recognition Skill-Based Behavior (Signs) Automated sensorimotor movements Feature formation Sensory input Signals Actions The SRK taxonomy of human performance

16. The role of information interpretation: signal, sign, symbols • The three levels are distinguished • according to the way in which workers interpret information from the environment (Signals, Signs and Symbols) • by the kinds of internal representation • Signals • Signals have a strong perceptual basis because they are continuous quantitative indicators of the time-space behavior of the environment • The changing distance between your car and the lane markers may be a time-space signal • Signs • Signs are arbitrary but familiar perceptual cues in the environment. They refers to the state of the world by convention or by prior experience • A red octagon with the word stop may be a sign • Symbols • Symbols are meaningful formal structures that represent the functional properties of the environment • Your knowledge of the car’s components are represented as symbols in a mental model (오작동 진단을 위해 구성품들 사이의 관계에 대한 지식을 사용하는 경우)

17. Signal, Sign, Symbol distinctions • The concepts refer to the way in which an observer interprets information, • not just to the form in which information is presented • The very same display may be interpreted as a signals as a sign or as a symbol • Signals and signs have a perceptual basis, whereas symbols have a semantic basis • Both signals and signs refer to perceptual properties of the environment such as distance, color and size • Symbols although they may take a particular form, refer to the meaning of the information, not its form

18. A hypothetical example showing the difference between signals, signs and symbols

19. Interactions between levels (I) • The interactions between levels are best appreciated by adopting a temporal perspective. • Synchronous activities • Those that occur online and in-real and deal with the current situation • Workers is directly coupled to spatial-temporal properties in the environment • Continuous steering your car to make sure that it is within the lane markers • Synchronic activities • Those that occur online and real time, but they deal with the sings that are used to trigger new SBB activities • Such activities fall into the realm of RBB • You hear your engine revving up to a familiar reference pitch, so you shift gears. • For an experienced driver, the act of changing gears would take [lace at the SBB level (direct coupling to the time-space properties of the environment

20. Interactions between levels (II) • Diachronic activities • Occur offline and require evaluation or planning. They involve using previous experience to interpret the world in terms of signs that could trigger SBB activities • Those deal with what has occurred in the past and may occur again in the future • Recognizing a familiar stretch of highway that in the past, has been followed by an icy patch of road. • Achronic activities • Occur offline and require evaluation or planning • They involve reasoning in symbolic terms , not associating actions based on familiar perceptual signs.

21. Achronic Planning in terms of functional reasoning by means of symbolic model: “As can be Knowledge-Based Domain OFF-LINE EVALUATION AND PLANNING Diachronic Synchronic Rule-Based Domain Planning in terms of recall of past and rehearsal of future, predicted scenario: “As has been and may be” Attention on cue classification and choice of action alternatives: “As is” Skill-Based Domain Synchronous “As is” ON-LINE REAL-TIME OPERATION Data-driven chaining of subroutines with interrupt to conscious, rule-based choice in case of ambiguity or derivation from current state of the internal world model. Interactions between the different levels of cognitive control

22. Avoiding some misconceptions (I) • A continuum with completely preattentive, automatic processing on the skill-based end and completely attentive, controlled processing on the knowledge-based end • The three ways of interpreting information are discrete concepts that do not lie on a continuum. • Nothing in between the two (signal and sign) • Actions directly on the world are only possible at the SBB • Actions on the world are driven by automated sensorimotor patterns at the SBB level (그림 참조)

23. Avoiding some misconceptions (II) • To classify tasks as belonging solely to one of the three levels in the taxonomy • Such an interpretation would be overly simplistic • There are interactions among three levels (앞에서 설명) • SBB level is always active and is responsible for directing attention, activating higher levels, controlling information gathering as well as transferring intentions into control of movements • The relationship between a particular task and levels of cognitive control is not fixed because it can be mediated by several variables • Worker’s level of expertise : novice may have to resort to KBB, but expert may enable RBB • The form in which information is presented : graphic form (lower level), alphanumeric form (higher level) • The degree to which workers reflect on their performance : Unreflective worker (using RBB and SBB), Reflective worker (using KBB)

24. Examples of its usage • A taxonomy of human performance models • Stages of skill acquisition • Theories of expertise effects in the memory recall • A framework for interface design for complex sociotechnical systems

25. Psychological problem-solving models, Information flow models, production systems, GPS-General problem solver Goals Symbols Knowledge-Based Behavior Identification Decision Planning Rule-Based Behavior Fuzzy set models, Production systems, Scripts Signs Association state/task Stored rules for activities Recognition Decision theory Social judgment theory, Attribution theory Skill-Based Behavior (Signs) Automated sensorimotor movements Feature formation Manual control models Optimal control models Sensory input Signals Actions Decision : Signal detection theory, Estimation theory, Attention allocation : Sampling theory, Queuing theory Mapping of human performance models onto the SRK taxonomy

26. Stages of skill acquisition (1) • Five stages of skill acquisition (Dreyfus & Dreyfus 1988) • Novice : students learn explicit facts, features and rules that can be readily verbalized. Context-free • Advanced beginner : students begins to take into account more contextual factors and thus develop more sophisticated rules for performing a task • Competent performer : students use specific goals to prioritize facts according to their relevance • Proficient performer : activity at this stage is the result of experience-based associations connecting context and current stimuli with plans that have proven to be successful • Expert : students are able to deal with task demands in an effortless and automatic fashion.

27. Stages of skill acquisition (2) • Reinterpretation by SRK taxonomy (Olsen and Rasmussen, 1989) • Expertise is not a fixed property of a person, but rather a dynamically varying relationship between the demands imposed by the environment and the resources of a particular person • Additional implications • Learning can take place within each level : ex) learning to talk or walk involves exploration and development within the SBB level • Differentiating expertise at different levels also highlights the issue of meta-expertise: meta-expertise가 부족하면 특정 수준에 머물게 됨.

28. Theories of expertise effects in the memory recall (1) • Relationship between expertise and memory recall performance • Chess master were able to reconstruct perfectly a chess position • Perceptual chunking theory • These chunks are perceptually structures that contains information about patterns of pieces on a chessboard • Experts can recognize and code familiar chunks of pieces • Conceptual chunking theory • This is based on the meaningfulness of the material to be recalled, not its perceptual form • Expert’s skill is a result of the organization of functional concepts in long-term memory

29. Theories of expertise effects in the memory recall (2) • To develop an integrative theoretical account of the chunking phenomenon by the SRK taxonomy • The perceptual account draws on the RBB level • The environment (i.e. the set of pieces to be recalled) is interpreted as signs (i.e. familiar perceptual cues) that correspond to groups of chess pieces. • Chunking is possible because of a recognition process • The conceptual account draws on the KBB level • The environment is interpreted as symbols (i.e. knowledge structures) that correspond to meaningful relationship between chess pieces. • Chunking is possible because of conceptual inference • These processes can interact and chunking can be driven by a combination of the two levels.

30. A framework for interface design for complex sociotechnical systems • 지금까지 사례들은 심리학과 인지과학에서의 research issue와의 개념적 일치성을 보이기 위해 SRK taxonomy사용. 앞으로 To develop design principles (Ecological Interface Design) • 3 steps of EID framework 이론 • Literature review on interface design • SRK taxonomy as an framework to integrate the variety of research results • To deduce a set of three principles for interface design

31. Lower levels are easier Lower levels are preferred Complex tasks require all levels Greater the demand, higher the level Design for use of lower levels All levels need to be supported Goal of interface design: Design interface that do not force cognitive control to a higher level than the demands of the task require, but that also provide the appropriate support for all three levels Theoretical rationale behind the development of the EID framework Greater the skill, lower the level

32. A framework for interface design • Interface should achieve twofold goals • To design interfaces in such a way as not to force cognitive control to a higher level than the demands of the task require • To provide the appropriate support for all three levels • 3 design principles • SBB- To support interaction via time-space signals, workers should be able to act directly on the display • RBB- Provide a consistent one-to-one mapping between the work domain constraints and the cues or signs provided by the interface. • KBB- Represent the work domain in the form of an abstraction hierarchy to serve as an externalized, faithful model that will support knowledge-based problem solving

33. For process control microworld (1) • Skill-Based Behavior • Workers should have basic perceptual skills • Workers should be able to interact directly with the display rather than using command language • Timing is a very important aspect of proficient control in DRUESS II (visualization : e.g. time constants, rates of change) • Rule-Based Behavior • Worker competencies at the RBB level can be divided into tow categories • To create salient perceptual cues • To be aware of the various shortcut and strategies that they can use to effectively control the process

34. For process control microworld (2) • Creating signs • Graphical visualization of the mass balance constraints • Ensuring that workers are aware of procedural constraints that can be exploited to control the process • Pumps should not be turned on before valves (In the control task and strategies analysis) • Knowledge-Based Behavior • By presenting these relationship (Identified in WDA) in the interface, we would be creating a faithful, external model of the process that workers could use during problem solving • Serve as the basis for a training program

35. Summary Cognitive Work Analysis Framework Cognitive Work Analysis Systems Design Identify Realize Form Build Develop Models of Intrinsic Work Constraints Conceptual Distinctions Modeling Tools Systems Design Interventions v 1. Work Domain 1. 1. 1. Sensors, models, database Abstraction- Decomposition v 2. Control Tasks 2. 2. 2. Procedures, automation, context-sensitive interface Decision ladder v Information Flow Map 3. Strategies 3. 3. 3. Dialogue modes, process flow v All of the above 4. Social-Organizational 4. 4. 4. Role allocation, organizational, structure 5. Worker Competencies 5. 5. 5. Selection, training, interface form