210 likes | 329 Views
Formative Research on the Heuristic Task Analysis Process. Charles M. Reigeluth Ji-Yeon Lee Bruce Peterson Mike Chavez Indiana University. Domain vs. Task. Domain. Task. Economics Psychology Physics Chemistry. Management Counseling Engineering Medicine.
E N D
Formative Research on the Heuristic Task Analysis Process Charles M. Reigeluth Ji-Yeon Lee Bruce Peterson Mike Chavez Indiana University
Domain vs. Task Domain Task Economics Psychology Physics Chemistry Management Counseling Engineering Medicine
What is Heuristic Task Analysis? • Domain vs. task expertise • Procedural vs. heuristic tasks • Heuristic tasks: using principles and guidelines to decide what to do when • Heuristics are often tacit knowledge • Heuristic Task Analysis (HTA): elicits and formalizes an expert’s heuristic knowledge
Why is HTA important? • Increasingly complex tasks in society • Many tasks are heuristic (continuum) • Heuristic task knowledge is often the most valuable and hardest to discover • Heuristic task analysis methodologies are underdeveloped
Literature on HTA • Knowledge elicitation: Extraction of domain-relevant knowledge from human experts using various techniques • Knowledge analysis: Interpretation of the data extracted from experts • Knowledge representation: Determined by the purpose of application (e.g. expert system or Artificial Intelligence system)
Literature on HTA • Most studies deal with procedural rather than heuristic tasks • Lack of research that provides guidelines on how to analyze and represent heuristic knowledge • Lack of research to inform instructional design
The Initial HTA Process The handout shows the version of the Heuristic Task Analysis Process investigated in this research.
Research Question “ What improvements can be made to the HTA process?”
Formative Research • Developmental research: improves a theory and its real-world application • Process: • Apply a theory to an instance • Collect data to improve the application • Suggest improvements to the theory • Iterations • Three cases
Study I: Context • Task: Group Counseling • Focused on personal growth group • Combination task with both procedural and heuristic elements • Constraints: • Due to the confidential nature of counseling, direct observation was impossible • Dealing with interpersonal dynamics which is hard to verbalize • Analyst unfamiliar with group counseling
Study I: Methods • Formative research: designed case • Data collection: • Interview: unstructured in-person interviews • Videotapes • Participant profile: • 3 experts experienced with personal growth group counseling • Expertise ranged from 3 to more than 20 yrs • Data analysis & interpretation: • Member checks with participants • Consultation with three other researchers
Study I: Results • To best elicit task experts’ heuristic process, we need to incorporate various interview and observation techniques. • The analyst’s task expertise needs to be considerable. • Working with multiple experts requires certain strategies. • Providing reference material during the interview was helpful.
Study II: Context • Task: Tutoring on Writing • Class of problems focused on higher-level writing concerns such as thesis, unity, and coherence • Constraints: • Experts had little time to devote to this project
Study II: Methods • Top-down approach to identify and analyze heuristic knowledge • Participant profile: • 2 experts experienced in tutoring college students • had passed an evaluation of their tutoring skills each term they had been tutoring • Data collection& analysis • Interview: In-person interviews (60 min. each) • Iterative interview & analysis process • Index cards used to record tutor’s responses
Study II: Results • Using the top-down process was effective • Using index cards was helpful • Helping the tutor identify guidelines was problematic • Helping the tutor access the context of the tutoring situation was helpful • Having the tutor list, prioritize, and choose the decisions made during the tutoring session was effective
Study III: Context • Task: • Selecting Artwork for Product Line • Constraints: • Analyst unfamiliar with business content • Experts had little time to devote to this project • Research focus: • Speed and effectiveness of HTA process in business setting
Study III: Method • Formative research: designed case • HTA guidelines converted into lay questions • 1 expert, 2 interviews, 1 1/2 hours each • Content, process analysis after interviews • Member checks w/ 1 content and 1 research expert
Study III: Results • Speed of HTA process: • Expert was satisfied with time frame • 50% of interview was spent eliciting heuristics • 1 hour was spent on closely associated tasks • 3x5 cards helped analyst get back on track • Effectiveness of HTA process: • Expert satisfied when heuristics were elicited • Words like feel, know, like were clues to tacit knowledge and mental models
Discussion • Success with the top-down approach (knowledge categories) varied in the studies • More guidance was needed for eliciting the heuristics • Using multiple experts presented challenges for the analyst • Index cards were helpful for all 3 analysts • The HTA process seemed to function well in all three fields.
Revisions to the HTA Process • Preparatory guidance was added to help focus the entire process • Guidance was added for multiple experts • Guidance added on how to elicit heuristics • Bottom-up approach was added (Revisions on handout are in italics)
Questions or Comments? For the paper, e-mail Charles Reigeluth at reigelut@indiana.edu