740 likes | 902 Views
Inquiry into effective teaching with one hand tied behind your back. 22 November 2006 UBC Institute for the Scholarship of Teaching and Learning Dan Bernstein, University of Kansas djb@ku.edu. Overview of session. Invited to discuss my research methods
E N D
Inquiry into effective teaching with one hand tied behind your back 22 November 2006 UBC Institute for the Scholarship of Teaching and Learning Dan Bernstein, University of Kansas djb@ku.edu
Overview of session Invited to discuss my research methods Among methodological issues in research on teaching, design and measurement are key Look at examples of some of my methods Embed them in larger framework as I go Inform some practices by research literature Recommend a different path from my own Propose a big tent, even for research done without the best possible methods
Research Methods Issues Design and Analysis Measurement
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment EXAMPLE
Experimental Procedures • Commercial and home-made multi-media • Used video and live lecture with same content as multi-media • Created text samples with same content • Assessment of levels of understanding with mixed methods of responding • Individual instruction with unlimited time • Participants from introductory psychology
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment Analyze components
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment Analyze components EXAMPLE
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment EXAMPLE
Upgraded the lecture component • Live lecture to portion of introductory class • Topic is reactive measurement and quantum mechanics • Matched to locally authored multi-media and text -- also irrelevant text control • Motivation sustained by possibility of test • Assessment layered by type of understanding and format of question
Distribution of total test performance resulting from four methods of instruction
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment Statistical comparison
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment Statistical comparison EXAMPLE
Pushed envelope on assessment • Looked for performance assessment • Topic was searching a jail cell • Verbal assessment is recognizing and stating guidelines for complete search • Authentic assessment is use of guidelines in searching a jail cell assembled for testing • High end interactive program developed by NETV for Nebraska Sheriff’s Association
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Control Treatment ¿Ethical questions? ¿Permission?
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Benchmarks of achievement
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Benchmarks of achievement EXAMPLE
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Benchmarks of achievement Movement of achievement
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study Benchmarks of achievement Movement of achievement EXAMPLE
[Universal] Design for Success • Presume students can learn • Discount need to sort or differentiate • Maximize overall course performance
Benjamin Bloom promoted mastery • Based on practice and feedback • Divide course into many smaller units • Take examinations and get results • Require taking exam again until high score • IFF 95% correct => study next unit
Fred Keller promoted mastery • IFF 95% correct => study next unit • Course grade is number of units passed • No penalty for repeating and learning • All who pass 12 units => grade of A • Do A work on 10 units => grade of C
Mastery Class 95% contingency No penalty for learning Immediate feedback No lecture required Lecture class Same exam questions Two attempts per test In class feedback No contingency Also taught conventional lecture
Total amount learned • Nearly twice as many at the high end of learning • Virtually no one failed to learn • Maximized learning for many students
Showed in amount and accuracy • Many more questions answered • Took 12 15-item tests • Lecture was three tests of 20 items each • Certify more learning • Overall percent correct also higher
No magic -- students studied better • They put in more time to their learning • There was more work asked for by the course • Note that they report doing the reading more • Preparation for class is key issue (later also) • Guideline in US -- 2 hours outside for every 1 hour in class
Research Methods Issues Design and Analysis Measurement Random Assignment Experiment Descriptive Study What do you think about the relative utility of these approaches?
Research Methods Issues Design and Analysis Measurement Direct Indirect
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Cognitive Affective
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Cognitive Affective ¿EXAMPLES?
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Affect Contextual application Repeat knowledge Abstract understanding EXAMPLE
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Affect Contextual application Repeat knowledge Abstract understanding YOUR EXAMPLES
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Affect Contextual application Repeat knowledge Abstract understanding EXAMPLE
Reasonably Conceptual Items • Describe the results of [Razran’s] second experiment on semantically mediated generalization (numbers) done with a 13-year old boy. Be sure to identify the respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than physically mediated generalization? • What do chicken scratches, raccoon rubbings and pig rooting have in common? What stimuli have pre-existing relations these bits of behavior? What does the “misbehavior of organisms” have to do with the generality of principles of learning?
Transform into Problems • Old question: • What were the reinforcing consequences in the Welsh, Bernstein & Luthans (restaurant) study? How were the consequences identified? • Problem-based Assessment : • Suppose you were asked to implement a motivational program in a business with 25 employees engaged in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the restaurant study, how would you improve the quality of the employees work by using access to activities as a motivator? Describe the costs and benefits of the program and make a recommendation about implementation.
Old question: • The behavioral context into which punishment is added must be considered in evaluating the likely effect of a given stimulus. What conditions are essential in this analysis? • New question: • Suppose it is your task to decrease the frequency of college students engaging in binge drinking or games involving high levels of intoxication. You have been asked to set up a punishment program to eliminate this program before someone is hurt. What context for binge drinking would you identify first? What punishing consequence would you use? What characteristics would you include in your punishing system to maximize effectiveness? Please give an example of an additional element you would need to include to make the plan maximally effective.
Old question: • Describe the results of the experiment on semantically mediated generalization done with a 13-year-old Soviet boy. Be sure to identify the respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than physically mediated generalization? • New question: • Generate your own example of classical conditioning using the neutral stimuli “idea” and “rock” along with the eliciting relation between a very loud noise and increased heart rate. Your example should include all of the following components: a description of a conditioning procedure that would produce differing reactions to the two stimuli, a description of the procedure that tests for the direct effects of conditioning, a description of a procedure that would test for physically mediated generalization, a description of a procedure that would test for semantically mediated generalization, and the likely results of the three test procedures.
Research Methods Issues Design and Analysis Measurement Direct Indirect Verbal surrogate Authentic Affect Contextual application Repeat knowledge Abstract understanding General interest Intent to act