Can Students and Faculty Learn to Respond to Challenges Facing Engineering Education?

1. Can Students and Faculty Learn to Respond to Challenges Facing Engineering Education? PurdueUniversity 6 October 2003 Jeff Froyd Director of Academic Development Texas A&M University

2. "There is an incredible evolution of learning or education as almost the sole source of competitive advantage in an economy that has changed so much." Howard Block, Managing Director Banc of America Securities An investment-bank and brokerage subsidiary of Bank of America.

3. Challenges in Engineering Education • Challenges • Challenge of lifelong learning • Challenge of problem solving • Challenge of engineering design • Challenge of transfer • Challenge of conceptual understanding

5. Lifelong Learning at Penn State Self-Directed Learning Readiness Survey (SDLRS), Guglielmino & Associates, http://www.guglielmino734.com/prod01.htm, March 2003. 27 “Although the data suggest a slight upward upward trend, the trend proved not to be statistically significant based upon an analysis of variance (ANOVA). Thus the cross-sectional study did not find evidence of an increase in readiness for self-directed learning, even for students in the later semesters who are taking elective courses and their capstone courses.” Litzinger, T., Wise, J., Lee, S., and Bjorklund, S. (2003) Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference

6. Challenge of Problem Solving “Despite individual professors’ dedication and efforts to develop problem solving skill, “general problem solving skill” was not developed in the four years in our undergraduate program. Students graduated showing the same inability that they had when they started the program. Some could not create hypotheses; some misread problem statements. During the four-year undergraduate engineering program studied, 1974-1978, the students had worked over 3000 homework problems, they had observed about 1000 sample solutions being worked on the board by either the teacher or by peers, and they had worked many open-ended problems. In other words, they showed no improvement in problem solving skills despite the best intentions of their instructors.” Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem Solving Program,” Journal of Engineering Education,

7. Challenge of Problem Solving • Ineffective approach #1. Give the students open-ended problems to solve; This, we now see, is ineffective because the students get little feedback about the process steps, they tend to reinforce bad habits, they do not know what processes they should be using and they resort to trying to collect sample solutions and match past memorized sample solutions to new problem situations.

8. Challenge of Problem Solving • Ineffective approach # 2: Show them how you solve problems by working many problems on the board and handing out many sample solutions • This, we now see, is ineffective because teachers know too much. Teachers demonstrate "exercise solving". Teachers do not make mistakes; they do not struggle to figure out what the problem really is. They work forwards; not backwards from the goal. They do not demonstrate the "problem solving" process; they demonstrate the "exercise solving" process. If they did demonstrate "problem solving" with all its mistakes and trials, the students would brand the teacher as incompetent. We know; we tried!

9. Challenge of Problem Solving • Ineffective approach #3: Have students solve problems on the board • Different students use different approaches to solving problems; what works for one won't work for others. When we used this method as a research tool, the students reported "we learned nothing to help us solve problems by watching Jim, Sue and Brad solve those problems!"

10. Challenge of Problem Solving • Through four research projects we identified why and how these and other teaching methods failed to develop process skills and which methods were successful in developing the skills • Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching Problem Solving Skills," Annals of Engineering Education, 1, 1, 238-243. • Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching Problem Solving" CHEM 13 News no. 155 (1985). • Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science Teaching, 22, 312-314. • Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College Science Teaching, 23, 57-58. • Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J. College Science Teaching, 23, 157-158.

11. Challenge of Engineering Design The literature is filled with positive comments from students, instructors, and industrial sponsors who have participated in capstone design courses. The vast majority of participants feel that the course benefited all involved. The nature of capstone design courses, however, often leads to a purely subjective evaluation with little or no “hard evidence” of actual benefits. Born, for example, does not attempt to prove the value of senior level design courses. He simply states that he is convinced from his experiences that such courses are valuable. Other educators have similar “feelings” as to the relative costs and benefits of capstone design courses. Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education

12. Challenge of TransferResearches posed this problem to people "Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the health tissue?"

13. Challenge of TransferConsider the following story "A small country was ruled from a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely, since the dictator need to move his troops and workers to and from the fortress. However, any large force would detonate the mines. Not only would this blow up the road, but it would also destroy many neighboring villages. It therefore seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his army into small groups and dispatched each group to the head of a different road. When all was ready he gave the signal and each group marched down a different road. Each group continued down it road to the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator."

14. Challenge of Transfer • After the subjects read and summarized this story, they were asked to solve the tumor problem under the guise of a separate experiment. • Given the clear analogy, you might think that performance would be near ceiling. Surprisingly, only 30% of the subjects offered a convergence solution. • Moreover, when these same subjects were given the suggestion that they should use the General story, 80% provided a convergence solution. • This finding demonstrates that half the subjects could apply the General story to the tumor problem when they were instructed to but did not do so on their own.

15. Challenge of Conceptual Understanding • Force Concept Inventory • Signals and Systems Concept Inventory • Heat Transfer Concept Inventory • Materials Concept Inventory

16. Hestenes Force Concepts Inventory ResultsHistogram of Number of Courses vs. Their Normalized Gain ( ) <g><p> x 100 <g> Pretest Posttest 100% 0% <p> Interactive Engagement TraditionalEngagement FIPE‘99-00 FIPE‘99-00 FIPE‘96-97,‘97-98 CC class, Spring ‘99 Number of Courses(62 Courses Enrolling 6542 Students) FIPE‘98-99 Bin Centers for Normalized Gain (%) From: R. Hake, Am. J. Phys.66, 64-74 (1998), or see hitchcock.dlt.asu.edu/media2/cresmet/hake/

17. One of the Questions from the Force Concept Inventory Hestenes, Wells and Swackhamer, The Physics Teacher30, 141 (1992); Revised edition available at http://modeling.la.asu.edu/R&E/Research.html

18. Hestenes, Wells and Swackhamer, The Physics Teacher30, 141 (1992); Revised edition available at http://modeling.la.asu.edu/R&E/Research.html

19. Signals and Systems Concept Inventory (SSCI) • Continuous-time SSCI • Discrete-time SSCI • Analysis of the 174 CT-SSCI exams revealed a normalized gain between pre- and post-test scores of 0.22 +/- 0.07 which is consistent with the results Hake reported for other concept inventory studies of traditional lecture courses. • http://ece.gmu.edu/~kwage/research/ssci/

20. Heat Transfer Concept Inventory • Another student was confident of “energy balances!” but asked “Should I understand the derivation of the heat diffusion equation and where each term comes from?” The heat diffusion equation is an energy balance equation. • One topic that is typically covered in depth over a one- to two-week period in a heat transfer is the use of fins to increase heat transfer. All of the students expressed uncertainty about this subject. • They appear unable to connect to what they know about thermodynamics (i.e. heat cannot be transferred without a temperature difference) and apply it to the problem at hand (two surfaces at the same temperature).

21. Materials Concept Inventory • Jointly developed between Arizona State and Texas A&M • Preliminary results show normalized gain of about 0.2 in lecture classes and 0.4 in a class with some active learning • Both developers are disappointed in the pre- to post-gains shown by their classes and are actively revising their approaches to teaching

22. C I Questions - MCI Example 3 spoonfuls of salt are stirred into a glass of water, leaving a spoonful of water-saturated salt on the bottom. If a small amount of salt is stirred in, the change in concentration of the salt in the solution is given by curve: _______ b* Pre-test - ASU 39%, TAMU 39%; Post-test -TAMU 66% Critical concept - Solubility limits in phase diagrams

23. C I Questions - MCI Example Student-Generated Distracters After a piece of copper wire from a hardware store is heated it becomes softer. This is because: • The bonds have been weakened • It has fewer atomic level defects * • It has more atomic level defects • The density is lower • There’s more space inside crystal lattice Student-generated distracters are underlined Critical concept is role of defects in metal strength and in metal processing. Pre Post

24. A “Favorite” Misconception from the MCI Nickel, and probably most other metals, cannot exist as a gas, but only as a solid or a liquid. Nickel can exist as: ____ a) solid only b) liquid only c) gas only d) liquid or solid only e) liquid or solid or gas *

25. Change in Engineering Education • Initiating and Sustaining Change • Resistance • Leadership • Culture • Learning, Assessment, Evaluation

26. Initiating and Sustaining Change • “It matters to me.” • Personal mastery and learning • “It matters to a network of my peers.” • Knowledge management • Communities of practice • “It matters to constituents and stakeholders.” • Faculty and curriculum development • Technology transfer and knowledge management

27. Resistance • Resistance is inevitable and comprehensible • Responses to resistance • Ignore • Steamroll • Understand and address • Anticipate, understand, and address

28. Possible Sources of Resistance to Change • “Whatever changes we make, we must be sure to teach the fundamentals!”

29. Possible Sources of Resistance to Change • “Why do we need to change? Our graduates are being hired.”

30. Leadership • Leadership takes place every day. It cannot be the responsibility of the few, a rare event, or a once-in-a-lifetime opportunity.” Heifetz, Ronald and Donald Laurie, “The Work of Leadership,” • Leadership is too important to be left in the hands of the few people near the top of the organizational hierarchy.

31. Change is hard work. Leadership begins with values Intellectual leads physical Real changes takes real change Leadership is a team sport Expect to be surprised Today competes with tomorrow Better is better Focus on the future Learning from doing Grow people Reflect Leadership for Change Sullivan and Harper, Hope is not a Method

32. What is the organizational culture? The culture of a group is a pattern of shared basic assumptions that the group learned as it solved problems … that has worked well enough to be considered valid, and therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems. Artifacts Visible structures and processes Espoused Values Strategies, goals, philosophies Basic Underlying Assumptions Unconscious beliefs, perceptions, thoughts, feelings

33. Changing Culture • “You cannot create a new culture. You can immerse yourself in studying a culture ... Until you understand it. Then you can propose new values, introduce new ways of doing things, and articulate new governing ideas. Over time, these actions will set the stage for new behavior. If people who adopt the new behavior feel that it helps them ... The organizational culture may embody a different set of assumptions, and a different way of looking at things ...” Edgar Schein, in Senge, Peter, The Dance of Change

34. Changing Culture “… culture is the result of all the daily conversations and negotiations between the members of an organization. They are continually agreeing (sometimes explicitly, usually tacitly) about the ‘proper’ way to do things and how to make meanings about the events of the world around them. If you want to change a culture you have to change all these conversations—or at least the majority of them.” (Richard Seel, 2000)

35. Questions?

36. References Fowler, D., Maxwell, D., Froyd, J. (2003) Learning Strategy Growth Not What Expected After Two Years through Engineering Curriculum, Proceedings, ASEE Annual Conference AbstractAs the pace of technological development continues to increase, consensus has emerged that undergraduate science, technology, engineering and mathematics (STEM) curricula cannot contain all of the topics that engineering professionals will require, even during the first ten years of their careers. Therefore, the need for students to increase their capability for lifelong learning is receiving greater attention. It is anticipated that development of this capability occurs during the undergraduate curricula. However, preliminary data from both first-year and junior engineering majors may indicate that development of these competencies may not be as large as desired. Data was obtained using the Learning and Study Skills Inventory (LASSI), an instrument whose reliability has been demonstrated during the past fifteen years. The LASSI is a ten-scale, eighty-item assessment of students’ awareness about and use of learning and study strategies related to skill, will and self-regulation components of strategic learning. Students at Texas A&M University in both a first-year engineering course and a junior level civil engineering course took the LASSI at the beginning of the academic year. Improvements would normally be expected after two years in a challenging engineering curriculum. However, data on several different scales appears to indicate that improvements are smaller than might be expected.

37. References Litzinger, T., Wise, J., Lee, S., Bjorklund, S. (2003) Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference Conclusions The goals of this study were to determine whether students’ readiness for self-directed learning increases as they proceed through an undergraduate engineering program and to determine whether capstone courses increase their readiness for self-directed learning. A related goal was to determine if there are any gender differences in readiness for self-directed learning. The cross-sectional study showed no statistically significant increase in SDLRS scores. This result suggests that most courses that students take in the undergraduate engineering programs do not ask them to undertake tasks that increase their readiness for self-directed learning. No gender differences were present in the cross-sectional study. Although some of the data show an increase in SDLRS scores between a pre-test and post-test in capstone courses in Mechanical and Electrical Engineering, the results failed to support the hypothesis that the complex problem solving required in these courses would lead to increases in readiness for self-directed learning. The lack of a statistically significant increase in SDLRS scores is of concern in an era when lifelong learning is more critical than ever for engineers. The results of the present study will be extended in future work to increase the sample size in the pre-test/post-test study and to document carefully the type of activities that the capstone courses require the students to undertake

38. References Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem Solving Program,” Journal of Engineering Education, Abstract This paper describes a 25-year project in which we defined problem solving, identified effective methods for developing students’ skill in problem solving, implemented a series of four required courses to develop the skill, and evaluated the effectiveness of the program. Four research projects are summarized in which we identified which teaching methods failed to develop problem solving skill and which methods were successful in developing the skills. We found that students need both comprehension of Chemical Engineering and what we call general problem solving skill to solve problems successfully. We identified 37 general problem solving skills. We use 120 hours of workshops spread over four required courses to develop the skills. Each skill is built (using content-independent activities), bridged (to apply the skill in the content-specific domain of Chemical Engineering) and extended (to use the skill in other contexts and contents and in everyday life). The tests and examinations of process skills, TEPS, that assess the degree to which the students can apply the skills are described. We illustrate how self-assessment was used.

39. References Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education Abstract Senior project or Capstone-type courses have existed at engineering schools for many years. Capstone courses provide student engineers the opportunity to solve real-world engineering projects, and have been highly regarded as important learning activities. A survey of Capstone courses in engineering departments throughout North America was conducted in order to understand current practices in Capstone education. This study was conducted for presentation at the 1994 Advances in Capstone Education Conference, held at Brigham Young University, which brought together engineering educators interested in improving Capstone experiences. This conference was sponsored by ASEE, ASME, SME, and NSF. Response to the study was very high, with 360 departments from 173 schools responding to the survey. Survey results were categorized into five major areas of interest: Profile of the Respondents, Course Description Information, Faculty Involvement in Capstone Education, Project Information, and Industrial Involvement in Capstone Education. Graphs provide the results of responses to survey questions.

40. References Medin, D., Ross, B., Markman, A. (2001) Cognitive Psychology, third edition, Fort Worth: Harcourt College Publishers, 470-471 Gick, M., Holyoak, K. (1980) Analogical problem solving. Cognitive Psychology, 12, 306-355 Gick, M., Holyoak, K. (1983) Schema induction and analogical transfer. Cognitive Psychology, 15, 1-38

41. References Hake, R.R. (1988) "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74, http://www.physics.indiana.edu/~sdi/ajpv3i.pdf AbstractA survey of pre/post test data using the Halloun-Hestenes Mechanics Diagnostic test or more recent Force Concept Inventory is reported for 62 introductory physics courses enrolling a total number of students N = 6542. A consistent analysis over diverse student populations in high schools, colleges, and universities is obtained if a rough measure of the average effectiveness of a course in promoting conceptual understanding is taken to be the average normalized gain <g>. The latter is defined as the ratio of the actual average gain (%<post> – %<pre>) to the maximum possible average gain (100 – %<pre>). Fourteen "traditional" (T) courses (N = 2084) which made little or no use of interactive-engagement (IE) methods achieved an average gain <g>T-ave = 0.23 ± 0.04 std dev). In sharp contrast, forty-eight courses (N = 4458) which made substantial use of IE methods achieved an average gain <g>IE-ave = 0.48 ± 0.14 (std dev), almost two standard deviations of <g>IE-ave above that of the traditional courses. Results for 30 N = 3259) of the above 62 courses on the problem-solving Mechanics Baseline test of Hestenes-Wells imply that IE strategies enhance problem-solving ability. The conceptual and problem-solving test results strongly suggest that the classroom use of IE methods can increase mechanics-course effectiveness well beyond that obtained in traditional practice.

42. References • Halloun, I., Hestenes, D. (1985) The Initial Knowledge State of College Physics Students, Am. J. Phys.53, 1043-1055 • Halloun, I., Hestenes, D. (1985) Common Sense Concepts about Motion, Am. J. Phys. 53, 1056-1065 • More information about the Force Concept Inventory is available at http://modeling.la.asu.edu/R&E/Research.html

43. References Wage, K., Buck, J., Welch, T., Wright, C. (2002) The Signals and Systems Concept Inventory, Proceedings, ASEE Annual Conference Abstract This paper describes the development of continuous-time and discrete-time signals and systems concept inventory exams for undergraduate electrical engineering curricula. Both exams have twenty-five multiple choice questions to assess students’ understanding of core concepts in these courses. The questions require little or no computation, and contain incorrect answers that capture common student misconceptions. The design of both exams is discussed, as are ongoing studies evaluating the exams at four campuses. Preliminary results from administering the continuous-time exam as a pre-test and post-test indicate a normalized gain of 0.24 ± 0.08 for traditional lecture courses, consistent with reported results for the Force Concept Inventory exam in lecture courses for freshman physics.

44. References Jacobi A., Martin, J. Mitchell, J., Newell, T. (2003) A Concept Inventory for Heat Transfer Proceedings, Frontiers in Education Conference, http://fie.engrng.pitt.edu/fie2003/papers/1192.pdf AbstractStudents enter courses in engineering with intuitions about physical phenomena. Through coursework they build on their intuition to develop a set of beliefs about the subject. Often, their understanding of basic concepts is incomplete and their explanations are not “correct.” Concept Inventories are assessment tools designed to determine the degree to which students understand the concepts of a subject and to identify the bases for misunderstandings. A cooperative effort between faculty at the Universities of Wisconsin and Illinois has been undertaken to develop a concept inventory for heat transfer. The process initiated with student identification of the conceptual problems rather than with faculty perceptions of student misunderstandings. Students then explored areas of conceptual difficulty and phrased questions that would test understanding of the concepts. Students working together with faculty developed a concept inventory for heat transfer. The presentation will report on the experience with using student groups and the resulting concept inventory.

45. References Krause, S., Decker, J., Griffin, R. (2003) Using a Materials Concept Inventory to Assess Conceptual Gain in Introductory Materials Engineering Courses, Proceedings, Frontiers in Education Conference, http://fie.engrng.pitt.edu/fie2003/papers/1160.pdf AbstractA Materials Concept Inventory (MCI) has been created to measure conceptual knowledge gain in introductory materials engineering courses. The 30-question, multiple-choice MCI test has been administered as a pre and post-test at Arizona State University (ASU) and Texas A & M University (TAMU) to classes ranging in size from 16 to 90 students. The results on the pre-test (entering class) showed both “prior misconceptions” and knowledge gaps that resulted from earlier coursework in chemistry and, to a lesser extent, in geometry. The post-test (exiting class) showed both that some “prior misconceptions” persisted and also that new “spontaneous misconceptions” had been created during the course of the class. Most classes showed a limited, 15% to 20%, gain in knowledge between pre and post-test scores, but one class, which used active learning, showed a gain of 38%. More details on these results, on differences in results between ASU and TAMU, and on the nature of students’ conceptual knowledge will be described.

46. References • Senge, P., Kleiner, A., Roberts, C., Roth, G., Ross, R., Smith, B. (1999) The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations, Double Day

47. References • Mauer, R. (1996) Beyond the Wall of Resistance: Unconventional Strategies That Build Support for Change, Bard Press, see also http://www.beyondresistance.com/index.html

48. References • Heifetz, R., Laurie, D. (1997). The work of leadership. Harvard Business Review, 75(1), 124-134.

49. References • Sullivan, G., Harper, M. (1996) Hope is Not a Method, New York: Random House

50. References Seel, R. (2000) “Culture and Complexity: New Insights on Organisational Change,” Organisations & People, 7(2) 2-9, see also http://www.new-paradigm.co.uk/culture-complex.htm