Designing a Support System for Research on Engineering Education

1. Designing a Support System for Research on Engineering Education Norman L. Fortenberry, Sc.D. Director, CASEE http://www.nae.edu/CASEE nfortenb@nae.edu Share the Future IV: Tempe, AZ March 17, 2003

2. Schedule • CASEE overview (5-min) • Overview of Potential Research Areas (10 min) • Selection of Group Leaders/Rapporteurs • Delving into Research Areas (45 min) • Break (10 min) • Report back and Discussion (50 min) Slides revised 3-16-03

3. Why Research on Engineering Education? • What if we could • Dramatically raise student retention and graduation rates, while also • Increasing the depth and breadth of learning? • What if we could • Increase the time new faculty devote to establishing their research programs because of greater confidence in the quality of their teaching? . . .

4. Why Research on Engineering Education? • What would it mean, in terms of, • Reduced Costs of Education, • Enhanced Workforce Productivity; • Realized Human Potential?

5. Why NAE –Wm. A.Wulf’s 4-legged stool • 1999, established the Committee on Engineering Education • Stream of reports, workshops, etc. of intrinsic value • Implicit, repeated message that NAE values engineering education • 2000, contributions to engineering education as a formal NAE membership criterion • 2001, created Bernard M. Gordon Prize for Innovation in Engineering and Technology Education • $500,000 on par with Draper (engineering and society) and Russ (bioengineering) Prizes • 2002, creation of a center for scholarship on engineering education

6. CASEE Mission • Enable engineering education to meet, in a significantly better way, the needs of employers, graduate schools, and society at large.

7. CASEE Goal(s) • Improve the quality of engineering education: • Increase its efficiency and effectiveness, • Broaden its appeal to diverse domestic communities, and • Improve the personal satisfaction and professional success of students and faculty

8. CASEE Objectives • Working collaboratively with key stakeholders, CASEE • Encourages rigorous research on engineering education, and • Seeks broad implementation of research findings.

9. Initial CASEE Activities • Build and extend human capacity for the conduct of high quality research on engineering education • Initiate and develop a Fellows program. Fellows will produce research and educational products as well as provide inspiration for change. • Build intellectual capital for the conduct of high quality research on engineering education. • Identify, pursue, and validate relevant research. [J-SEER] • Promote implementation and use of research results. • Initiate and develop an Affiliates program. Affiliates will serve as a test bed for the activities of Fellows, serve as implementation sites, serve to diffuse validated research, and provide a sustaining base for change.

10. CASEE web site • NOW • General information on CASEE, its advisory committee, etc. • Invitations for organizational and individual affiliation • Links to scholarly resources • News • FUTURE • Reports from organizational and individual affiliates

11. Goals/objs of Depts, Univs, Prof. Socs, Emplrs, etc. Teachers & Learners Tools (e.g., curric, labs, tech) Ext. Influences On profession Input Teaching and Learning Processes Output The Engineering Education System Inspired by Hubka and Eder (1988)

12. Research Areas • Teaching, learning, and assessment • Teachers and learners • Instructional/learning tools and technologies • Educational management and goal system • Political, economic, and social influences on engineering education • Diffusion of Educational Innovations • ????

13. Research Areas • Teaching, Learning, and Assessment • Research on instruction, learning and assessment in formal, informal, extracurricular, and lifelong educational settings (e.g, which instructional approaches are most effective under what sets of circumstances); • Research on student and (faculty and graduate student) instructor assessment; and • Research on experiential (laboratory, mentored research, and workplace) learning environments and community (service-learning) practice. • ????

14. Research Areas • Teachers and learners • Research on the behavior and interactions among individual and groups of instructors and students in educational and extracurricular settings, including motivation, advisement, mentoring, and career development of instructors and students; • Research on the images of engineers and engineering held by faculty and students, how these images affect the quality of teaching and learning, and how these images can be productively modified; and • Research on race, ethnicity, gender, physical ability, and social class as lenses for performing critical analyses and evaluations of access to, participation in, and success of engineering study and engineering practice. • ????

15. Research Areas • Instructional/learning tools and technologies • Research on the how curricula, instructional materials, and teaching practices interact to affect learning; • Research on methods and effects of integrating the latest content knowledge into curricula; and • Research on the use of technologies to structure educational environments and transform educational practice. • ????

16. Research Areas • Engineering education management and goal system • Research on the goals, roles, and interactions among and between individual learners, (faculty and graduate student) instructors, departments, colleges, universities, governing boards, advisory committees, accreditors, employers, professional societies, and other stakeholders in higher education; • Research on and evaluation of theory and practices related to issues of efficiency, effectiveness, and improvement of educational organizations; • Research on, development of, and use of educational indicators, including indicators of student learning outcomes, course and curricular quality, staff, academic climate and conditions, demographics, and finance, and the relationship of indicators to accountability. • ???

17. Research Areas • Social, economic, and political influences on engineering education • Research on the relationships between education processes and social, political, and economic contexts in which they operate including the community, climate, and culture aspects of educational settings; • Research on educational finance and its impact on policy for all levels and aspects of education; and • Research on images of engineers and engineering held by the public and various national constituencies, how these images affect the quality of teaching and learning within engineering, and how these images can be productively modified. • ????

18. Research Areas • Diffusion of Innovations • Research on the nature of diffusion of educational research knowledge, expanding understanding of research results, and the practical application of research results; • Research on the teaching and proper usage of quantitative and qualitative educational research methods; and • Research on how education research is used to improve education policy and practice. • ???

19. Selection of Group Leaders and Rapporteurs • Group Leaders • Keep the discussion moving • Keep an eye on time • Rapporteurs • Capture key points of the discussion for later presentation

20. Research exists on A B C New or continuing research is needed on D E F Research topic “x” What do we know, and where are the gaps? What new areas should be pursued? How do we get what is known better used in practice?

21. Reality check • Is the model of engineering education reasonable? • Are there additional issues that should be addressed? • Is the suggested mode of reporting out the research agenda reasonable? • In addition to this forum, what other inputs might help move this effort forward? • How best should this effort be communicated and in what venues?

22. 3-minuteBREAK!

23. REACTIONS: • Technology should be broken out from “Instructional/learning tools and technologies” as a separate research area. • Other disciplines (e.g., medicine, law, business) with relevant research on education should be included. • Offices of Institutional Research and Schools of Education are natural on-campus partners. • Emphasis should be placed on diffusion, transition of results to faculty, and practical implementation. • Counterpoint to above – successful researchers in technical engineering research are not asked to do this; if education researchers do, will they always be viewed as less scholarly?

24. Research exists on: See bibiliog from “How People Learn” session led by Jeff Froyd See Journal on Eng Ed, Journal of Research in Science Teaching, Review of Educational Research, Journal of the Learning Sciences, SEE CASEE weblist etc. Research is needed on: Need baseline studies on current state (faculty attitudes, goals and expectations of students) What kinds of awareness and attitudes exist on existing teaching methods – [National Survey of Faculty Engagement in development by George Kuh and others] Teaching and Learning IMPLEMENTATION: Development of separate track, within departments and/or institution as a whole, to help faculty become better teachers

25. Research exists on: Lots of stuff on gender and ethnicity “Women and Men of the Engineering Path” by Cliff Adelman, US Dept. of Ed. Research is needed on: Need more on economic subgroups Need more on influences on career selection (esp at K-12 levels) What teaching attributes (role modeling, etc) support engineering career selection Effect of engineering examples on career selection (ballistic models versus social equity models of engineering principles. Post high school preparation for collegiate engineering study (useful or not) Effective modes of education in distance learning environments CAEE looking at questions related to being and becoming an engineer Teachers and Learners IMPLEMENTATION:

26. Research exists on: No research, but we know high attrition rates, lack of diversity, declining enrollments Progress through concept inventories (tests) Things that work in the classroom Research is needed on: Need to understand why current bad situation Contextualization (tied to use of concept inventories) also tied to applications of concepts Gender diversity within and among engineering disciplines (why do more women go into IE than EE) – See studies by Rosser, etc. Efficacy of courses and curricula (need longitudinal studies and feedback from industry etc. Likely to be qualitative research Need more on faculty self assessment and student assessment Instructional Tools and Technologies IMPLEMENTATION: Need dollars to support studies Need research on transfer/diffusion mechanisms

27. Research exists on: Some in institutional research lit on role of accreditation – Journal of Research in Education and Higher Education Journal C Research is needed on: Employer feedback (some work at GaTech and UTK) – recognize differences by level in industrial organization; how to reconsile Learner and faculty expectation Whether Arts and Sci faculty dominate institutional goal systems Longitudinal studies on effects of accreditation Management and Goal System IMPLEMENTATION: Make explicit what is to be taught (learning goals) Adopt more of a market orientation Sensitivity to learning styles of students versus teaching styles of faculty (generational issue) Need more funds for studies (critical for requisite rigor) especially longitudinal which go beyond 3-year typical grant timeline. Engage learned and professional societies – they need to publish new journals targeted to Engineers (targeted in terms of presentation, methods, continuous improvement model) – e.g., does it look like what engineers are used to seeing. Univ of Denmark did 20-yr study of problem-based study

28. Research exists on: A B C Research is needed on: Effective of collaboration across disciplinary lines (outside of engineering), ethnic and gender lines, etc. Modes of moving organizational attitudes on research on ed (impt for P&T) B C Soc/Pol/Ec Influences IMPLEMENTATION: Need to understand how to get respect for education researchers within engineering (obviously funding and right administrators necessary, but not sufficient). ABET has made progress, but activities live within committees that may not be sensitive. Need institution to support education research as a valued activity

29. Research exists on: Rogers model is only one, others include Clayton Christiansen “Innovators Dilemma” Disruptive technologies Parker Palmer – movement approach to change Peter Senge – Fifth discipline (has a workbook called “Fifth discipline Fieldbook”) Organizational change literature (bsns) Research is needed on: Next steps in moving diffusion forward Need predictive models, not just descriptive Diffusion of Innovation IMPLEMENTATION: Need clarity in applying models in specific situations. Studying dynamic situations – how to study a changing system? Need means to overcome known barriers Issue with ABET visitors as an ineffective vector?

30. Research exists on: Limited data on effectiveness of various technologies – mostly pilots not full longitudinal studies Lots of data on personal response systems (e.g., Mazur at Harvard) Research is needed on: Full scale longitudinal studies with large sample sizes Technology IMPLEMENTATION: Lots of existing technologies (distance, personal response systems, etc.) Need more rigorous and conclusive research with large sample sizes Questions on methodologies, bias within studies Are we using new tools to do the same old “bad” pedagogies?