Designing a Support System for Research on Engineering Education

1. Designing a Support System for Research on Engineering Education Norman L. Fortenberry, Sc.D. Director, CASEE http://www.nae.edu/CASEE nfortenb@nae.edu 202-334-1926 University of Idaho May 1, 2003

2. Schedule • Overview of Potential Research Areas (15 min) • Delving into Research Areas (45 min)

3. Research Areas • Teaching, learning, and assessment • Teachers and learners • Instructional/learning tools and technologies • Educational management and goal system • Political, economic, and social influences on engineering education • Diffusion of Educational Innovations • ????

4. Research Areas • Teaching, Learning, and Assessment • Research on instruction, learning and assessment in formal, informal, extracurricular, and lifelong educational settings (e.g, which instructional approaches are most effective under what sets of circumstances); • Research on student and (faculty and graduate student) instructor assessment; and • Research on experiential (laboratory, mentored research, and workplace) learning environments and community (service-learning) practice. • ????

5. Research Areas • Teachers and learners • Research on the behavior and interactions among individual and groups of instructors and students in educational and extracurricular settings, including motivation, advisement, mentoring, and career development of instructors and students; • Research on the images of engineers and engineering held by faculty and students, how these images affect the quality of teaching and learning, and how these images can be productively modified; and • Research on race, ethnicity, gender, physical ability, and social class as lenses for performing critical analyses and evaluations of access to, participation in, and success of engineering study and engineering practice. • ????

6. Research Areas • Instructional/learning tools and technologies • Research on the how curricula, instructional materials, and teaching practices interact to affect learning; • Research on methods and effects of integrating the latest content knowledge into curricula; and • Research on the use of technologies to structure educational environments and transform educational practice. • ????

7. Research Areas • Engineering education management and goal system • Research on the goals, roles, and interactions among and between individual learners, (faculty and graduate student) instructors, departments, colleges, universities, governing boards, advisory committees, accreditors, employers, professional societies, and other stakeholders in higher education; • Research on and evaluation of theory and practices related to issues of efficiency, effectiveness, and improvement of educational organizations; • Research on, development of, and use of educational indicators, including indicators of student learning outcomes, course and curricular quality, staff, academic climate and conditions, demographics, and finance, and the relationship of indicators to accountability. • ???

8. Research Areas • Social, economic, and political influences on engineering education • Research on the relationships between education processes and the social, political, and economic contexts in which they operate including the community, climate, and culture aspects of educational settings; • Research on educational finance and its impact on policy for all levels and aspects of education; and • Research on images of engineers and engineering held by the public and various national constituencies, how these images affect the quality of teaching and learning within engineering, and how these images can be productively modified. • ????

9. Research Areas • Diffusion of Innovations • Research on the nature of diffusion of educational research knowledge, expanding understanding of research results, and the practical application of research results; • Research on the teaching and proper usage of quantitative and qualitative educational research methods; and • Research on how education research is used to improve education policy and practice. • ???

10. REACTIONS: • ???

11. Research exists on A B C New or continuing research is needed on D E F Research topic “x” What do we know, and where are the gaps? What new areas should be pursued? Implementation How do we get what is known better used in practice?

12. Research exists on: A B Research is needed on: How do we help students learn? How has technology helped students? What assessment methods are appropriate for eval. use of technology? How to improve and assess student retention of information? How to assess which students should be retained in eng. progs. – academic performance? How do we assess which students should be recruited to engineering? – Issue is ability to predict match between departmental expectations and the ability of students to meet those expectations? Teaching and Learning IMPLEMENTATION: A.

13. Research exists on: A B Research is needed on: How do you know baseline knowledge of students and instructors at course beginning? How do you measure growth/value-added of instruction within course? How do you assess baseline and value-added within impeding the teaching and learning processes? How to separately assess individual performance within context of group performance? Do engineers consider the profession noble? – Relates to non-material Teachers and Learners IMPLEMENTATION:

14. Research exists on: A B C D Research is needed on: Have advances in technology helped or hindered engineering estimation and engineering judgment? How to develop a competency- and value-driven curriculum. How to balance virtual reality and real-life in classroom/laboratory? (How virtual is virtual?) Effect on human interaction and informal interactions if interactions is limited to formal interaction at set times Effect of video monitoring (-- related to cameras in the courtroom) Differences in student and teacher reactions? Instructional Tools and Technologies IMPLEMENTATION:

15. Research exists on: A B Research is needed on: Proper indicators for depts and reward system at college, dept, and individual faculty levels? Benefits of incorporating education practice classes/workshops for new faculty? How to attract diverse student groups? If research has been done, how to broaden faculty access to that knowledge? Management and Goal System IMPLEMENTATION:

16. Research exists on: A B C Research is needed on: Perceived differences between scientists and engineers – relevant to perception of engineering by an important peer community and what can be adopted between those communities and how we advise students in choosing career paths within them. How do we best facilitate the growth of developing learners so that they begin to think broadly – ethics, societal impacts? How do social and economic standing (within society at large) affect one’s view of the nobleness of engineering? – May help to recruit more students to study engineering. Soc/Pol/Ec Influences IMPLEMENTATION:

17. Research exists on: A B Research is needed on: N N Diffusion of Innovation IMPLEMENTATION:

18. Summary Results from Previous Workshops

19. Research exists on: See bibiliog from “How People Learn” session led by Jeff Froyd See Journal on Eng Ed, Journal of Research in Science Teaching, Review of Educational Research, Journal of the Learning Sciences, SEE CASEE weblist etc. Research is needed on: Need baseline studies on current state (faculty attitudes, goals and expectations of students) What kinds of awareness and attitudes exist on existing teaching methods – [National Survey of Faculty Engagement in development by George Kuh and others] Teaching and Learning IMPLEMENTATION: Development of separate track, within departments and/or institution as a whole, to help faculty become better teachers

20. Research exists on: Lots of stuff on gender and ethnicity “Women and Men of the Engineering Path” by Cliff Adelman, US Dept. of Ed. Research is needed on: Need more on economic subgroups Need more on influences on career selection (esp at K-12 levels) What teaching attributes (role modeling, etc) support engineering career selection Effect of engineering examples on career selection (ballistic models versus social equity models of engineering principles. Post high school preparation for collegiate engineering study (useful or not) Effective modes of education in distance learning environments CAEE looking at questions related to being and becoming an engineer Teachers and Learners IMPLEMENTATION:

21. Research exists on: No research, but we know high attrition rates, lack of diversity, declining enrollments Progress through concept inventories (tests) Things that work in the classroom Research is needed on: Need to understand why current bad situation Contextualization (tied to use of concept inventories) also tied to applications of concepts Gender diversity within and among engineering disciplines (why do more women go into IE than EE) – See studies by Rosser, etc. Efficacy of courses and curricula (need longitudinal studies and feedback from industry etc. Likely to be qualitative research Need more on faculty self assessment and student assessment Instructional Tools and Technologies IMPLEMENTATION: Need dollars to support studies Need research on transfer/diffusion mechanisms

22. Research exists on: Some in institutional research lit on role of accreditation – Journal of Research in Education and Higher Education Journal C Research is needed on: Employer feedback (some work at GaTech and UTK) – recognize differences by level in industrial organization; how to reconcile Learner and faculty expectation Whether Arts and Sci faculty dominate institutional goal systems Longitudinal studies on effects of accreditation Management and Goal System IMPLEMENTATION: Make explicit what is to be taught (learning goals) Adopt more of a market orientation Sensitivity to learning styles of students versus teaching styles of faculty (generational issue) Need more funds for studies (critical for requisite rigor) especially longitudinal which go beyond 3-year typical grant timeline. Engage learned and professional societies – they need to publish new journals targeted to Engineers (targeted in terms of presentation, methods, continuous improvement model) – e.g., does it look like what engineers are used to seeing. Univ of Denmark did 20-yr study of problem-based study

23. Research exists on: A B C Research is needed on: Effective of collaboration across disciplinary lines (outside of engineering), ethnic and gender lines, etc. Modes of moving organizational attitudes on research on ed (impt for P&T) B C Soc/Pol/Ec Influences IMPLEMENTATION: Need to understand how to get respect for education researchers within engineering (obviously funding and right administrators necessary, but not sufficient). ABET has made progress, but activities live within committees that may not be sensitive. Need institution to support education research as a valued activity

24. Research exists on: Rogers model is only one, others include Clayton Christiansen “Innovators Dilemma” Disruptive technologies Parker Palmer – movement approach to change Peter Senge – Fifth discipline (has a workbook called “Fifth discipline Fieldbook”) Organizational change literature (bsns) Research is needed on: Next steps in moving diffusion forward Need predictive models, not just descriptive Diffusion of Innovation IMPLEMENTATION: Need clarity in applying models in specific situations. Studying dynamic situations – how to study a changing system? Need means to overcome known barriers Issue with ABET visitors as an ineffective vector?

25. Research exists on: Limited data on effectiveness of various technologies – mostly pilots not full longitudinal studies Lots of data on personal response systems (e.g., Mazur at Harvard) Research is needed on: Full scale longitudinal studies with large sample sizes Technology IMPLEMENTATION: Lots of existing technologies (distance, personal response systems, etc.) Need more rigorous and conclusive research with large sample sizes Questions on methodologies, bias within studies Are we using new tools to do the same old “bad” pedagogies?