What’s Happening to Our “Freshout” Engineers?

2. What’s Happening to Our “Freshout” Engineers? https://sites.googboisestate.edu/faculty/svillachica.htmle.com/a/boisestate.edu/ieeci/e2r2p

3. Engineering Education Research to Practice (E2R2P) Portions of this material are based upon work supported by the National Science Foundation under Grant No. 1037808. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. • The Research Team • Don Plumlee, PhD. • Steve Villachica, PhD. • Tony Marker , PhD. • Linda Huglin, PhD. • Shannon Rist • Amy Chegash • Lorece Stanton • Jessica Scheufler

4. Agenda • Share our research • Ask for your help interpreting the data we’ve collected • Wrap up

5. Why Should You Care? • 64% engineering employers are “somewhat satisfied” with quality of new hires.(Trevelyan & Tilli, 2008; Trevelyan, 2010; Blom & Sakei, 2011) • Professional skills for the engineering workplace include teamwork, communication, coordination, data analysis and problem solving.(Hoey & Gardner, 1999; Jonassen et al., 2006; Grant & Dickson, 2006; Korte, Sheppard, & Jordan, 2008; Trevelyan, 2007, 2008; Borrego & Bernhard, 2011; Passow, 2012; ASEE & NSF, 2013)

6. Research Design Boundary Crossing Competencies Communication, teamwork, networks, critical thinking, global understanding, perspective, organizational culture, project management, etc. What sort of engineers do engineering firms REALLY want to hire? ME Many Systems Many Disciplines Deep at least one discipline Deep at least one system (c.f. Brown, 2005; Spohrer, 2010; ASEE & NSF, 2013)

7. Literature Review • Company Costs • Training • Errors • Mentoring • Salary • Opportunity • Other projects • Others? Desired Competency Promotion! Actual Competency $$$ Performance New Task/Project Leave University/Enter Workforce { } Time • Improve Starting Skills • Change Performance Curve • Make Boundaries Porous REDUCE CO$T

8. Literature Review There is a significant disconnect between engineering education and engineering practice. (Bucciarelli & Kuhn, 1997; NAE, 2005; Jonassen et al., 2006; Spinks et al., 2006; Korte et al., 2008; Trevelyan, 2008, 2010; McCrohon & Gibson, 2009; Sheppard et al., 2009; Morgan & O’Gorman, 2010; Anderson et al., 2009, 2010; Duderstadt, 2010; Stump et al., 2011; ASEE & NSF, 2013; Winters et al., 2013)

9. Literature Review The time for freshout engineers to fit into their jobs and perform them competently is a significant workplace cost. • 2 to 5 years ramp-up.(Trevelyan & Tilli, 2008; Jonassen et al., 2006) • Socialization and onboarding are long-lived. (Dai & De Meuse, 2007; Bradt & Vonnegut, 2009; Roethle, 2012; Jones, 2013) • The engineering workplace supports socialization and onboarding variably well. (Montesano, 2007; Roethle , 2012; Korte & Lin, 2013) • Bad onboarding is co$tly. (Ramlall, 2004; Rollag et al., 2005; Snell, 2006; Kowtha, 2008; Lindo, 2010; Roethle, 2012; Korte & Lin, 2013)

10. Literature Review Academics, industry, and government agencies own this shared problem, and it requires a systemic solution. Unfortunately, we know little about • Engineering practice for freshout engineers.(Kowtha, 2008; Trevelyan, 2007, 2008; Brunhaver et al., 2010, in press; Winters et al., 2013) • What freshout engineers do successfully and unsuccessfully in the workplace. (Trevelyan & Tilli, 2008; Trevelyan ,2008, 2009) • Barriers to desired workplace performance.(Korte et al., 2008; Atman et al., 2010; Brunhaver et al., 2010, in press; Anderson et al., 2010) E2R2P is an opportunity to collaborate systemically to decrease ramp-up time to competent performance.

11. Research Design Decrease Ramp-up Time to Competent Job Performance in the Engineering Workplace • Research Questions • What are newly graduated and hired “fresh out” engineers doing/not doing in the workplace that they should? • What are the consequences of performance/non-performance in the workplace? • What are the root causes of workplace nonperformance? • Mixed Design: Focus Groups & Surveys • Engineering managers, engineering leads, HR personnel, and technical scientists who work with fresh out engineers • Fresh out engineers • Professional engineering organizations

12. Thanks to Our Professional and Industry Sponsors! • Practicing engineers at ISPE • BSU COEN Advisory Council • Focus Groups at local engineering firms

13. MethodPracticing Engineer Survey ISPE (2012), n = 23 Short survey measuring: Types of work assigned to freshouts. Typical time to competence Costs and risks that organizations incur when freshouts can’t perform to standards Typical project organization for groups of engineers Organizational support for freshouts

14. Time to Competency for "Fresh-Outs" (in months)

15. Typical Assignment Size

16. Organizational Support

17. MethodFocus Groups Collect Incidents Critical Incident Technique(Flanagan, 1954) Generate Categories Nominal Group Technique (Delp et al., 1975) Negotiate Categories Select Incidents of Non-performance Rank Categories Group Incidents Under a Root Cause Cause Analysis Group Incidents under Categories

18. Collect IncidentsCritical Incident Technique (Flanagan, 1954)

19. Focus Group ResultsPerformance Categories Freshout-Defined Categories n = 10 Manager-Defined Categories n = 20

20. Focus Group ResultsConsequences

21. Categories, Activities and Consequences

22. Categories, Activities and Consequences (con’t)

23. Categories, Activities and Consequences (con’t)

24. Root Cause AnalysisInstrumentation INFORMATION MOTIVATION TOOLS ENVIRONMENT PERSON

25. Root Cause AnalysisResults

26. Root Cause AnalysisResults

27. Limitations • Validity and reliability of the Practicing Engineer Survey is unknown. • Small exploratory study using a convenience sample of local engineering firms. • No post-focus group data checking with participants and their company sponsors (managers).

28. Conclusions • Decreasing time to engineering workplace competency is a shared problem. • Freshout engineers are variably prepared to enter the workplace. • The engineering workplace supports freshout performance variably well. • Socialization and onboarding involves a lot of self-reported learning. • We don’t know about the extent to which fixing the workplace environment and introducing it to students sooner would decrease ramp-up time.

29. Next Steps • Seek funding to expand research. • Include other engineering populations. • Regional, national, or international sample. • Scale up and automate processes. • Investigate research questions about: • Blurring traditional academic and industry boundaries. • The extent to which a smarter workplace environment introduced in academics could decrease ramp-up time.

30. Summary • Research questions and importance • Literature review • Method • Results • Limitations • Conclusions What are your lessons learned? How might you apply them back on the job?

31. References Anderson, K., Courter, S., McGlamery, T., Nathans-Kelly, T., & Nicometo, C. (2009, June). Understanding the current work and values of professional engineers: Implications for engineering education. Paper presented at the American Society for Engineering Education, Austin, TX. Anderson, K.J.B., Courter, S.S., McGlamery, T., Nathans-Kelly, T.M., & Nicometo, C.G. (2010). Understanding engineering work and identity: A cross-case analysis of engineers within six firms. Engineering Studies, 2(3), 153-174. doi: 10.1080/19378629.2010.519772 ASEE & NSF. (2013). Transforming undergraduate education in engineering: Phase I--Synthesizing and integrating industry perspectives. Arlington, VA: American Society for Engineering Education. Retrieved from http://www.asee.org/TUEE_PhaseI_WorkshopReport.pdf Atman, C.J., Sheppard, S.D., Turns, J., Adams, R.S., Fleming, L.N., Stevens, R., . . . Lund, D. (2010). Enabling engineering student success: The final report for the center for the advancement of engineering education. San Rafael, CA: Center for the Advancement of Engineering Education. Retrieved from http://www.engr.washington.edu/caee/CAEE%20final%20report%2020101102.pdf Blom, A., & Saeki, H. (2011). Employability and skill set of newly graduated engineers in India.The World Bank South Asia Region Education Team. Retrieved from http://elibrary.worldbank.org/docserver/download/5640.pdf?expires=1337278148&id=id&accname=guest&checksum=E557A65C13E9CA650433F9E5798C4242 Borrego, M., & Bernhard, J. (2011). The emergence of engineering education research as an internationally connected field of inquiry. Journal of Engineering Education, 100(1), 14-47. Bradt, G.B., & Vonnegut, M. (2009). Onboarding: How to get your new employees up to speed in half the time. Hoboken, NJ: John Wiley & Sons. Brown, T. (2005). Strategy by design. Fast Company, (June 1). Retrieved from http://www.fastcompany.com/52795/strategy-design Brunhaver, S., Korte, R., Lande, M., & Sheppard, S. (2010, June). Supports and barriers that recent engineering graduates experience in the workplace. Paper presented at the American Society for Engineering Education, Vancouver, British Columbia. Brunhaver, S.R., Korte, R., Barley, S.R., & Sheppard, S.D. (in press). Bridging the gaps between engineering education and practice. Bucciarelli, L.L., & Kuhn, S. (1997). Engineering education and engineering practice: Improving the fit. In S. R. Barley & J. E. Orr (Eds.), Between craft and science: Technical work in the United States (pp. 210-229). Ithica, NY: Cornell University Press. Dai, G., & De Meuse, K.P. (2007). A review of onboarding literature. Minneapolis, MN: Lominger Limited. Retrieved from http://boardoptions.com/onboardingevidence.pdf

32. References Dean, P.J. (1997). Thomas f. Gilbert, ph. D: Engineering performance improvement with or without training. In P. J. Dean & D. E. Ripley (Eds.), Performance improvement series: Vol. 1. Performance improvement pathfinders: Models for organizational learning systems (pp. 45-64). Silver Spring, MD: International Society for Performance Improvement. Delp, P., Thesen, A., Motiwalla, J., & Seshardi, N. (1977). Nominal group technique. In P. Delp (Ed.), Systems tools for project planning (pp. 14-18). Bloomington, IN: International Development Institute. Duderstadt, J.J. (2010). Engineering for a changing world. In D. Grasso & M. Burkins (Eds.), Holistic engineering education: Beyond technology (pp. 17-35). New York, NY: Springer. Retrieved from http://milproj.dc.umich.edu/pdfs/2009/Engineering%20for%20a%20Changing%20World.pdf Flanagan, J.C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358. doi: 10.1037/h0061470 Grant, C.D., & Dickson, B.R. (2006). Personal skills in chemical engineering graduates: The development of skills within degree programmes to meet the needs of employers. Education for Chemical Engineers, 1(1), 23-29. doi: http://dx.doi.org/10.1205/ece.05004 Hoey, J.J., & Gardner, D.C. (1999). Using surveys of alumni and their employers to improve an institution. New Directions for Institutional Research, 1999(101), 43-59. doi: 10.1002/ir.10103 Jonassen, D., Strobel, J., & ChweeBeng, L. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139-151. doi: 10.1002/j.2168-9830.2006.tb00885.x Jones, K. (2013). Virtual onboarding for today’s global workforce. Oakland, CA: Bersin by Deloitte. Retrieved from http://communication.on24.com/virtual-onboarding-for-todays-global-workforce Korte, R., & Lin, S. (2013). Getting on board: Organizational socialization and the contribution of social capital. Human Relations, 66(3), 407-428. doi: 10.1177/0018726712461927 Korte, R., Sheppard, S., & Jordan, W. (2008). A qualitative study of the early work experiences of recent graduates in engineering. Paper presented at the American Society for Engineering Education, Pittsburgh, PA. Kowtha, N.R. (2008). Engineering the engineers: Socialization tactics and new engineer adjustment in organizations. IEEE Transactions on Engineering Management, 55(1), 67-81. doi: 10.1109/TEM.2007.912809

33. References Lindo, D.K. (2010). New employee orientation is your job! SuperVision, 71(9), 11-15. McCrohon, M., & Gibson, P. (2009, December). Student experiences in the direct applicability of their engineering education to professional practice. Paper presented at the 20th Annual Conference for the Australasian Association for Engineering Education, Adelaide, Australia. Montesano, A. (2007). Orienting new employees for career success: Effective onboarding ensures smoother integration and faster productivity. Canadian HR Reporter, 20(18), 23-24. Morgan, M., & O’Gorman, P. (2010, July). Developing industry-ready engineers: A regional university perspective. Paper presented at the International Conference on Engineering Education ICEE-2010, Gliwice, Poland. National Academy of Engineering. (2005). Educating the engineer of 2020: Adapting engineering education to the new century. Washington, DC: National Academies. Retrieved from http://www.nap.edu/catalog.php?record_id=11338#toc Passow, H.J. (2012). Which ABET competencies do engineering graduates find most important in their work? Journal of Engineering Education, 101(1), 95-118. doi: 10.1002/j.2168-9830.2012.tb00043.x Ramlall, S. (2004). A review of employee motivation theories and their implications for employee retention within organizations. The Journal of American Academy of Business, Cambridge 5(1/2), 52-63. Roethle, J. (2012). Developing a new employee onboarding program in a small engineering department. Milwaukee School of Engineering, Milwaukee, WI. Rollag, K., Parise, S., & Cross, R. (2005). Getting new hires up to speed quickly. MIT Sloan Management Review, 46(2), 35-41. Sheppard, S., Macatangay, K., Colby, A., & Sullivan, W.M. (2009). Educating engineers: Designing for the future of the field (Vol. 9): Jossey-Bass San Francisco, CA. Snell, A. (2006). Researching onboarding best practice: Using research to connect onboarding processes with employee satisfaction. Strategic HR Review, 5(6), 32-35. Spinks, N., Silburn, N., & Birchall, D. (2006). Educating engineers for the 21st century: The industry view. London: The Royal Academy of Engineering. Stump, G.S., Hilpert, J.C., Husman, J., Chung, W.-t., & Kim, W. (2011). Collaborative learning in engineering students: Gender and achievement. Journal of Engineering Education, 100(3), 475-497.

34. References Trevelyan, J. (2007). Technical coordination in engineering practice. Journal of Engineering Education, 96(3), 191-204. Trevelyan, J. (2008). A framework for understanding engineering practice. Paper presented at the American Society for Engineering Education, Pittsburgh, PA. Trevelyan, J., & Tilli, S. (2008). Longitudinal study of Australian engineering graduates: Perceptions of working time. Paper presented at the American Society for Engineering Education, Pittsburgh, PA. Trevelyan, J.P. (2009). Engineering education requires a better model of engineering practice. Paper presented at the Proceedings of the Research in Engineering Education Symposium, Palm Cove, QLD, Australia. Trevelyan, J.P. (2010). Mind the gaps: Engineering education and practice. Paper presented at the 21st Annual Conference for the Australasian Association for Engineering Education, Sydney, Australia. Winters, K.E., Matusovich, H.M., Brunhaver, S., Chen, H.L., Yasuhara, K., & Sheppard, S. (2013, June). From freshman engineering students to practicing professionals: Changes in beliefs about important skills over time. Paper presented at the American Society for Engineering Education, Atlanta, GA.