Navigating a pathway between academic standards and outcomes-focused thinking in engineering

1. Navigating a pathway between academic standards andoutcomes-focused thinkingin engineering Wageeh Boles National Teaching Fellow ALTF Forum Presentation – June 2013 www.nationalteachingfellowshipboles.com Support for the fellowship has been provided by the Australian Government Office for Learning and Teaching. The views in this presentation do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.

2. Context and Background Outline • Methodology • Fellowship activities • Fellowship impact and outcomes • Concluding remarks Wageeh Boles

3. Context and Background • The International Scene • National scene • The Local scene Wageeh Boles

4. Context and Background (cont.) The International Scene International Accreditation agreements Focus on what is learned rather than what is taught Worker Mobility Wageeh Boles

5. Context and Background (cont.) The National Scene Tertiary Education Quality and Standards Agency, TEQSA Excellence in Research Australia, ERA Academic Standards and Threshold Learning Outcomes Australian Qualifications Framework, AQF Focus on Quality Wageeh Boles

6. Context and Background (cont.) The Local Scene Responsibilities of Heads of Schools or Departments Demographics of the Academic workforce Challenges facing early- and mid-career Academics Wageeh Boles

7. Context and Background (cont.) FellowshipObjectives • Identify and evaluate international and Australian initiativesfor effectiveteacherengagementand curriculumreform. • Assist in mapping program outcomes onto appropriately aligned assessment tasks. • Establish a developmental model for designingassessment tasks capable of providing evidenceoflearning. • Facilitate the developmentofearly-andmid-careeracademics in teaching as well as research. Wageeh Boles

8. Fellowship Program Overview Wageeh Boles

9. Focus on people and outcomes Research … Why? Mentoring … Why? Teaching … Why? Wageeh Boles

10. Methodology ProfessionalPracticeResearch* is used with investigations characterised as: • “Critically-informed,” situated within global and national trends in engineering education, • “Politically-activist,” with the researcher acting as an agent of curriculum and cultural change, and • “Action-oriented,” involving university leadership, and working with Heads of Schools as mentors. * Macpherson, I., Brooker, R., Aspland, T., & Cuskelly, E. (2010). Constructing a territory for professional practice research: Some introductory considerations. In Action research in education: Fundamentals of applied research, Vol. 1, Sage Publications Ltd. [ISBN 9781848606838] Wageeh Boles

11. Locations visited in the USA Overseas Study Wageeh Boles

12. People interviewed in the USA • Dr.Norman Fortenberry, Director, American Society of Engineering Education, ASEE, Washington D.C., USA • Dr. Michael Milligan, Executive Director, Accreditation Board for Engineering and Technology, ABET, Baltimore, MD, USA • Dr. Ashley AterKranow, Managing Director, Accreditation Board for Engineering and Technology, ABET, Baltimore, MD, USA • ProfessorMilton Cox, Miami University of Ohio, USA • AssistantProfessor Peter Jamieson, Miami University of Ohio, OH, USA • Professor Anna Dollar, Miami University of Ohio, OH, USA • Professor Osama Ettouney, Miami University of Ohio, OH, USA • Dr. Donald Ucci, Miami University of Ohio, OH, USA • Dr. James Moller, Miami University of Ohio, OH, USA • Mr. Peter Jamieson, Miami University of Ohio, OH, USA • Dr.DmitriyGarmatyuk, Miami University of Ohio, OH, USA • ProfessorJeffrey Froyd, Director of Faculty and Organisational Development, Texas A & M University, TX, USA • AssociateProfessor Susan Lord, Director, Optoelectronics Laboratory, University of San Diego, CA, USA • Professor Geoffrey Orsak, Dean of the SMU Lyle School of Engineering, Southern Methodist University, TX, USA • Associate Professor James Dunham, Associate Dean, Academic Affairs, Lyle School of Engineering, Southern Methodist University, TX, USA. • ProfessorRichard Felder, North Carolina State University, NC, USA • Associate ProfessorMary Besterfield-Sacre, Swanson School of Engineering, Director, Engineering Education Research Center, University of Pittsburgh, PA, USA. • Professor Sam Spiegel, Associate Director, Engineering Education Research Center, University of Pittsburgh, PA, USA. • Professor Larry Schuman, Senior Associate Dean for Academic Affairs, Swanson School of Engineering, University of Pittsburgh, PA, USA • Assistant Professor Melissa Bilec, Director, Construction Management Program and Green Construction Program, University of Pittsburgh, PA, USA. • Professor Joe McCarthy, William Kepler Whiteford, Department of Chemical and Petroleum Engineering, University of Pittsburgh, PA, USA. • Assistant Professor Karen Bursic, Undergraduate Program Director, University of Pittsburgh, PA, USA • Assistant ProfessorNatasaVidic, research interests in assessment and evaluation), University of Pittsburgh, PA, USA • Professor Annette Jacobson, Teaching Professor of Chemical Engineering, A/ Dean for Undergraduate Education, Director, Carnegie Mellon University, PA, USA • Professor Paul Steif, ,Mechanical Engineering, Engineering education leader, Carnegie Mellon University, PA, USA • Associate Professor Marsha Lovett, Associate Teaching Professor, Director, Learning and Problem Solving Lab, Carnegie Mellon University, PA, USA • Professor Michael Prince, Professor of chemical engineering, engineering education leader, Bucknell University, PA, USA • Associate Professor Candace Stefanou, Associate professor of education, Bucknell University, PA, USA • Associate professor Margot Vigeant, Associate professor of Chemical engineering, Bucknell University, PA, USA • Associate Professor David Kelley, Chair, Electrical Engineering Dept. Bucknell University, PA, USA • Professor Keith Buffinton, Dean of Engineering and Professor of Mechanical Engineering, BucknellUniversity, PA, USA • Associate professor Geoffrey Schneider, Associate professor of economics Director of the Teaching and Learning Center, Bucknell University, PA, USA • Associate professor Timothy Raymond, Associate Professor of chemical engineering, Bucknell University, PA, USA. • Professor Maurice Aburdene, Professor of Electrical Engineering (in charge of accreditation), Bucknell University, PA, USA. • Rosemary Aguilar, Director, Professional Development and Curriculum, The Infinity Project, Southern Methodist University, TX, USA. • Diana McAtee, Director, Academic Relations, The Infinity Project, Southern Methodist University, TX, USA. • Professor Sam Spiegel, Associate Director of Outreach & Development, EERC, Pittsburgh, PA. Wageeh Boles

13. Places and foci - 1 • Miami University, Oxford, Ohio Faculty Learning Communities • Accreditation Board for Engineering and Technology, ABET, Baltimore, Maryland Focus on Outcomes • American Society for Engineering Education, ASEE, Washington DC Global Issues • Southern Methodist University, Dallas, Texas Industrycollaboration Wageeh Boles

14. Places and foci - 2 • Bucknell University,Lewisburg, PA Addressing misconceptions • Pittsburgh University, Pittsburgh, PA Model Eliciting Activities, MEAs, and inverted classroom • Carnegie Mellon University, Pittsburgh, PA Concept Inventories • Frontiers in Education, FIE, conference, Rapid City, South Dakota Celebrating innovation Wageeh Boles

15. FIRE = Fellow-In-Residence Engagement • WHAT, WHY, WHO, and HOW? • What is the FIRE program? • What were the objectives? • How was the program implemented? • Which Australian universities participated? • Who was involved at the participating universities? • How was it managed? FIRE program Wageeh Boles

16. FIRE program activities with: University and faculty leadership: • Meetings with the deputy vice-chancellor (academic) or equivalent; the executive dean; the assistant dean (learning and teaching) or equivalent. • An open seminar (university wide) on the issues of assessment and mentoring. • A workshop (co-facilitated by an international scholar, when possible). Wageeh Boles

17. FIRE program activities with: Head of school: • Explore available mentoring programs and how they are achieving their objectives. • Work as a shadow mentor with the head of school for academic staff. • Discuss relevant issues, as may be determined during the visits. Wageeh Boles

18. FIRE program activities with: Academics: Individual and small group meetings to explore: • How can assessment be used for learning? • Designing specific assessment tasks. • How can classroom activities encourage student learning? • What are possible resources (websites, papers, books, packages, etc.)? • What are teaching for learning needs and possible ways to meet them? • Networking opportunities, being part of a learning community. • How to achieve steady enhancement without too much overhead. • Ideas on balancing teaching. • Research, etc. Wageeh Boles

19. A sample discussion starter • Which of the following three scenarios best describes your assessment practices? • Your answer must reflect what actually happens, rather than what you think should happen. • Ask yourself, if someone else examined my assessment, will they arrive at the same conclusion? What about evidence of learning? Wageeh Boles

20. Scenario #1 Learning Objectives Assessment task Wageeh Boles

21. Scenario #2 Learning Objectives Assessment task Wageeh Boles

22. Scenario #3 Learning Objectives Assessment task Wageeh Boles

23. Evidence of learning • Why is evidence of learning important? • For students so that they know how they are progressing. • For academics so that they take appropriate actions to support learning, modify teaching methods, examine or provide resources, link with course objectives, etc. • For accreditation bodies so that they can make an informed assessment of how institutions are meeting their obligations to deliver certified and competent engineers Wageeh Boles

24. Fellowship impact • Impact on participating Australian universities • Impact on individuals and groups • Impact on US participants • Impact through the reference group • Insights gained • Impact on the fellow • Evidence of change Wageeh Boles

25. Fellowship outcomes • Seminars, workshops and papers • Assessment design process and exemplars • Assessment resources • Mentoring guides • The FIRE program as a model for academic staff development and leadership support • Data from the US interviews • Website: www.nationalteachingfellowshipboles.com Wageeh Boles

26. Concluding remarks • The fellowship was motivated by the changes in the higher education environment, including the demographics of academics. • The program had a focus on academic standardsand evidence of learning,and also onmentoring. • Two components; overseas study, and a Fellow-In-Residence Engagement, FIRE,program. Wageeh Boles

27. Concluding remarks (cont.) • The links between course objectives, unit objectives and how specific assessment items can address those objectives systematically, are often missing or not made explicit. • When the evidence required to demonstrate the achievement of specific learning outcomes is not clearly identified, it will be difficult to choose the most appropriate assessment instrument. Wageeh Boles

28. Concluding remarks (cont.) • The lack of understanding the role of assessment or the ability to design it appropriately, makes it difficult to defend claims of attainment of objectives, academic standards, or student learning outcomes. • This presentation gave a very brief overview of the fellowship activities and impact. Challenges faced and how they have been addressed or managed are highlighted in the fellowship final report. Wageeh Boles

29. A quote ... “One might argue that we already have sufficient research on student success (at university) ... What is missing in our view is the ability to transform the knowledge that we have into practical knowledge.” Vincent Tinto Professor and chair of the higher education program, Syracuse University Speaking at the US National Symposium on Student Success at College & University November 2006 Wageeh Boles

30. ... and a thought... I believe that our role is not only to help students become better learners, but to also contribute to building their characters; enabling them to become valuable members of society. Wageeh Boles

31. Questions? Wageeh Boles