An Industry Leader's View of Engineering Education

1. An Industry Leader's View of Engineering Education Steve Kirschstk@propel.com

2. Agenda • My qualifications • Ideas on improving • pre-school • design and process • feedback

3. Why should you listen to me? • I have no credibility in the field of education • …but the EECS auditorium at MIT is named after me • I represent the customer • A fresh perspective

4. Engineering a “better education” • 3 ways to make a better product • Better parts (“raw materials”) • Better product design and manufacturing process • Use feedback for quality assurance (QA) and for ideas on how to continuously improve • Same is true of education • Better educated students coming into the system • Improve what we teach, how we teach it, and who teaches it • Ask students 0-15 years later for feedback and actually use it to change the previous 2 steps

5. Improving the quality of the incoming students

6. ’95 TIMSS results42 countries

7. Texas had state/local control… imagine what a focus on testing can do for the rest of the country!

8. The vision All students in the USA receive the best K-12 education in the world TIMSS 12th grade 4th grade

9. How do we do that? • Establish clear, measurable goals • Example: For K-12, it could be NAEP or TIMSS improvements • Change what we do • Copy what works • Stop doing stuff that doesn’t work • Measure and adjust This isn’t rocket science. It’s just that we don’t do it very well.

10. Changes required • Understand why we have failed in the past • Avoid those mistakes • Realize that K-12 is a federal problem not a local problem: the states have had their chance for over 200 years, yet no state “stands out” of the pack (NAEP). State/local control can make things dramatically worse, e.g., Texas’ results on TASP. • Understand why other countries have succeeded and adopt “best practices” of top performing countries. Copying first to get to parity, innovating later. • Have the leadership and courageto do things dramatically differently than we are today • Create avision, clear goals,and abelievable strategybased on what hasproven to work

11. Why isn’t there a checklist for education that we actually use? • In aviation, a pilot uses an extensive proven checklist to ensure a safe flight… Why not offer a substantial on-going cash incentive to enable schools to pass a provenchecklist of statisticallyprovenreplicable requirements that ensures a quality education?

12. Why don’t we run K-12 schools like airlines? A. Airlines governed by national safety standards (not set by airlines!) B. All pilots are qualified and certified to fly the plane type (no unqualified substitutes allowed!) C. On-going pilot training is required D. Pilots who don’t perform can be fired E. Planes that don’t meet code can’t be used F. Pilots free to determine how to fly the plane, but not the destination G. Require pilots to go through a proven safety checklist (that ensures a successful flight) before takeoff

13. How bad are things today? • There are >50 state standards for what students are to learn. Everyone else has 1 national standard. • Alignment (standards, curriculum, materials, assessments) is therefore virtually impossible • Imagine if Bill Gates were forced to write Windows for each state? • We are set up to fail. • There is no good reason we can’t have national standards. That is step #1. • NAEP is perfect proof of “de facto” national standards. It can be done.

14. Changing how we teach

15. Why the sudden discontinuity after 12th grade?

16. Methods to improve the process • Set clear, measurable goals • Understand very clearly what you are trying to achieve • Copy what works • Copying is always the most efficient way to get to world class performance • Innovate later • Innovation is required for progress AFTER you are #1

17. Metric ideas • TIMSS-like standardized assessment tests • Self-perceived customer satisfaction • Peer/supervisor rated satisfaction • % employed after 2 years • % who were fired in first 10 years • Average salary improvement after 4 years • % who got jobs after graduation

18. Wrong way: NCLB • # of schools that NCLB was tested on before being rolled out nationally • ZERO • Amount of improvement that can be expected from NCLB • Nobody has a clue. Could be negative. • Even worse: we knew Bush’s model (TASS) failed We should NEVER be experimenting on our kids at mass scale like this!!

19. Why not treat education policy like drugs? • Efficacy • Require proof of efficacy, i.e., it has to work • Safety • Require testing on a diverse population to look for unintended consequences (negative side effects) • Scalability • Prove that it can be rolled out in scale and still work (e.g. GMP manufacturing standards)

20. Right way: NCEE’s America Choice • Research best practices • Spend 11 years studying best practices in other countries • Prototype • Create a prototype approach based on the learnings but adapted for the US market • Test and validate • Test it at few schools • Get a third party to validate results by comparing to other schools of similar profile • Scale • Roll out to more schools • Re-test • Measure again to make sure it scaled with same results and without any unintended consequences

21. Changing what we teach Give people what they need to be successful

22. The Kirsch dietIs it good for you? • Genetically, we are all virtually identical. So shouldn’t this pie chart work for everyone? You can just pick the foods you want, but EVERYONE is required to stick to the %’s

23. MIT • 360 units to graduate • 48 units humanities (pick from traditional subjects) • x units core subjects • y units elective • z units General Institute requirements • They review this every 50 years, whether they need to or not • Why aren’t there multiple pie charts? Does one pie really fit all? • executive • Manager • entrepreneur • researcher • engineer

24. MIT • In 2004, they discovered that communication is more than technical writing

25. Presentation skills Teaching skills Leadership Teamwork Negotiation skills Nonverbal communication Decision-making Company politics Giving feedback Receiving feedback Sales skills Attire Philanthropy Behavioral change Hiring and firing people Interviewing skills Social skills, e.g., how to get a date Basic project management skills Managing complex projects Organization skills Coaching/mentorship Working a room Running a meeting Conflict resolution Public policy Why aren’t we teaching soft skills? “I would love to take such a course” - Tony Eng, MIT

26. Soft skills • If there is one change you can make right now to improve engineering education, this is it • Few offerings are comprehensive • Even fewer universities (none?) require this to graduate • Don’t try to integrate it into existing course; copy what works • These skills have been taught very successfully in standalone single-topic seminars

27. MIT hasn’t “got it” yet (after 20 years of prodding) • 6.UAT/6.ThT  Preparation for Undergraduate Advanced Project/Masters of Engineering Thesis • Upon completion of the course, students will have learned how to: • propose and define research problems and think about solutions • critically evaluate technical presentations • architect technical presentations • present technical material in oral and written forms to different audiences at different levels of detail • give and receive constructive feedback • write progress reports

28. Why not also teach basic survival skills? • Basic investment skills • Do you know why a stock goes up when they announce bad news? • Do you know why most people who trade stocks frequently lose money? • Personal relationships

29. What else should we teach? • The answer is in the feedback section!

30. Changing who is doing the teaching

31. Faculty changes • Require a 6 week training course for new faculty • West Point does that • Require people who teach engineering to have “real world” experience in industry • Example: take off a semester and get a full time job • Change incentives to reward collaboration, teamwork, and teaching • Not just individual research!

32. Measure and adjust Creating a feedback system for quality assurance and continuous improvement

33. Education is an open-loop op amp Jobexperience Collegeeducation Students in Students out

34. Example of using feedback: Why executives fail • Interpersonal skills (CFO, VP Engr) • Lack of teamwork • Lack of confidence in the team to work together (VP Sales) • Inability to execute • Use their knowledge and skills to lead a team to success (VP Marketing)

35. What’s common? • They failed on the soft skills • Nobody failed due to a lack of understanding of the basic subject matter in their core area of expertise • Everyone failed due to an inability to translate their knowledge into action

36. The Paul Cook success model • Q: How do you spend your time? • A: On people problems. • So feedback should measure: • what skills are the most used • what skills they could use to be more effective • what skills were required for success • what skills led to failure

37. My experience at MIT was lopsided • Technical skills • BEFORE<<AFTER • People skills • BEFORE = AFTER

38. Why isn’t there a feedback system in place? • What are our goals? • Do we have metrics aligned to those goals? • Do we know what they are? • Is it used as a basis for compensation? • Is it used as a basis for adjusting the process and the design?

39. Methods for driving change • Change the “change process” • Internally driven changes at universities are typically done on geological time scales • To change, copy what has worked at universities who have solved the “lack of change” problem • Start simply: Pick one or two metrics for driving improvement • ABET can drive changes • Establish standardized customer satisfaction metrics • Phase in a few key checklist items: Are new faculty members required to learn how to teach? • Phase in a few key optional items: Do students demonstrate mastery of soft skills? • … etc. • CASEE could create and market a single “J.D. Power” metric for rating engineering schools on customer satisfaction • Could be coupled with a prize for “most improved” • A single combined metric is easier to focus on • Can be component metrics

40. Influence Public Policy e.g., your vote on Nov. 2

41. Significant negative impacts from Bush decisions • Cutting federal grants: NSF, etc. • Putting our kids at a permanent disadvantage • Deregulating mercury emissions from power plants has led to the possibility of permanent brain damage in 15% of the children born in the US today • The requirement to remove 95% of the mercury emissions would have costed 1% of the cost of the plant • Ignoring/distorting science • Cutting funding on his own education bill from what Congress proposed • Believing unbelievable test results (TASS) then using that as a basis for national policy • Unfunded mandates

42. Summary • Improving education is a lot like building a better computer: • Quality components: Demand the highest quality components from our suppliers; help them improve • Customer driven design: Adjust the design of the product to fit what people want to buy (a successful career) • Quality volume manufacturing: Improve the manufacturing process by copying best practices and making sure we have clear manufacturing specs (goals) and that the products meet the spec (testing) at high volume (scalability) • Feedback: Take a look at the product after it’s been in use for a few years to ensure quality and provide ideas for improvement

43. Summary • Nothing in this talk is new • It’s just that we don’t do it (very well)

44. Leading indicators of success • A few clear, measurable goals established and used • People skills being required to graduate • Changes are prototyped and proven to work before scaling (no more NCLB) • Adoption of national standards (K-12) • Importing best practices from other countries • Federal incentives to adopt strategies that are proven to work • New faculty are required to learn “how to teach” • Changes take a year instead of a century • Changes driven by customer satisfaction metric(s)

45. Latest copy of this talk On my website at: www.skirsch.com