Electronic Portfolios and the Authentic Assessment of Economics Majors

1. Electronic Portfolios and theAuthentic Assessment of Economics Majors Steven C. Myers The University of Akron Myers@uakron.edu http://GoZips.UAkron.edu/~myers Presented to the Robert Morris Conference on Teaching Economics: Instruction and Classroom Based Research February 18, 2005

2. Electronic Portfolios and theAuthentic Assessment of Economics Majors By the end of this presentation you will • Be able to define an electronic portfolio • define and explain authentic assessment • Compare and contrast authentic and traditional assessment • Know how to set up an authentic assessment • Know that portfolios are authentic • Learn four questions to guide your assessment stategy • Understand how the Economics department at The University of Akron is beginning its assessment strategy • See an example of a student portfolio Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

3. An Electronic Portfolio is defined as • A collection of authentic and diverse evidence, • Drawn from a larger archive representing what a person or organization has learned over time • On which the person or organization has reflected, and • Designed for presentation to one or more audiences for a particular rhetorical purpose Definition of the National Learning Infrastructure Initiative (NLII) cited by Barbara L. Cambridge, American Association for Higher Education (AAHE), “Economic Portfolios: Why Now?” on Educause Live, Feb 11, 2004. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

4. What is Authentic Assessment? Reference: Authentic Assessment Toolbox http://jonathan.mueller.faculty.noctrl.edu/toolbox Jon Mueller Professor of Psychology North Central College Naperville, IL Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

5. What is Authentic Assessment? A form of assessment using • real world tasks • application of essential knowledge and skills • problems likely to be found after formal education is completed • demonstrations of mastery of competencies Borrowed heavily from Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

6. What is Traditional Assessment? “the curriculum drives assessment” • The body of knowledge is determined first. • That knowledge becomes the curriculum that is delivered. • The assessments are developed and administered to determine if acquisition of the curriculum occurred. Borrowed heavily from Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

7. What is Authentic Assessment? “the assessment drives the curriculum” • First determine the tasks that students will need to perform to demonstrate their mastery • Then develop the curriculum that will allow that performance • And designed so that the acquisition of essential knowledge and skills must occur. Borrowed heavily from Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

8. Selecting a response Contrived Recall / Recognition Teacher-structured Indirect evidence Performing a task Real-life Construction / Application Student-structured Direct Evidence Attributes of Traditional Assessment vs Authentic Assessment Quoted from Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

9. Why Use?Authentic Assessments … • Are direct measures • Capture the constructive nature of learning • Integrate teaching, learning and assessment • Provide multiple paths to demonstration Borrowed heavily from Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

10. How do you create Authentic Assessments? • Set standards or proficiencies • Design authentic tasks • Look for achievement of criteria • Discriminate performance based on a rubric --Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

11. How do you create Authentic Assessments? 4 questions! • Set standards or proficiencies“What should students be able to do?” • Design authentic tasks“What indicates students have met these standards?” • Look for achievement ofcriteria“What does good performance on this task look like?” • Discriminate performance based on arubric“How well did the students perform?” --Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

12. Are Portfolios Authentic? • Considers process and development of skills • Not just test scores • Focuses on meaningful collection of student performance • Includes meaningful student reflection • Includes evaluation of that work. --Jon Mueller Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

13. Walstad suggests student portfolios in assessment of the economic major… • “Assessing the economics major is a more difficult challenge than assessing student performance in a particular economics course. • “Assessment using student portfolios offer a solution for evaluating whether a student has the attributes or proficiencies of an economic major. • “Those students with difficulties would be given time to prepare new items or revise previous work. The process of portfolio assessment … merits further exploration.” William B. Walstad, “Improving Assessment in University Economics,” Journal of Economic Education, Summer 2001. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

14. Bill McKeachie suggests… • portfolios “…demonstrate the progress that has been made, … the student’s best work, or better yet, the student’s descriptions of how the work helped his or her development” (p. 92). • He often finds evidence of learning he would have otherwise missed, but warns that “unless you provide instruction on how to construct a portfolio and describe the criteria you will use in evaluating it, assessing reliably will be difficult” (p. 94). Bill McKeachie. McKeachie's Teaching Tips: Strategies, Research, and Theory for College and University Teachers. 11th ed. Boston: Houghton Mifflin, 2002. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

15. OK, with that motivation, lets address Assessment of the Economic Major… • What learning outcomes, standards or proficiencies do we care about? • How aligned are our processes to these outcomes? • How committed are we to continuous improvement? • How can we integrate undergraduate learning across the various major and allied courses ? Zippy Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

16. (1) What Learning Outcomes Do We Care About? The Hansen proficiencies… • “Completing the major, even with … grades that may suggest … mastery, does not mean that graduates are effectively equipped to use their knowledge and skills, … nor be good value to prospective employers, society and themselves.” • “… the economic profession gives too little attention to what we expect majors to do with their knowledge after they graduate.” W. Lee Hansen. “Expected Proficiencies for Undergraduate Economics Majors,’ Journal of Economic Education, Summer 2001 Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

17. (1) What Learning Outcomes Do We Care About? The Hansen proficiencies… • “the proficiencies approach focuses on what graduating majors can do with the knowledge and skills they acquire in the major.” • “The question we face is how to best move towards a proficiencies approach to economics teaching and learning.” W. Lee Hansen. “Expected Proficiencies for Undergraduate Economics Majors,’ Journal of Economic Education, Summer 2001 Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

18. (1) What Learning Outcomes Do We Care About? The Hansen proficiencies… … via the undergraduate curriculum: • Access to existing knowledge. • Display command of existing knowledge. • Interpret existing knowledge. • Interpret and manipulate economic data. • Apply existing knowledge. … via a Senior Project • Create new knowledge. W. Lee Hansen. “Expected Proficiencies for Undergraduate Economics Majors,’ Journal of Economic Education, Summer 2001 Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

19. (1) What Learning Outcomes Do We Care About? (an example for comparison) Six Principles of Undergraduate Learning • Core communication and quantitative skills • Critical thinking skills • Intellectual depth, breadth, and adaptation • Integration and application of knowledge • Understanding society and culture • Values and ethics ePort project of IUPUI Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

20. Review the curriculum. What kind of students are we producing? How do we currently meet the Learning Outcomes in our present courses? How can we insure that the current courses produce the desired learning outcomes? What’s missing at UAkron? A Computer Skills for Economic Analysis course. Doing economics Scaffolding computer skills Creating an electronic portfolio A Capstone Course Senior project that creates new knowledge. Perfecting the electronic portfolio (2) How aligned are our processes to the Learning Outcomes? Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

21. 2000 Curriculum revision Capstone Course Electronic Portfolio Course Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

22. (2) How aligned…Why a course to scaffold computer skills? • Students can not adequately function in class or in the job market without good computer skills. • Students have different initial endowments of computer skills. • Too much class time wasted in upper division classes on computer skill scaffolding. • Professors will not require or expect a skill if they do not have the time to provide the scaffolding needed. • Professors may try and students get frustrated and the professor gets disappointed. • Too much opportunity for learning is missed without the requisite skill being assumed. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

23. (2) How aligned…Why electronic portfolios? • Students learn to build and update a website. • Students learn the value of appearance and presentation of their work. • All submissions to the portfolio is instantly available to all faculty in the department including their advisor. • Students may choose to place their portfolio on the world wide web (but not required). • Students may represent themselves to potential employers by offering a web address or providing a CD. • The department requires the portfolio for graduation. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

24. (3) How committed are we to continuous improvement? • The faculty needs to know whether students are achieving the Hansen proficiencies. • At least one artifact per proficiency required • A written reflection on the artifact is included • An assessment of the artifact and how it meets the Learning Outcome. • Students demonstrate growth by the progression of artifacts and reflections included in the portfolio. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

25. (3) How committed are we to continuous improvement? • The portfolio allows for both formative assessment and summative assessment of competencies learned. • Begins as a course portfolio for assessment by the instructor. • The portfolio allows for formative and summative assessment of the major through graduation. • Becomes a comprehensive portfolio for assessment by the advisors and other faculty. • The portfolio allows for summative assessment of the curriculum in the department of economics. • Emerges as a professional portfolio to show employers. • Becomes an artifact in the department’s portfolio of portfolios for assessment by the faculty. • Provides authentic assessment and evidence for accreditation visits. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

26. (3) How committed are we to continuous improvement? PortfolioEvaluationForm Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

27. (3) How committed are we to continuous improvement? Comprehensive Assessment (formative and summative) includes: • CATs (Classroom Assessment Techniques) • During class, often applied • Graded Assignments (summative and mastery) • SETs or Assessment of Learning • Midterm assessment • End of course (college requirement) • Electronic Portfolios • Once a semester • Exit Interview and Alumni Surveys • At graduation and periodic intervals Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

28. Not on the web On the web Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

29. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

30. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

31. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

32. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

33. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

34. How can we integrate undergraduate learning across the various major and allied courses? Electronic Portfolios • Heightens students’ perspective that courses are indeed related across the curriculum. • Are available to all faculty who can look up students past work in relevant courses. • Other students can learn from prior students’ experiences. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

35. How can we integrate undergraduate learning across the various major and allied courses? The Senior Project • Culminates a theory track and a quantitative track of study. • Integrates by definition much of what has been experienced before. • Is the opportunity for the student to “create new knowledge” (Hansen’s 6th) • Becomes linked in the portfolio. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers

36. Reviews of Electronic Portfoliosfrom Syllabus, NLII, Educause, AAHE, … • Batson, Trent. “The Electronic Portfolio Boom: What’s It All About?” Syllabus, December 2002. • NLII. “Electronic Portfolios -NLII 2002-2003 Key Theme, National Learning Infrastructure Initiative” (website) at http://www.educause.edu/nlii/keythemes/eportfolios.asp, cited March 2004. • Barrett, Helen. “Using Technology to Support Alternative Assessment and Electronic Portfolios,” (website) http://www.helenbarrett.com/portfolios.html, cited March 2004. • Cambridge, Barbara L. Electronic Portfolios: Emerging Practices for Students, Faculty and Institutions, American Association for Higher Education (AAHE), 2001, 204 pp. (http://webcenter1.aahe.org/electronicportfolios) • Ittelson, John C. “Building an E-dentity for Each Student,” Educause Quarterly, Number 4, 2001. Electronic Portfolios and AssessmentGoZips.uakron.edu/~myers