ePortfolio Scaffolding Study

1. ePortfolio Scaffolding Study Sandra A. Lathem, Joyce L. Morris, Jing Qi The University of Vermont Kirk Vandersall Arroyo Research Services SITE 2007 San Antonio, Texas March 27, 2007

2. Electronic Portfolio Connection • PT3 grant awarded 2003 • University of Vermont • Lyndon, Johnson & Castleton State • Champlain College • IBM Reinventing Education • Apple Computer • Vermont Dept of Education • Arroyo ResearchServices • www.uvm.edu/pt3

3. UVM Scaffolding Study • Conducted spring semester 2006 • Examines three programs • Elementary education • Middle level education • Physical education • Relationship between instructional scaffolding and … • portfolio development • technology proficiency • artifact and reflection quality

4. Research Question 1 What is the evidence that instructional scaffolding in portfolio development and technology skills occurs at the micro (faculty member/student) and macro (department /program) level?

5. Research Question 2 What is the relationship between the instructional scaffolding that pre-service teachers receive and the quality of artifacts and reflections found in the licensure portfolio?

6. Research Question 3 • What are UVM student attitudes about the licensure portfolio process in their programs?

7. Research Question 4 Will the quality in artifact development and reflection differ between paper and electronic portfolios?

8. Licensure in Vermont • Governed by Vermont Standards Board for Professional Educators (VSBPE) • VSBPE oversees accreditation process through ROPA (Results Oriented Program Approval)

9. Five Standards for Teachers: A Vision for Schooling • Learning • (Expertise in Endorsement Area) • Professional Knowledge • (Methodology and Pedagogy) • Colleagueship • Advocacy • Accountability

10. ROPA Scoring Rubric • Common statewide assessment for all teacher education programs • Six entries • Teaching Episodes (two episodes) • Understanding Student Learning • Accommodating Students Identified • Teaching Over Time • Colleagueship and Advocacy • Self-reflection and Vision

11. Scaffolding • Intentional planning of curricula to organize tasks and activities to build understanding • Two levels -- • Micro (teacher/student) • Macro (program or organization) • Consistent with contemporary theory (Bransford) on How People Learn • Prior knowledge • Metacognition

12. PT3 Professional Development • Portfolio support to faculty and students • Creation of html templates structured to meet ROPA standards

13. Methodology • Exploratory • Mixed Methods • Student Surveys (n=100) • Faculty Interviews (n=10) • Faculty & Student Observations (n=3) • Student Focus Groups (n=3; students= 38) • Course Syllabi (n=34) • Licensure Portfolio Review (n=35)

14. Findings-Question 1What is the evidence that instructional scaffolding in portfolio development and technology skills occurs at the micro (faculty member/student) and macro (department/program) levels? • Micro Level • Many experiences in artifact development • Assignments geared toward licensure portfolio artifacts • Reliance on student teaching for artifact development • Technology integration less evident • Macro Level • Middle level program most intentional in portfolio construction for undergraduates • Courses identified for portfolio artifact development • Three undergraduate portfolios developed • Elementary education program intentional artifact development in specific courses • Artifact development driven by the ROPA scoring rubric

15. Findings- Question 2What is the relationship between the instructional scaffolding that pre-service teachers receive and the quality and reflections found in the licensure portfolio? • 35 licensure portfolios scored by Arroyo Research Services • Elementary program • 12 paper • 15 electronic (using template) • Middle level • 8 electronic (using template) • Physical education • Not scored - unable to collect • Small sample size is a limitation to findings.

16. Question 2 - Continued • Arroyo Research Scoring Rubric (not ROPA) • Artifact Selection • Reflection • Collaboration • Assessment • Technology • Scale • 1 = no evidence • 2 = limited or negative evidence • 3 = some evidence (but needs development) • 4 = ample evidence

17. Findings 2 - continued • Weak scores in reflection, collaboration, assessment. • Statistically significant differences found between elementary & middle level programs in several areas. Further research needed. • Evidence of technology-supported teaching strategies was very low (mean 1.3143, std. .6761)

18. Findings - Question 3What are UVM student attitudes about the licensure portfolio process in their programs? Do they feel adequately prepared by their course experiences? • Student surveys • 100 students (79 female, 19 male, 2 missing) • Overall, a 50-50 split in agreement about purpose of portfolio process -- concerning introduction to portfolios process, its value as a tool to demonstrate student learning, usefulness of collaboration, reflection, & feedback. • Focus Group Sessions • Portfolios extremely stressful exercise • Like “taking a test” • Lack of technology skills very stressful for some students, especially middle level graduate students

19. Middle Level Focus Group

20. Findings - Question 4Will the quality in artifact development and reflection differ between paper and electronic portfolios? • Licensure portfolios (35 scored) • 27 elementary education • 12 = paper • 15 = electronic using html template • 8 middle level • 8 = electronic using html template • Results • Paper portfolios receiving scores of 3 or 4 predominate in every scored criterion, even evidence concerning technology supported teaching strategies. • Statistically significant difference found. • Small sample size is a major limitation for these findings

21. Conclusion - 1 • Scaffolding at both macro & micro levels is occurring. • Middle level - most developmental approach at undergraduate level; but post baccalaureate students don’t benefit from this approach. • ROPA scoring rubric drives artifact development • Students are ambivalent about the process; many view it as a “test” rather than a portrayal of professional growth. Portfolio construction is a stressful process.

22. Conclusion - 2 • Reflection is weak - lack of time to develop reflections, given heavy reliance on student teaching experience. • Technology proficiencies needed to produce electronic portfolios using html editors may be hindering quality & integrity of portfolios. • Quality of paper portfolios reviewed were overall higher than those produced with a template and html editor. • Continued research is needed.

23. www.uvm.edu/pt3 This paper, student survey, ROPA rubric, portfolio scoring rubric, and The Five Standards documents are available on our web site Click Research Click Papers and Presentations

24. Questions? For additional information, please contact the authors - Joyce Morris, Sandy Lathem, Jing Qi University of Vermont 802-656-4140 http://www.uvm.edu/pt3 Email: jmorris@uvm.edu, slathem@uvm.edu, jqi@uvm.edu Kirk Vandersall, Arroyo Research Services Email: kirk@arroyoresearchservices.com http://www.arroyoresearchservices.com 213-291-1556