The Second Time Around:

1. The Second Time Around: Responding to NCATEStandards 2 and 4 Thursday September 18, 2008

2. ACCREDITATION ACTION REPORT National Council for Accreditation of Teacher Education The University of North Carolina at Asheville Asheville, North Carolina March 2006 ACCREDITATION DECISION:Continuing accreditation with conditions is granted at the initial teacher preparation level. The decision continues accreditation at the initial teacher preparation level for a two-year period. A visit focused on Standards 2 and 4 must take place no later than spring 2008 to continue accreditation beyond this period.

3. AREAS FOR IMPROVEMENT: Standard 2. Assessment System and Unit Evaluation The unit has not developed a comprehensive assessment system. The unit has not established procedures to ensure credibility of assessments, fairness, consistency, and avoidance of bias. The unit does not regularly analyze assessment and evaluation information on the unit’s operations. The unit does not maintain a record of formal complaints or document the resolution of complaints. The unit provided little evidence of its efforts to utilize data for the improvement of unit operations.

4. AREAS FOR IMPROVEMENT: Standard 4. Diversity Candidates do not have opportunities to interact and work with diverse peers. Candidates do not have opportunities to interact with diverse faculty. The unit does not ensure that all candidates interact with diverse P-12 students.

5. Standard 2. • Assessment System and Unit Evaluation

6. Standard 2:A Data Acquisition/Analysis Point of View Status at the initial visit (2005): • Paper (Primary) • Spreadsheet (Secondary) • Large volume of physical “data” • Reactive • No discernable process • Anecdotally, knew we were successful • Why were we in this shape? Historically, this approach has served us well…

7. Standard 2:A Data Acquisition/Analysis Point of View Status at the focused visit (2008): • A recursive unit-wide formal process for assessment • Data driven decisions (See “Decision-Making Flow Chart”) • Isolation of primary key indicators for data acquisition and analysis (A.K.A. Decision points) • Database (Primary) • Spreadsheet (Secondary) • Reactive and proactive approach to indicators (data) • Still a large volume of “data” but more focused, relevant, reliable, manageable, and captured closer to the source • Numerical measures of learning outcomes • One-stop-shop for data analysis and comprehensive website for the visiting assessment team (information presentation was improved) • Departmental culture of data and assessment • Assessing the assessment • Assessing each instrument

8. Standard 2:A Data Acquisition/Analysis Point of View What is next (2012): • Continuation and refinement of processes • Database (Primary) (Web accessible, more integrated, greater range of data) • Refined reactive and proactive approaches to indicators and other data analysis • Larger volumes of “data” but even more focused, relevant, reliable, manageable, and captured closer to the source • Fully developed comprehensive website for the visiting assessment team • Campus-wide numerical measures of learning outcomes • Campus-wide culture of data and assessment

9. Decision Points (2008)

10. Decision-Making Flow Chart (2008)

11. Candidate Data Flow (2008)

12. Unit Data Flow (2008) Additional Information Budget Candidate GPA Evaluation of Assessment System Advising lists Faculty workloads Faculty ratings of teaching Demographics of Unit Etc…

13. Where We Are Heading (2012)

14. Evaluation of the Assessment System for Fairness, Accuracy and Consistency of Assessment Data • “Area for Improvement: • The unit has not established procedures to ensure credibility of assessments, fairness, consistency, and avoidance of bias.”

15. Evaluation of Candidate Assessment Instruments • ILT Employer Survey and Teacher Self-Survey • Student Teaching Exit Criteria • Candidate Dispositions Form • Capstone Unit Plan Rubric • Capstone Field Experience Evaluation

16. Evaluation - General Guidelines • Content Validity is judged by fit with the Conceptual Framework and authentic teaching contexts. • Fairness is established through the analysis of the instruments for unbiased items and statistical analysis of results for various subgroups of candidates, e.g. gender, program, etc. • Consistency in the application of the assessment instruments is achieved through training workshops for all users.

17. Evaluation - Steps we followed • Assessment Review Team • Training sessions for users of each instrument to ensure consistency • UNCA’s Institutional Research (IR) Office • Annual review and evaluation of the Unit Assessment System

18. Evaluation - Candidate Assessment Instruments:ILT Employer Survey • Revision of old ILT survey form to conform to the Conceptual Framework for content validity. • Review of the procedures for conducting the surveys and possible bias due to rates of return. • IR to conduct ILT surveys ensuring objectivity and adequate return rates. • Statistical reliability analysis of ILT Employer Survey conducted by IR.

19. Evaluation - Internal Reliability of ILT Employer Survey Return Rate: 87.5 %

20. Evaluation - Pearson Product-Moment Correlation of ILT Survey with Student Teaching Exit Criteria

21. Evaluation - Student Teaching Exit Criteria • Discussion of content validity and reliability of EXIT Criteria by Education faculty. • Revision of form to TPAI in use by public schools; validity established. • Modification of TPAI form to align with the conceptual framework. • Training session for Ed faculty for consistent use of form. • Review of form by clinical faculty and approved for validity. • Acceptance of TPAI and training in its use. • Application of TPAI as formative and exit criteria for student teachers. • Review of use of Exit Criteria and acceptance for continued use. • Plan for statistical studies of subgroup performance.

22. Standard 4. • Diversity

23. NC is Flat: Learning Communities of Diverse Peers “Areas for Improvement: • Candidates do not have opportunities to interact and work with diverse peers. • Candidates do not have opportunities to interact with diverse faculty. • The unit does not ensure that all candidates interact with diverse P-12 students.”

24. Part I: Candidates Interacting with Candidates – Spring 2007 • Contact between UNCA and WSSU • BLOG Discussions (http://nc-education-issues.blogspot.com/) • What have you seen that helps to balance cultures or inhibits the bridging of cultures in classrooms? • What strategies have you experienced in your own teacher education program that have raised your consciousness regarding the role of teacher as change agent • Candidates met in groups at each institution and their responses were added to the BLOG • Assessing learning and planning next steps

25. Part II: Video Conference Seminars on Cultural Competence – Spring 2007 • Small group discussion sessions • Videoconference exchanges • Threaded discussions • Assessing learning and next steps

26. Part III: Conference on Cultural Competency Combining videoconferencing with on site conference Providing common experiences Assessing Learning and next steps

27. Result “Continued accreditation”