340 likes | 426 Views
USING Tk20 SOFTWARE TO OPTIMIZE OUTCOMES ASSESSMENT: A CASE STUDY. John W. Rogers Ph. D. VP Institutional Effectiveness American International College. Founded in 1885 Small Private, Non Profit College 3500 students: half UG/half graduate
E N D
USING Tk20 SOFTWARE TO OPTIMIZE OUTCOMES ASSESSMENT: A CASE STUDY John W. Rogers Ph. D. VP Institutional Effectiveness American International College
Founded in 1885 • Small Private, Non Profit College • 3500 students: half UG/half graduate • 40 UG programs and 15 graduate programs: include Nursing, Graduate Business, and Masters programs in a variety of areas • Masters and Doctorates in Education are offered in US and international satellite locations
NEASC 10 Year Review January, 2010: Monitor institution’s progress: • Improvement in financial health; • “establishing a campus-wide assessment program focused on enhancing institutional effectiveness, especially with regard to assessment of student learning outcomes;” • Faculty development and academic program improvement; • Student information literacy
The Challenge • Centralized data collection vs. decentralized data management • Each program needs to take ownership of its outcomes assessment plan • Campus wide responsibility for aggregating data • Continuous improvement and evidence based resource allocation • Accrediting bodies need to see a coherent story of institutional quality and performance management
Criteria for Software Selection • Hosting in-house • Cash flow advantage • Flexibility in application • Portfolio durability • Alumni relations • Usability: common user interface • Survey function is fully integrated with system and is confidential • In-house hosting process: we build server, we design the system, integrate with SIS, vendor sets it up and provides maintenance
Implementation Planning • Train the trainer model • Select early adapters • Communicate to the entire community: Assessment Day • Involve non-academic departments: • Administration – Finance, Business Office, Financial Aid, Admissions • Student Activities – Residence Life, Athletics, Security, Career Services
The Assessment Process • The Mission of the College • Learning Objectives and Broad based Goals • Define program responsibilities and a hierarchy of authority • Program Learning Objectives related to artifacts for measurable results • Map artifacts to the program curriculum • Gather data and aggregate it for continuous improvement
Tk20 Applications • Manage assessment plans: learning objectives, artifacts, curriculum map, and streams of evidence • Electronic course evaluations • Practica and internships • Portfolios for assessment and career planning • Faculty artifacts and development • Student advising • Surveys of faculty, students, and staff • Assess “non-academic” programming • Strategic Plan Dashboards
Level of Mastery Categories
Electronic Course Evaluations • Replace paper and pencil/Scantron system • Give real time feedback on instructor performance • Timely aggregation and reporting to deans and program directors
E-Portfolio • Critical aspects of assessment software • Already required in physical or limited electronic form by program specific accreditors • Electronic Portfolio system becomes easy and comprehensive method of assessment; assessment is “baked in”
Analogy of the Map The curriculum map allows an institution, college, or program to demonstrate: • where you are going (goals/outcomes to be met) • how you are getting there (skills taught and assessments used).
Where Are You Going? Huba and Freed. Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Needham Heights: Allyn & Bacon, 2000.
What are transition points? Key points in a program when a unit assesses candidate knowledge, skills, and professional dispositions to determine if candidates are ready to proceed to the next stage in a program.
Should be defined per program Minimum of 4 Stages Defining Transition Points
Reports • The following types of reports are available based on data collection has begun: • Success Rate at Each Transition Point • Distribution of Students Meeting each Transition Point Requirement • Percentage of Students Meeting each Transition Point Requirement
Distribution of Students in Early Childhood Education Program by Transition Points
Standard Mapping • Align course work to standards for: • State Boards of Education • Professional accrediting bodies: Business, Nursing, PT, OT, CJ, Education • Regional accreditors • Relate competencies developed in curriculum to specific standards for accreditation
Reports NAEYC Initial Licensure Standards in Early Childhood Education
Field Experience • Collect demographic data on sites where students are placed • Students use electronic templates to submit field experience work • Mentors assess student work in relation to agreed criteria; and comment on student preparedness • Aggregate data on: • % of students who complete practica • Effectiveness of sites in achieving program objectives
Faculty Artifacts • Demonstrate that faculty have credentials and levels of engagement required to fulfill program objectives • Reports provide evidence of faculty qualification, research, teaching effectiveness, and institutional contributions • Treat faculty as the essential human capital of the institution
Strategic Plan Dashboards • Align Broad based Goals and Strategic Initiatives with: • Action Plans • Budgets • Assessment Measures • Dashboard for tracking and monitoring • Include “non academic” departments
Benefits to the Institution • Track student progress • E-Portfolio aid students and advisors in developing resume, graduate school applications • Identify trends in programs • Identify areas of improvement • Generate reports to fulfill accreditation needs • Communicate with senior management and Board on Performance Management – “close the loop” with Strategic Plan
Questions/Comments Thanks for attending the session John W. Rogers Ph. D. john.rogers@aic.edu