1 / 31

Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at

Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at New Jersey Institute of Technology. Carol Siri Johnson Associate Professor and Chair. Outline of Presentation. Personal Introduction New Jersey Institute of Technology ABET

gus
Download Presentation

Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at New Jersey Institute of Technology Carol Siri Johnson Associate Professor and Chair

  2. Outline of Presentation • Personal Introduction • New Jersey Institute of Technology • ABET • Writing / Technical Writing Assessment • Using Online Portfolios • Using Analytic Scoring • Examples of Data Analysis Results

  3. Personal Introduction • Mount Holyoke College • English Ph.D., but few jobs • New – Technical Writing Field • Tenure-track Position at NJIT • Chair, Research EA and TC History

  4. My Assessment History • Knew nothing about it • Disliked standardized tests • Was unconvinced of its value • Then I saw a best-paper reading • The Power of Assessment

  5. Assessment at NJIT • Department of Humanities active in assessment for ~20 years • Previous Chair, NJCBSPT • Norbert Elliot and ETS • Course / Program Assessment • Analytic Online Portfolio Assessment

  6. On a Scale – Norbert Elliot

  7. What is Assessment? • Giving a student a grade? • End-of-semester course evaluations? • Faculty review of portfolios? • Student surveys? • The chair’s opinion?

  8. What is Assessment? • For Teachers, GRADES • For Departments, COMMON EXAMS • For Administration, COURSE EVALUATIONS • For Learning, OUTCOMES – stating and reaching curricular goals in Courses and Programs

  9. Assessment Can Be: • Program assessment • Curricular assessment • Method of automatic updating • Method of teaching collaboration • Gathering statistics for multiple purposes.

  10. ABET 2000- Accreditation Board for Engineering and Technology

  11. ABET’s 8 General Criteria:

  12. #4 Continuous Improvement “The results of these evaluations must be systematically utilized as input for the continuous improvement of the program.”

  13. Outcomes Assessment Cycle

  14. NJIT Core Curriculum – GURs(General University Requirements) • English Composition – 6 credits • Cultural History – 6 credits • Humanities Electives – 6 credits • Senior Seminar – 3 credits

  15. GURs Outcomes Assessment • English Composition – every semester • Cultural History – every other spring • Humanities Electives – every other fall • Senior Seminar – every semester

  16. Assessment Basics – Validity • The issue of “validity” means we are measuring what we want to measure; the test is appropriate to the subject • The problem with writing assessment has been using multiple choice tests only – they alone are not valid as a measure of writing • Multiple measures are often the best

  17. Assessment Basics – “Associative” Validity Correlating Multiple Inputs: • Machine-scored essay scores • Portfolio scores • Grade in course • Student GPA

  18. Assessment Basics – Reliability Can we be sure that the measurement, if taken again, would yield the same results? • Inter-reader Reliability (scored by two separate readers, discrepancies resolved by a third) • Content Reliability (portfolio contains work from entire semester)

  19. Two Important Steps: • Decide on and define goals or criteria • Develop method to assess outcomes (goals/criteria)

  20. Large Scale Writing Assessment Initiatives • 1950s ETS – Holistic Scoring • 1970s California (Edward White) • 1980s New York (CUNY) • 1990s New Jersey (NJCBSPT) • 2000s NJIT – Program Assessment

  21. Example: Technical CommunicationAnalytic Online Portfolio Assessment • Established Criteria by “Online Delphi” – email exchanges by faculty • 2004 – Failed effort (network down) • 2005-2009 – Successful Analytic Online Portfolio Assessment, every semester • 2009-Present – once per year

  22. Technical Communication Cycle

  23. Using Online Portfolios

  24. Procedure Have Students Make Online Portfolios with Examples of the Main Assignments Collect URLs Select Random Sampling (~60-100) All Instructors Meet ~1 Hour Calibration with Sample Portfolios where instructors discuss scoring processes ~3 Hours Scoring Assessment Director Conducts Necessary Adjudications Enter Scores into Excel or SPSS Dataset

  25. Old Humanities Rubric

  26. Old Score Sheets

  27. Revised Score Sheet

  28. Sample Data Collection

  29. A Community of Assessment • Shared Instructional Content • Self-Evaluation and Norming • Curricular Goals Evaluation • Continual Improvement (ABET) • Interesting Statistics

More Related