1 / 29

Evaluation at the Defense Language Institute Dr. Thomas S. Parry

Evaluation at the Defense Language Institute Dr. Thomas S. Parry Directorate of Evaluation and Standardization Defense Language Institute Bureau for International Language Coordination Budapest, Hungary May, 2006. Why Evaluation?. Evaluation can make you rethink what you have been doing!.

selah
Download Presentation

Evaluation at the Defense Language Institute Dr. Thomas S. Parry

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation at the Defense Language Institute Dr. Thomas S. Parry Directorate of Evaluation and Standardization Defense Language Institute Bureau for International Language Coordination Budapest, Hungary May, 2006

  2. Why Evaluation? Evaluation can make you rethink what you have been doing!

  3. What is Evaluation? The process of finding the value or amount of something—to determine it’s significance or quality. -- Webster’s New World Dictionary

  4. DLI Facts • Average daily resident attendance: 3,079 • Languages taught: 22 • Number of students taught annually through all Continuing Education programs: 5,000 • Highest enrollment languages: • Arabic • Chinese • Korean • Persian-Farsi • Russian • Spanish • Serbian-Croatian

  5. Typical DLI Student Day • 0515: Physical Training • 0630: Breakfast • 0730: Formation with Service Unit • 0755: Class start • 1530: Class end • 1600: Military Training • Cumulative: • 6 hours of class instruction • 45 minute special assistance or enhancement training • 3 hours of homework plus weekend assignments • 2-3 hours of study hall (M-Th) • Developing & sustaining basic warrior-linguist skills, common task training, and physical readiness

  6. Student Load by Difficulty 94% of students taking CAT III & IV Languages Resident Courses at Presidio of Monterey DLIFLC Basic Course LanguageFY06 Student LoadFaculty Category IV Languages (63 weeks) Arabic 1165 272 Korean 833 216 Chinese 558 126 Japanese3510 2591 (71%) 624 Category III Languages (47 weeks) Dari 87 24 Pashto 134 34 Persian Farsi 248 68 Russian 144 58 Uzbek, Hindi, Urdu, & 6 other lang21194 824 (23%) 278 Category II Languages (34 weeks) German 278 27 (1%)8 Category I Languages 26 weeks) Spanish, French, Italian, Portuguese 19656 196 (5%) 56 Totals 3638 966

  7. DLI Evaluation Goals • Ensure student and teacher accountability • Provide for continuous academic program improvement • Faculty and staff development • Curriculum development and implementation • Gauge our return on investment • Ensure quality of life and support services are conducive to learning

  8. Teachers Evaluate Students Students are held to high levels of accountability for their learning

  9. Teachers EvaluateStudents • Teachers evaluate student performance in the classroom through: • Participation in classroom activities • Teacher-developed chapter/unit tests • Periodic counseling • Diagnostic assessments • Learning style assessment • Performance tests • Oral Proficiency Interviews (OPI) • Defense Language Proficiency Tests (DLPT) • Listening • Reading

  10. Students Evaluate Teachers Program of Study Quality of Life

  11. Evaluation at DLI is Bi-directional Teachers Evaluate Students Students Evaluate Teachers

  12. Teacher Evaluation • Teacher evaluation at the DLI includes: • Complete student evaluation of the teacher • Scheduled and unscheduled classroom observations by the supervisor • Periodic reviews of performance objectives with the supervisor • The combination of these three teacher evaluation elements provides an accurate assessment of the quality and effectiveness of foreign language instruction

  13. Teacher Evaluation • Each teacher is evaluated on classroom performance by all the students he/she teaches • A personal performance profile is created for each teacher by tracking his/her performance throughout each of their courses Teacher evaluations by students have implications for merit pay and rank advancement

  14. ISQ Interim Student Questionnaire • Administered via computer to all students enrolled in 47- and 63-week language courses 60% of the way through the program • Takes roughly 60-90 minutes to complete • Survey sections include: • Program Evaluation (PE) • Teacher Evaluation (TE) • Quality of Life (QL)

  15. ESQ End-of Course Student Questionnaire • Administered via computer at the end of language courses to all graduating students • Takes roughly 60-90 minutes to complete • Survey sections include: • Program Evaluation (PE) • Teacher Evaluation (TE) • Quality of Life (QL)

  16. Analysis and Reporting • Evaluation feedback from ISQ/ESQ delivered within two working days from survey administration • Deans • Chairs • Team leaders • Individual teachers • Special monthly reports provided to service commanders and individual units via separate questionnaires • Annual summaries for each instructor sent to Provost, Vice Chancellors and school Deans

  17. ISQ/ESQ Response Scale • Items are rated from 1-4 with an option for no opinion: 0 No opinion 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree

  18. Sample Criteria Teacher Evaluation (TE) • Class preparation and quality of instruction • Ability to teach language and related concepts effectively • Adjustment of teaching strategies • Ability to provide a variety of instructional activities

  19. Sample Criteria Teacher Evaluation (TE) • Active use of foreign language in classroom • Ability to provide challenging instruction • Counseling and assistance offered • Feedback given to students

  20. Team and Individual Instructor TE Analysis

  21. Teacher Improvement

  22. Sample Criteria Program Evaluation (PE) • Testing and grading • Team teaching • Homework • Curriculum and supporting materials • Overall evaluations

  23. Sample Criteria Quality of Life (QL) • Barracks, family housing and child care • Quality of food, mess hall and related services • Medical and dental care • Chaplain and community services • Military support • Sport and recreation opportunities • Transportation and travel office services

  24. Other Areas of Concern Quality of Life (QL) • Equal Opportunity • Prejudice • Discrimination • Sexual harassment • Military • Training • Duties • General quality of life

  25. PE and QL Analysis

  26. Serious Issues • Depression • Suicide threats • Sexual assaults and harassment • Fraternization and other misconduct Immediate turnaround; sent directly to Command Group for action

  27. Evaluation of Continuing Education Programs • ESQ questionnaires distributed world-wide to training sites and include: • Video tele-training sites • Mobile training team deployments • Language Teaching Detachments • Command Language Programs • Special, customized programs • Questionnaires completed and returned to DLI via on-site program managers • Analysis and feedback provided to instructor’s school and training site within two weeks

  28. Conclusion • DLI evaluation practices: • Are well established—honed through years of practice • Are rigorous and thorough • Bring about faculty and program improvement • Establish baselines for student/teacher accountability • Give us cause to rethink what we are doing and consider how we might do things better

  29. Evaluation at the Defense Language Institute Questions? Discussion?

More Related