Assessing Students Ability to Communicate Effectively—

1. College of Technology and Computer Science Abstract: Based upon East Carolina University's decision to initiate a Quality Enhancement Plan (QEP) related to the writing skills of undergraduate students, it is apparent that communication is a critical professional skill for today's graduates.  In the College of Technology and Computer Science, all of our departments assess their students' communication skills.  This poster highlights some of the ways in which that is done, and provides ties to the ongoing QEP work and its affect on some aspects of the college’s curricula. Assessment Case 1 Desired OutcomeThe goal is to prepare technology students to compete and succeed in a global economy; students who can assess global issues and events from multiple perspectives and apply critical thinking skills to address global challenges. A two prong approach was employed within sections of ITEC 3290 Technical Writing. Measures Assessment Case 2 Background ABET specifies a set of learning outcomes that every accredited engineering program must assess. One of these outcomes is that the graduates of the engineering program will demonstrate an ability to communicate effectively. This learning outcome is assessed directly using work samples from selected courses and indirectly using the results of student surveys. Work samples may include homework or laboratory assignments, targeted exam questions, or project reports or presentations. Student surveys include senior surveys and end of course surveys in which students assess their understanding of course objectives. Process The assessment of each of the student learning outcomes measured by the engineering program has a lead faculty member, known as the outcome coordinator. It is the responsibility of the outcome coordinator to evaluate assessment data for his/her assigned outcome annually. The learning outcome related to students’ ability to communicate effectively is assessed via direct and indirect measures in a number of courses. Results At the end of each academic year, each outcome is given an overall achievement rating by its outcome coordinator. The overall rating is determined by examining all related student work samples and survey questions. A sample outcome report from 2009-10 is provided below. Faculty meet to discuss the outcome reports, with particular attention focused on outcomes whose achievement rating is below 3. • Development across Tasks Evaluation • How well did technology students communicate clearly with multiple audiences, including those from other cultures? • Did each student meet the minimum criteria for a passing grade across all assignments? • Task Specific Evaluation • How well did technology students adopt the strategies and suggestions for communicating across cultures? • Did each student meet the minimum criteria for a passing grade? • Task Specific –Instructional Memo to an English as a Second Language Audience • Reinforces communication across cultures. • Integrates graphics for multi-cultural audiences. • Development Across Tasks –Multiple Successive Assignments • Emphasizes recognition of cultural variances impacting readers’ perceptions. • Focuses on language choices and variant meanings in graphics due to color and symbol usage. • A review loop used for student growth Results Data were gathered during the 2010-11 and 2011-12 academic years. Students were identified by major and the scores for an assessment rubric related to the instructional memo described above are reported. Assessment results are summarized in the table below. Assessing Students Ability to Communicate Effectively— Findings from the College of Technology & Computer Science • Outcome: Graduates of the engineering program will demonstrate an ability to communicate effectively. • OVERALL RATING OF OUTCOME ACHIEVEMENT: 4 • 5 = Well above expectations: All measures exceed target levels Target levels for Assessment and Evaluation Plan: • 4 = Above expectations: Most measures at or above target levels Capstone Assessment: 70% of samples at 3 (satisfactory) or 4 (superior) level • 3 = Outcome achieved: Most measures at or near target levels Student Work Samples: 70% of samples at 3 (satisfactory) or 4 (superior) level • 2 = Outcome not achieved: Most measures below target levels Targeted Exam Questions: 60% correct for each question • 1 = Well below expectations: All measures below target levels Student Surveys: 70% 4 or 5 responses for each related course outcome • Work Samples: Student Surveys: • ENGR 2050 = 73% rated 3 or 4 ENGR 1012 = 90% rated 4 or 5 ENGR 4010 = 85% rated 4 or 5 • ENGR 3024 = 59% rated 3 or 4 ENGR 1016 = 98% rated 4 or 5 ENGR 4020 = 100% rated 4 or 5 • ENGR 3400 = 71% rated 3 or 4 ENGR 2050 = 96% rated 4 or 5 • ENGR 4010 = 100% rated 3 or 4 ENGR 3024 = 94% rated 4 or 5 • ENGR 4020 = 71% rated 3 or 4 ENGR 3400 = 73% rated 4 or 5 • Comments: • Capstone Assessment: For both courses, the target level is met. • Work Samples: For all but one course (ENGR 3024) student work samples met the target level. In that course, students struggled to correctly document their calculations. • Targeted Exam Questions: Not applicable for this student learning outcome. • Student Surveys: For all courses, student surveys met the target level. • Recommendations: • When possible, it may be good to require documentation of calculations for students in senior level courses. For example, students who take ISYS 3010 work with decision trees and there are numerous monetary calculations required in support of a final decision. Requiring students to properly document such calculations and assessing their ability to do so may allow us to have a better idea of how well our seniors can correctly document calculations.. • Discussion • The current data indicates that, for the most part, students are meeting the outcomes assessed via the instructional memo assignment in ITEC 3290, Technical Writing. Though the results from IET are lower than other majors, it should be noted that the sample size for IET is quite small (fewer than 7 students each year). Additionally, continued assessment among program coordinators will lead to review and analysis of additional years of results in an effort to insure outcomes are being met by students in all majors. Lessons on intercultural communication continue to be updated and stressed throughout the delivery of this course. • Conclusions • Accessing student achievement of the desired outcome using a writing intensive course serving fivemajors has provided multiple benefits: • Inter-major interaction and discussions provided expanded opportunities to appreciate the need for preparedness in a global economy. • Comparative analysis across majors aids in identifying “best practices” that can be utilized within the degree programs. Impact of the Quality Enhancement Plan The Quality Enhancement Plan (QEP) at East Carolina University, with a slogan of “Write Where You Belong”, focuses on the writing skills of undergraduate students. The work of the QEP and resulting modifications to how writing is taught at East Carolina University will better enable students to see connections in their writing and link what they learn in their composition courses to writing in their majors. The second writing course taught by the English department will serve as a bridge between ENG 1100 and discipline-specific writing required in upper-level course in the major. We anticipate the effect of this second writing course will be improvement in the written communication skills of our college’s students.