1 / 45

Cluster 5 Spring 2005 Assessment Results

Cluster 5 Spring 2005 Assessment Results. Wellness Domain. Cluster 5 Course Requirement. Complete 1 course in each domain. Assessment Day Data Collection Scheme: Pre/Post.

mckenzie-petersen
Download Presentation

Cluster 5 Spring 2005 Assessment Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cluster 5 Spring 2005 Assessment Results Wellness Domain

  2. Cluster 5 Course Requirement • Complete 1 course in each domain

  3. Assessment Day Data Collection Scheme: Pre/Post Students in each cohort are tested twice on the same instrument – once as incoming freshmen (0 credit hours) and again in after they’ve completed 45-70 credit hours. COHORT 1 COHORT 2 Cohort 2: First time for Cluster 5 to have pre/post data since new objectives/instruments. COHORT 3

  4. Wellness Domain Instruments • Knowledge of Wellness and Health Test (KWH1) • Scenario-based multiple-choice test created by the faculty • Health and Wellness Questionnaire (HWQ) • Part 1: self-report survey to measure student’s own health and wellness behaviors • Part 2: 33 health and wellness statements; students identify whether they are facts or myths • Well-Being Scale (WBS) • Self-report measure of psychological well-being

  5. Results of KWH1 • First: Examine the results of the KWH1 for 644 2nd semester sophomores in Spring 2005 • Overall Results • By JMU vs. Transfer/AP • By domain “completers” vs. “non-completers” • By which course completed • Rel. between course grade and KWH1 score • Second: Examine pre/post results for the 394 students who took the KWH1 in both Fall 2003 and Spring 2005 • Overall pre/post difference • Pre/post difference by domain “completers” vs. “non-completers” • Pre/post difference by which course completed • Pre/post difference by item for “completers”

  6. Overall KWH1 Results:2nd Semester Sophomores(N=644) On average, students are answering 60% of the items correctly. Most scores are between 19 & 25.

  7. JMU (n=628) vs. Transfer/AP Students (n=16) • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU? • No. • JMU students: M = 21.16, SD = 3.30 • AP/transfer students: M = 20.38, SD = 3.31 • Difference was not statistically, t(560)=.93, p = .35, nor practically significant (d = .23).

  8. Domain “Completers” vs. “Non-Completers” “Non-Completers” N = 82 “Completers” N = 555 More courses completed, the higher the mean. Differences among the means, however, are of small practical significance (d=.06) Means for students completing two Wellness classes not reported due to small N

  9. KWH1 Means by Course Completed“Completers” Only There were minor differences among the KWH1 means (d= .20)

  10. Relationship between KWH1 and Course Grade There was a positive and moderate relationship between KWH1 Total score and course grade (r = .31) in both GKIN100 and GHTH100.

  11. Pre/Post Results (N = 394) • Spring 2005: M = 21.13, SD = 3.24 • Fall 2003: M = 19.00, SD = 3.07 • The difference between these means is of medium to large practical significance (d = .67) Average increase of 6.1% in wellness domain knowledge

  12. Pre/Post for “Non-Completers” (n = 16) vs. “Completers” (n = 355) “Non-Completers” N = 16 “Completers” N = 355 Means on the KWH1 for “completers” and “non-completers” are very similar. This indicates that gains made on the KWH1 may simply be a function of maturation, not a function of having completed one’s wellness domain requirement.

  13. Pre/Post by Wellness Course Completed N = 215 N = 135 The same gains (~ 2 pt increase) can be expected in average KWH1 scores for students fulfilling their wellness domain requirement by taking either GHTH100 or GKIN100.

  14. Pre/Post Difference in Items for “Completers” Even in Spring 2005, there were still 14 items where less than half of the students were obtaining the correct answer.

  15. Items with Large Pre/Post Gains 7% or more students answered these items correctly in Spring 2005 than in Fall 2003 There were three items (5,11, & 15) with sizable gains, meaning that ~20% or more of the students responded correctly to these items in Spring 2005 than in Fall 2003.

  16. Items with Small (or No) Pre/Post Gains Items with no change, or change in the wrong direction are shown below for items that are of medium to extreme difficulty. Items as a whole don’t seem to be related to any one goal or objective, but they do seem to be related to the recall of specific information.

  17. Conclusions KWH12nd Semester Sophomore Data • 2nd semester sophomores are scoring on average ~60% • AP/Transfer students are scoring no differently than JMU students • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers” • Same average score regardless of what course student took to completed wellness domain requirement • There is a relationship between course grade and score, indicating that test content and course content are related

  18. Conclusions KWH1Pre/Post Data(1) • Average increase of 6.1% in wellness domain knowledge between 0 credit hours and 45-70 credit hours - 53.9%  60% • “Completers” and “Non-Completers” both make similar gains over time • Might be evidence that score gains are a result of maturation not a result of taking wellness course • the lack of a sizable gain for the “completers” in comparison to the “non-completers” may be a function of the instrument used to assess the wellness domain goals and objectives. • Same gains (~ 2 pt increase) are being made on the test for students who fulfilled their requirement by taking different wellness courses

  19. Conclusions KWH1Pre/Post Data(2) • Still 14 items in Spring 2005 where < 50% of the students were obtaining the correct answer • Not seeing gains on easy items • Large increases were seen for items in the middle difficulty range as well as several items in the extreme difficulty range • Three items (5, 11, & 15) had sizable gains - ~ 20% or more of students responded correctly in Spring 2005 than in Fall 2003 • Items with no change, or change in the wrong direction were not related to any one goal or objective. - Seem to be related to the recall of specific information - Lack of improvement may be a function of the item being confusing or ambiguous and in need of revision

  20. Results of HWQ1- Part1 • First:Examine the results of the HWQ1-Part1 for 644 2nd semester sophomores in Spring 2005 • Overall Results • By JMU vs. Transfer/AP • By domain “completers” vs. “non-completers” • By which course completed • Rel. between course grade and KWH1 score • Second:Examine pre/post results for the 305 students who took the HWQ1-Part1 in both Fall 2003 and Spring 2005 • Overall pre/post difference • Pre/post difference by domain “completers” vs. “non-completers” • Pre/post difference by which course completed • Pre/post difference by item for “completers”

  21. Overall HWQ1-Part1 Results: 2nd Semester Sophomores(N = 644) JMU (n = 628) vs.Transfer/AP Students (n = 16) • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU? • No. • JMU students: M = 40.35, SD = 5.57 • AP/transfer students: M = 42.06, SD = 5.32 • Difference was not statistically, t(560)=-1.22, p = .23, nor practically significant (d = .31).

  22. Domain “Completers” vs. “Non-Completers” “Completers” N = 555 “Non-Completer” N = 82 More courses completed, the higher the mean. Differences among the means, however, are of small practical significance (d=.17)

  23. HWQ1-Part1 Means by Course Completed“Completers” Only There was a minor difference between the HWQ1-Part1 means (d = .14)

  24. Relationship between HWQ1-Part1 and Course Grade There was a positive yet small relationship between HWQ1 – Part1 Total score and course grade in both GHTH100 (r = .14) and GKIN100 (r = .05).

  25. Pre/Post Results (N = 305) Average increase of 2.1% in healthy behaviors • Spring 2005: M = 40.26, SD = 5.46 • Fall 2003: M = 38.99, SD = 5.30 • The difference between these means is of small practical significance (d = .24)

  26. Pre/Post for “Non-Completers” (n = 21) vs. “Completers” (n = 283) “Non-Completers” N = 21 “Completers” N = 283 Means for “completers” and “non-completers” are very similar. This indicates that gains in healthy behavior may simply be a function of maturation, not a function of having completed one’s wellness domain requirement.

  27. Pre/Post by Wellness Course Completed Did not report the averages for GEIC101 since only 6 students completed that course N = 165 N = 114 The same gains (~ 1 pt increase) can be expected in average HWQ1- Part1 scores for students fulfilling their wellness domain requirement by taking either GHTH100 or GKIN100.

  28. Pre/Post Difference in Items for “Completers” The pattern of averages across items for both administrations is quite similar. Relative frequencies stay about the the same over time. • Students reported engaging in Q11 and Q12 either Often or Frequently (4-6 times or 7 or more times/week). • There are increases over time in reported frequency of engaging in most healthy behaviors, with the exception of Q9, Q11 and Q12, which are decreasing over time.

  29. Pre/Post Item Differences As second semester sophomores, students are spending less time socializing with friends or family and eating more hamburgers, hotdogs and sausages than they did as incoming freshmen. The largest increases in healthy behaviors include the frequency with which students exercise to strengthen muscles (Q10) and do stretching exercises (Q13).

  30. Conclusions HWQ1-Part12nd Semester Sophomore Data • AP/Transfer students are scoring no differently than JMU students • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers” • Same average score regardless of what course student took to completed wellness domain requirement • There is a small, yet positive relationship between course grade and score, indicating that test content and course content are related

  31. Conclusions HWQ1-Part1Pre/Post Data(1) • Average increase of 2.1% in healthy behaviors between 0 credit hours and 45-70 credit hours • “Completers” and “Non-Completers” both make similar gains over time • Might be evidence that score gains are a result of maturation not a result of taking wellness course • the lack of a sizable gain for the “completers” in comparison to the “non-completers” may be a function of the instrument used to assess the wellness domain goals and objectives. • Same gains (~ 1 pt increase) are being made on the test for students who fulfilled their requirement by taking different wellness courses

  32. Conclusions HWQ1-Part1Pre/Post Data(2) • Same pattern of averages across items for both administrations • Almost all healthy behaviors show an increase in how often students are engaging in them - Q9, Q11, and Q12 are exceptions, these decrease • Students have increased the frequency for which they exercise to strengthen muscles and stretch

  33. Results • First: Examine the results of the HWQ1-Part2 for 644 2nd semester sophomores in the Spring 2005 • Overall Results • By JMU vs. Transfer/AP • By domain “completers” vs. “non-completers” • By which course completed • Rel. between course grade and KWH1 score • Second: Examine pre/post results for the 305 students who took the HWQ1-Part2 in both Fall 2003 and Spring 2005 • Overall pre/post difference • Pre/post difference by item for “completers”

  34. Overall HWQ1-Part2 Results: 2nd Semester Sophomores (N = 644) On average, students are answering 60% of the items correctly. The majority of students have scores that range from 22-28.

  35. JMU (n = 628) vs.Transfer/AP Students (n = 16) • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU? • No. • JMU students: M = 24.28, SD = 3.40 • AP/transfer students: M = 24.31, SD = 3.88 • Difference was not statistically, t(560)=.04, p = .97, nor practically significant (d = .009).

  36. Domain “Completers” vs. “Non-Completers” “Completers” N = 555 “Non-Completers” N = 82 Although the Part 2 average was higher for “completers”, the difference between the two averages was minor (d = .11).

  37. HWQ1-Part2 Means by Course Completed“Completers” Only There was no difference in the averages for students taking GHTH100 as opposed to GKIN100 (d = .18)

  38. Relationship between HWQ1-Part2 and Course Grade r = .25 Removal of the outlier did not substantially change the descriptive statistics and correlations r = .08

  39. Pre/Post Results (N = 305) • Spring 2005: M = 24.35, SD = 3.33 • Fall 2003: M = 22.57, SD = 3.39 • The difference between these means is of moderate practical significance (d = .54) Average increase of 3.4% in ability to discern “myths” from “facts”

  40. Pre/Post Difference in Items for “Completers” • Statements that were easy for students to identify correctly as myths or facts as freshman (90-100% obtained the correct answer) remain easy for them as second semester sophomores. • For the majority of the remaining items, there is improvement over time in students’ ability to identify statements correctly as myths or facts.

  41. Easy Items Items in which 88% or more of incoming freshmen are identifying correctly as myths or facts Since so many students know these concepts coming into JMU, little to no time needs to be allocated to this content in the wellness domain course

  42. Items With Large & Moderate Gains

  43. Items with No or Very Small Gains Proportion Correct Item Item Content Fall 03 Spring 05 Diff.

  44. Conclusions HWQ1-Part22nd Semester Sophomore Data • 2nd semester sophomores are scoring on average 60% • AP/Transfer students are scoring no differently than JMU students • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers” • Same average score regardless of what course student took to complete wellness domain requirement • There is a small, yet positive relationship between course grade and score, indicating that test content and course content are related

  45. Conclusions HWQ1-Part2Pre/Post Data • Average increase of 3.4% in ability to discern between “myths” from “facts” between 0 credit hours and 45-70 credit hours - 70%  73.4% • Statements easy for freshman (90-100% answered correctly) remained easy for them as 2nd semester sophomores - Since so many students know these concepts coming into JMU, little to no time needs to be allocated to this content • For the majority of the remaining items, there is improvement over time in students’ ability to identify statements correctly - May be a result of them having completed their wellness domain requirement, or maturation • More time may need to be allocated towards items with small or no change or statements should be revised

More Related