1 / 22

Meaningful Assessment and the Impact of Technology

Meaningful Assessment and the Impact of Technology. Catherine Kelley, Ph.D. Senior Faculty Consultant Assistant Professor. Assessment of Mobile Computing at Seton Hall. Catherine Kelley, Ph.D. Assistant Professor, Psychology Senior Faculty Consultant, TLTC. Why assess?.

jfreddie
Download Presentation

Meaningful Assessment and the Impact of Technology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Meaningful Assessment and the Impact of Technology Catherine Kelley, Ph.D. Senior Faculty Consultant Assistant Professor Meaningful Assessment and the Impact of Technology

  2. Assessment of Mobile Computing at Seton Hall Catherine Kelley, Ph.D. Assistant Professor, Psychology Senior Faculty Consultant, TLTC Meaningful Assessment and the Impact of Technology

  3. Why assess? • Track progress being made on your project • Greater understanding of teaching & learning • Ensure high-quality experience for your students • Communicate successes / challenges to colleagues Meaningful Assessment and the Impact of Technology

  4. Traditional (Flawed) theoretical model: • Teaching => Learning • Teaching + Computers => Improved Learning • What is wrong with this model? Meaningful Assessment and the Impact of Technology

  5. Traditional (Flawed) Research Design: • Control group - no computer • Experimental group - computer • Measure exam scores • What is wrong with this design? Meaningful Assessment and the Impact of Technology

  6. Some problems with traditional model • Hawthorne effects • Multivariate nature of change means that an experiment would be hopelessly confounded • Your outcome variables may (will probably!) change with new teaching techniques • You may not know yet what the really important outcome variables are! Meaningful Assessment and the Impact of Technology

  7. What about a multivariate model? • So Many Variables! • Each So Difficult to Measure! • Huge n required to do any meaningful multivariate study • Not a bad idea, but was outside of the intended scope of our project Meaningful Assessment and the Impact of Technology

  8. So what do you do? • What about educational quality variables? • Theoretical models of educational quality: e.g. Chickering & Gamson, “Powerful Partnerships,” etc. • Common themes emerge Meaningful Assessment and the Impact of Technology

  9. What is “educational quality”? • Student - professor contact • Student - student contact • Prompt feedback on performance • Explore course materials in many ways • Take responsibility for learning • Etc. Meaningful Assessment and the Impact of Technology

  10. Seton Hall Instruments • Two Versions: “Standard Form”, “New Form” • Standard Form has been used for several years • New Form has been piloted with considerable success Meaningful Assessment and the Impact of Technology

  11. Standard Form • “Compare your mobile courses, or those incorporating information technology, to your non-mobile courses, or those not incorporating information technology. Indicate how the use of information technology impacts each of the following items.” Meaningful Assessment and the Impact of Technology

  12. Standard Form (examples) • Have contact with my professors on course content • Spend time with other students on course related materials • Discuss, write and think about course material in depth • Get prompt feedback on my performance from my professors …. • 29 of these questions Meaningful Assessment and the Impact of Technology

  13. New Form • Same questions, but students answer only about one course • And then indicate how much IT is used in that class • But also, how many traditional pedagogical techniques are used in that class Meaningful Assessment and the Impact of Technology

  14. Results:Old Form • Mobile courses look better on educational quality - Sum across all measures Meaningful Assessment and the Impact of Technology

  15. Results: Old Form Meaningful Assessment and the Impact of Technology

  16. Results: New Form • The more IT, the higher the educational quality score Meaningful Assessment and the Impact of Technology

  17. Results: New form Meaningful Assessment and the Impact of Technology

  18. Results: New Form • The more traditional techniques, the higher the educational quality score – but effect is very weak Meaningful Assessment and the Impact of Technology

  19. Results: New Form Meaningful Assessment and the Impact of Technology

  20. Summary of findings • Students perceive that educational variables are higher in Mobile classes • Student satisfaction with Mobile remains high Meaningful Assessment and the Impact of Technology

  21. The future • Institutionalizing the assessment process • Risks: over-surveying students, poor response rate • Asking some tough questions: Retention, student skill, negative impact of technology • Surveys of faculty, administrators Meaningful Assessment and the Impact of Technology

  22. The future • Assessment reports should also make use of archival data, not just surveys • E.g. how many courses make use of technology? And to what extent? • Are we realistic in our expectations of faculty? Administrators? Meaningful Assessment and the Impact of Technology

More Related