1 / 38

Helen Shen and Chen-hui Tsai The University of Iowa

Helen Shen and Chen-hui Tsai The University of Iowa. Evaluating Reading Proficiency Gain and Program Effectiveness: A Three-tiered Web-Based Reading Assessment Approach. Introduction.

richard-rivers
Download Presentation

Helen Shen and Chen-hui Tsai The University of Iowa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Helen Shen and Chen-hui Tsai The University of Iowa Evaluating Reading Proficiency Gain and Program Effectiveness:A Three-tiered Web-Based Reading Assessment Approach

  2. Introduction This presentation reports the evaluation system for “The comprehensive web-based Chinese Reading Program” supported by a DOE grant. • The program • The evaluation

  3. The Program: Purpose • To remedy the acute dearth of appropriate Chinese reading materials (beyond the class textbook) • To furnish a superior Chinese reading environment in which reading can take place anytime, anywhere • To improve users’ vocabulary knowledge, reading skill, and cultural knowledge through extensive reading

  4. The Program: The Targeted Users • College students who are taking Chinese courses at beginning, intermediate, and advanced levels • High-school students who are taking Chinese AP courses • Individuals who work in government agencies or private organizations and who wish to improve their reading skills in a systematic way

  5. The Program: Content • Three levels of reading materials (900 Lessons) • 90 achievement tests • 15 proficiency tests

  6. The Program: Format • Flash-embedded HTML files • Image files • NJStar files • Audio HTML files

  7. The Program: Academic strength • Proficiency-guided definable reading levels • Cultural theme-based reading unit organization • Fluency-oriented progressive reading skill building • Emphasizing Intentional vocabulary gain • Supporting Constructive responses • Three-tiered web-based assessment system

  8. The Program: The Design

  9. The Evaluation A Three-tiered web-based assessment system is developed to assess both individual reading achievement and the overall program effectiveness • The flash-based built-in reading outcome assessment for individual users • The FileMaker Pro reading database • The Quia web survey tool

  10. The Flash-based Reading Outcome Assessment for Individual Reading Proficiency Gain • Vocabulary pre- and post tests • Multiple choice and open-ended Reading comprehension questions • Reading speed and comprehension rate (REI) report • Achievement tests • Proficiency tests

  11. The FileMaker Pro Database This readingdatabase allows us to record: • User’ contact information • User’s reading activities (reading time, reading comprehension score, reading lessons completed, vocabulary tests score.) • Built-in on-line survey

  12. The Evaluation:Quia Web Survey • The Quia survey data allow us to collect users’ additional quantitative and qualitative feedback on certain arising issues during the use of the program. http://www.quia.com/sv/246903.html

  13. The Evaluation: Data Analysis Data from FileMaker Pro • Distribution of Users in the world • Distribution of Reading levels and topics among users

  14. The Evaluation: Data Analysis Distribution of Users in the world (78 countries; 1771 valid entries)

  15. The Evaluation: Data Analysis Distribution of Reading levels and topics among US users (2741)

  16. The Evaluation: Data analysis Data from The Quia web survey: One month reading activity and survey (February 2009) • Distribution of users and age groups in the world • Qualitative assessmentof program effectiveness • Quantitative assessment of program effectiveness

  17. The Evaluation: Data analysis Distribution of users’ nationality and age groups (valid entries: 87), Feb. 09

  18. The Evaluation: Qualitative Qualitative data

  19. The Evaluation: Quantitative Asia: from a 7-point scale ; 35 users images are clear.

  20. The Evaluation: Quantitative • America: From a 7-point scale; 37 users images are clear.

  21. The Evaluation: Quantitative T-test on the comparison of the ratings between Asia and US

  22. Conclusion: The three-tiered web-based assessment system is an excellent approach conducting the formative assessment for the program, which helps us to know users needs, expectations, and problems they are encountering so that we can adjust the program based on users’ direct and indirect feedback. This system contributes to maintain a long-lasting high qualityprogram.

  23. The reading program is free to all users!!! http://chinesereadingworld.org http://collections.uiowa.edu/chinese Contact information: Helen-Shen@uiowa.edu Chen-hui-Tsai@uiowa.edu

  24. The End

  25. HTML file Back

  26. NJStar Back

  27. Audio files Back

  28. FileMaker Back

  29. Pretest Go to Post-test

  30. Posttest Back

  31. multiple Go to Open Q

  32. Open-ended Questions Back

  33. Final Report Back

  34. Achievement Test Back

  35. Proficiency Test Back

  36. FileMaker Back

  37. Built-in Survey Back

  38. Quia Survey

More Related