1 / 20

Development of Evaluation Instruments based on ICT Skills Standards for Teachers (ISST)

Development of Evaluation Instruments based on ICT Skills Standards for Teachers (ISST). 7th May, 2014. CONTENTS. Request of MOE. I. Proposal for Evaluation. II. Development Strategy & Schedule. III. Results & Expectation. IV. Request of MOE. I. Request of MOE. Mixed Model(method).

pahana
Download Presentation

Development of Evaluation Instruments based on ICT Skills Standards for Teachers (ISST)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of Evaluation Instruments based on ICT Skills Standards for Teachers (ISST) 7th May, 2014

  2. CONTENTS Request of MOE I Proposal for Evaluation II Development Strategy & Schedule III Results & Expectation IV

  3. Request of MOE I

  4. Request of MOE • Mixed Model(method)

  5. Definition of Mixed Methods Evaluation • Mixed methodology is a design for collecting, analyzing, and mixing both quantitative and qualitative data in a single study or series for studies to understand an evaluation problem(Creswell and Plano Clark, 2007). Mixed Method Quantitative Data Qualitative Data • Interviews • Observations • Documents • Audio-visual materials • Instruments • Checklist • Records

  6. Proposal for Evaluation II

  7. Proposal for Objective and Valid Evaluation • Master Teacher • Leader Teacher • Multiple Choice • Estimate scientific, directive, and objective performance. • Take long time to develop the test sheet and also difficult to develop identical test sheets. • Respondents’ position, difficult to respond. • Before developing the test times, it is necessary to check the characteristics of respondents • Likert Scale Evaluation Instruments • Multiple Choice • Likert Scale • Easy to verify the reliability and validity of test items. • Respondents’ position, Easy to respond. • The same test items can be used. • It’s difficult to estimate real ability because it is based on the respondent’s perception. • Respondents can misrepresent about their ability • Leader Teacher

  8. Evaluation Framework of ISST • Domain of ICT competence for teachers is technology, communicative, pedagogical, research/investigative, management and design.

  9. Teacher Competencies for ICT Use in Education • According to the MEN (2013), there are three levels of all teachers’ competencies: Explorer, Integrator, and Innovator.

  10. Development Strategy & Schedule III

  11. Development Strategies and Schedule Likert Scale • Preliminary test with • 300 samples • 6. 1 ~ 6. 7 • Develop & Confirm • Indicators • 5.10 ~ 5. 20 Setting the criteria 6. 15 ~ 6. 25 • Analysis and set up • the tests • 6. 8 ~ 6. 15 • Expert review • 5.20 ~ 5. 30 Multiple Choice • 2st Preliminary test • With 300 samples • 7. 10 ~ 7. 20 • 1st Preliminary test • With 300 samples • 6. 25 ~ 7. 5 • Development of • taxonomy • 6. 1 ~ 6. 10 Setting the criteria 7. 25 ~ 7. 31 • Analysis and revise • the tests • 7. 5 ~ 7. 10 • Analysis and set up • the tests • 7. 20 ~ 7. 25 • Item development • 6. 10 ~ 6. 25

  12. Development Methodology • Validation of Evaluation Instruments • Content validity (through experts’ review) • Construct validity (through statistic program) • Item response theory (through statistic program) • Method for Setting the Criteria ( Standard-setting methods) • Angoff & Jaeger method

  13. Results & Expectation IV

  14. Representation of Evaluation Result • Radial Chart Investigative

  15. Setting the Level of Teachers based on Criteria Integrator Explorer Innovator Score Score Score

  16. Output • Likert Scale • At least 72 items • Multiple Choice • At least 100 items

  17. ISST & Evaluation Items Development

  18. ISST & Evaluation Items Development Draft ISST Test Item ( 4. 23 ~ 5.8 ) Validate Test Items ( 5. 15 ~ 5.30 ) Development of ISST ( 3.4~4.30 ) Conduct Expert Review ( 5. 8 ~ 5.15) Construct Test Sets ( 5. 30 ~ 6.5) Prepare Draft Report ( 6. 17 ~ 7.25) Review and Confirm Test Sets ( 6. 5 ~ 6.11 ) Submit Test sets ( 6. 17 ) Review by MOE and Revise the Report ( 7. 25 ~ 8.18 ) Pilot Test and Port the Online Test System ( 6. 11 ~ 6.17 ) Submit Final ISST Research Report ( 8. 18 )

  19. ISST & Evaluation Items Development • Construct Test Sets • ( 4. 30 ~ 5.5 ) • Draft Likert Scale Item • ( 5. 10 ~ 5.20 ) Development of ISST ( 3.4~4.30 ) Preliminary Test with 300 samples ( 6. 1 ~ 6.7) • Review and Confirm Test Sets • ( 5. 5 ~ 5.10 ) • Conduct Expert Review • ( 5. 20 ~ 5.30 ) Setting the Criteria ( 6. 15 ~ 6.25 ) Prepare Report & Revise ( 6. 17 ~ 8.18 ) Submit Final ISST Research Report ( 8. 18 ) Analysis and Set Up the Test ( 6. 8 ~ 6.15 ) SubmitLikert Scale Test set ( 6. 17 ) Submit Multiple Choice Test set ( 7. 25 ) Item Development ( 6. 10 ~ 6.25 ) Analysis and Revise the Test ( 7. 5 ~ 7.10) Analysis and Set Up the Test ( 7. 20 ~ 7.25 ) Development of Taxonomy ( 6. 1 ~ 6.10) 1st Preliminary Test ( 6. 25 ~ 7.5 ) 2st Preliminary Test ( 7. 10 ~ 7.20) Setting the Criteria ( 7. 25 ~ 7.31)

  20. Question & Answers

More Related