1 / 17

Critical Issues in Evaluating Educational Technology

Critical Issues in Evaluating Educational Technology. Mark Hawkes Dakota State University Presentation at the American Evaluation Association Annual Conference, Nov. 2, 2000. 1997-98 1998-99 1999-2000. $5.2 Billion $6.0 Billion $7.2 Billion. School Expenditure on Technology.

welcome
Download Presentation

Critical Issues in Evaluating Educational Technology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critical Issues in Evaluating EducationalTechnology Mark Hawkes Dakota State University Presentation at the American Evaluation Association Annual Conference, Nov. 2, 2000

  2. 1997-98 1998-99 1999-2000 $5.2 Billion $6.0 Billion $7.2 Billion School Expenditure on Technology

  3. The Secretary’s Conference on Educational TechnologyEvaluating the Effectiveness of Technology • The effectiveness of technology is embedded in the effectiveness of other school improvement efforts. • Standardized test scores offer limited formative information with which to drive the development of a school’s technology program.

  4. Critical Issues… • Current evaluation practices need broadening. • Schools must document and report their evaluation findings in ways that satisfy diverse stakeholders’ need to know. • Everyone must agree on a common language and standards of practice for measuring how schools achieve that end.

  5. Critical Issues . . . • The role of teachers is crucial in evaluating the effectiveness of technology in schools. • Some existing policies need to be “transformed” to match the new needs of schools using technology.

  6. Scope of Evaluation: status then • Technology focus: Drill-and-practice software, integrated learning systems, videotaped lessons, computer-animated picture books, low-level problem-solving and simulation computer software • Professional Development: Participation, topics, accessibility, quality • Hardware capability: Processing speed, memory, application functionality • Student products • Rate of implementation

  7. Scope of Evaluation: status now • Technology: information-processing and productivity tool • Professional Development: relationship to student performance with technology • Breadth of technology vision • Level and breadth of ownership • Community involvement • Propensity for new practice

  8. Metrics: status then • Standardized measures (IGAP, ITBS, CTBS) • Student-to-computer ratio • Survey: • Self-report (competency, nature of use) • Affect oriented • Behavioral indicators • Attendance/Graduation rates • Principal referrals • Post-secondary options • Teacher discourse: Technology-centered

  9. Metrics: status now • Critical incident analysis • Incidents of learning activities that reflect job-related skills and performances • Token Economies • Media Production • Comparative science • Nature of discourse, signs of: • Teacher-centered—nature of pedagogy • Student centered—constructivist pedagogy • Classroom observation of innovative uses

  10. Role of Teacher: status then • Hands on time with the technology • Trips to the lab • Types of activities • Tools used • Report affective measures • Student attitudes toward technology • Document categories of outcomes (motivational, behavioral)

  11. Role of Teacher: status now • Teacher activity logs • Journaling on how teachers implemented technology facilitated instructional strategies with corresponding outcomes • Content area professionals in examining how technology is linked to state learning standards • Examining relationship between instructional practices and teaching and learning roles

  12. Technology Policy: status then • Lab topology; limited access detached form the learning experience • Access to technology resources (equity) • Focus with use policy and security • Acceptable Internet use • Email accounts

  13. Technology Policy: status now • Aligning school support systems with technology outcomes • Libraries as information centers • Continuous financial support to sustain system components • Exploring technology use opportunities that extends learning beyond the school day • Extended lab hours • Lap top check-out

  14. Inputs Outputs • Infrastructure Student motivation • Wireless features Accessibility • Restructured leadership Improved PD • New roles for Improved use of support services services • Individual Staff Renewed teacher development plans skill building • Competitive Improved efficiency collaboration grants in technology use

  15. Over the two years . . . • Continued reliance on standardized measures • Policy at all levels remain prohibitive in many cases • Improved support for technology purchases and professional development • Limited examination of systemic indicators: - Vision of technology use - Support for technology implementation - Community roles and involvement - Administrative effectiveness

  16. Over the two years . . . • Parsing the effects of technology from the numerous school improvement ventures in the school proves virtually impossible • Growth needed in collaborative “gap analysis” and deployment of technology to address those gaps • Efforts to evaluate look quasi-experimental, have more attributes of research than evaluation

  17. Slides available at:www.homepages.dsu.edu/hawkesm/ Presentations.htm Paper available at: www.ed.gov/Technology/TechConf/1999/confsum.html

More Related