Measuring Technological Literacy

1. Measuring Technological Literacy Michael Walach

2. For Anna, Matthew, Julie And Emily

3. Technology & Technological literacy • Technology: The way people modify the natural world to suit their own purposes...the diverse collection of processes and knowledge that people use to extend human abilities and to satisfy human needs and wants. (International Technology Education Association, 2000/2007, p. 2). • Technological Literacy: The ability to use, manage, assess, and understand technology (International Technology Education Association, 2000/2007, p. 9).

4. Why study technological literacy? • Little to no research on student performance in the area of technological literacy. • NAEP will measure technological literacy in 2014. • National focus on STEM education. • RI recently approved technology and engineering GSEs. • Inform teacher practice to improve and advance the field of technology education.

5. Questions • 1. Are there statistically significant group differences on a measure of technological literacy based on gender, race and/or socio-economic status? • 2. What factors are common among the highest scoring technologically literate students? • 3. What common factors exist among students who struggle to achieve technological literacy?

6. Method • Mixed method • Quantitative • Technological Literacy Assessment • Qualitative • Interviews of highest and lowest scorers

7. Technological Literacy Assessment (TLA) • Developed by the researcher • Input from technology teachers in Rhode Island (RITEEA) • Input from STEM specialists at Boston Museum of Science • Questions aligned to the Standards for Technological Literacy (ITEEA) • Questions validated by 25 high school students

8. Population Rhode Island High School Students 9th-12th grade Participants by grade • 9th grade: 63 • 10th grade: 6 • 11th grade: 10 • 12th grade: 11 • Total 90

9. Test sites Breakdown • 1 Urban 4 participants • 1 Rural 22 participants • 2 Suburban 64 participants Obstacles • Reluctance of Administration to allow access or sacrifice class time • Difficulty getting informed consent at two of the test sites • Random sampling not allowed

10. Testing Procedure • TLA software developed by the researcher • 3 separate programs developed (testing, reviewing, random sample/password generation) • Software run from USB flash drive • 50 USB drives purchased • Class rosters collected from each test site • Software customized to contain users at each test site • Username and password required for test activation • Scores automatically saved back to flash drive in hidden file folder • Informed consent from student and parent required for testing

11. Incentives • One RC helicopter purchased for each test site • All test takers eligible to win • $20 Gift card offered as alternative to helicopter

12. TLA Data • Males scored statistically significantly higher than Females on raw score.

13. Tla data • No statistical difference based on race

14. Tla data • No statistical difference based on Socio-economic status

15. Tla data • No statistical difference based on father’s education

16. Tla data • No statistical difference based on mother’s education.

17. Father education vs. mother education • While not statistically significant, the relationship between father’s education and TLA score was stronger than the relationship between mothers education and TLA score. • P=0.473 father • P=0.701 mother

18. Standards by Gender • STL-8 understanding of the attributes of design • STL-9 understanding of engineering design • STL-14 medical technologies • STL-15 agricultural and related biotechnologies • STL-16 power and energy technologies • STL-17 information and communication technologies • STL-18 transportation technologies • STL-19 manufacturing technologies • STL-20 construction technologies

21. Highest scores by standards Engineering as a common thread

22. Interest Inventory Which of the following would you enjoy doing? Select all that apply. • building something with tools • taking something apart to see how it works • fixing a car or small engine • programming a computer • playing a computer game • none of these interest me Which of the following would you enjoy doing? Select all that apply. • planting a garden • working with animals • taking care of people who are sick • inventing products to assist people with disabilities • raising fish for food • none of these interest me

23. Interest inventory Supports findings by Mitts (2008) females prefer life science and social interaction, males prefer to build.

24. Games played as child

25. The interviews • Bruce • Nicholas • Jasmine • Donna

26. Bruce • TLA Score 82.93 • Highest scoring student in the state • Teacher did not understand why I wanted to interview this student • Strong interest in math and science • Liked to take things apart, did not like to rebuild them. • Spends most of his time on a computer • Not very social • Not interested in working with tools • Likes theory more than application

27. Nicholas • TLA score: 73.17 • In the top 10 • Building a hydrogen fuel cell for fun • Interested in aviation, science and math • Wants to be an engineer and astronaut • Watches on-line science videos • Taking extra math class in order to reach calculus II AP • Builds a lot of experiments at home

28. Jasmine • TLA Score: 75.61 2nd highest female score in state (#1 did not want interview) • Enjoys tumbling and sailing • Travels a lot with family • Interested in a career in Law • Mother runs junior solar sprint

29. Donna • TLA score: 60.98 • 10 points above female mean score • Rural high school • Divorced parents • Living with boyfriend • Loves machines • Artist • Likes to take things apart • Likes to build but not with plans • Good understanding of technology and its effects on society

30. What did the interviews add? Top scoring students: • were very “book smart” • Top students like to take things apart, but were not good at or didn’t like putting things together. • Did not like following plans/blueprints • Free thinkers • Had strong understanding of technology (Donna, Bruce, Nicholas). Low scoring students: • Not interested in fixing things or seeing how they work • Interested in art, history and English • Most played sports • Had a simple or poor understanding of technology (Paul, Addison, Samantha).

31. Are there statistically significant group differences on a measure of technological literacybased on gender, race and/or socio-economic status? • Gender-Yes • Race-No, but inconclusive n=11 • SES No, but inconclusive n=14

32. What factors are common among the highest scoring technologically literate students? • Scored highest on engineering and transportation. • Played with blocks and/or liked to build things as a child. • Played with toy vehicles as a child. • Prefer Math and Science

33. What common factors exist among students who struggle to achieve technological literacy? • Scored highest on engineering design and medical technology • Enjoyed playing with dolls/action figures as a child • Prefer History and English

34. Recommendations • Promote Medical, Agricultural and Biotechnology programs. • Develop gender friendly technology activities. • Use engineering as the vehicles for teaching about technology. • Give students choice in design activities. • Have design problems that help people or society. • Include life sciences in technology (plants, animals, people).

35. Further Research • Expand the TLA questions and target a larger more diverse sample • Case studies of Medical and Biotechnology programs • Student response to design problems of various topics • Are technology activities male focused?

36. Questions? • ???

37. Thank you!