1 / 34

Rapid Research in the Educational Technology Development Process

Rapid Research in the Educational Technology Development Process Cara Lane , PhD Research Scientist Catalyst Research & Development Office of Learning & Scholarly Technologies UW Technology University of Washington Session Outline Part I: Research overview

Ava
Download Presentation

Rapid Research in the Educational Technology Development Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rapid Research in the Educational Technology Development Process Cara Lane, PhD Research Scientist Catalyst Research & Development Office of Learning & Scholarly Technologies UW Technology University of Washington

  2. Session Outline Part I: Research overview Part II: Needs assessment & usability Part III: Best practices & institutional data

  3. http://catalyst.washington.edu

  4. Our Work • Web Tools • 13 modular tools • Free to UW community • Learning Spaces • Student computer labs • Computer classrooms • Advanced facilities • Knowledge Services • Center for Teaching, Learning, & Technology • Faculty and student workshops • Research

  5. Types of Research • Assessing user needs • Conducting usability studies • Identifying best practices • Gathering institutional data

  6. Approaching Research as Service • Our research team • 1.5 staff members and 1 graduate assistant • Collaboration with designers, outreach staff, and anyone else we can find • Process: • Unlearning what we learned about research in graduate school • Overcoming unique challenges • Developing new practices

  7. Unlearning Research Myths Myth #1: All research should be grounded in theory Myth #2: All research must present your best thinking Myth #3: Research is an independent or small team endeavor

  8. Research Challenges • Frequent organizational change • Fitting research into development timeline • Does UW really want to learn best practices from the IT shop? • Perception of bias

  9. Organizational Change

  10. Discussion • Where does this type of research fit within your institution? • How do you currently gather information about technology use and needs? • Are there projects you are currently working on where it would be useful to add a research component?

  11. Part II: Needs Assessment & Usability Studies or How fast do you need that?

  12. Needs Assessment • CommonView • Course or project workspaces • Integrate multiple Web tools; maintain flexibility of use • Questions • How do Catalyst users integrate multiple tools? • How do users of courseware use features of those tools? • What are the unmet needs of both groups?

  13. Challenge: Time • Pace of development cycle • 1 month to develop study, recruit participants, and gather data • Our response: • Collected internal evaluation data only • Expanded team: designers & outreach staff • Developed new focus group procedures • Streamlined data analysis

  14. Focus Group Format • Traditional model • IRB approval for research study • Formal, written consent procedures • Audio-recording and transcription of session • New procedures • Evaluation data for UW internal use • No identifying information recorded • 1-2 note-takers per session • Observations added to notes/gaps filled

  15. Data Analysis Process • By population • Meeting of leaders and note-takers • Review of notes • Documentation of findings • Most students reported accessing their course Web sites using bookmarks. They had difficulty finding the sites from computers other than their own.

  16. Challenge : Bias • UW committees want to know user needs • We have the most information • Can others trust our information? • Our response: • Share overall summary of findings (incl. those unflattering to our group) • Detail purpose of study and limitations of findings • Explain research and development process

  17. Usability Studies • Conducted by design team • Several times during design and development cycle • Challenge: • Quickly finding sufficient numbers of diverse users to test tools • Our response: • Creation of Catalyst User Experience (CUE) project

  18. CUE • Began in 2007 • 241 members • Initial membership survey • 11 targeted usability studies and surveys • CUE: 60-90% response to invitations • Old methods: 5-20% response • Biannual newsletter

  19. Discussion • Examples of other fast-paced research techniques? • What strategies do you use to share information about user needs among IT groups and with other segments of university of college community?

  20. Part III: Best Practices & Institutional Data or Who wants to learn this stuff from the IT shop?

  21. Identifying Best Practices • Inter/National Coalition for Electronic Portfolio Research (cohort 1) • 3-year study of ePortfolio use • UW contests • Identify spectrum of student use • Spring 2005 & 2007 • Case study: Expository Writing Program • Gather knowledge about implementation path • 2005/6 pilot & 2006/7 option

  22. Our Process • IRB: Minimal risk research • Data Gathered: • Interview transcripts for students, TAs, and administrators • Surveys of contest applications, EWP students, and EWP TAs • Review of student portfolios and instructional materials

  23. Challenge: Getting campus to listen • Are we the right people to be discussing best practices? • Is pedagogy our focus right now? • Our response: • Practical reports for UW audience • Breadth of data; can tell multiple stories • Academic publications for larger audience

  24. Gathering Institutional Data • 2008 Faculty, Teaching Assistant, and Student Surveys on Learning and Scholarly Technologies • Campus-wide surveys • All instructors of record (approx. 4000) • Stratified sample (by broad disciplinary group) of 1000 TAs • Stratified sample of 5000 students

  25. Challenge: Time • How can we gather enough information to write up-to-date surveys when the survey itself is our main vehicle for gathering information on current needs? • Our Response: • Autumn focus groups with faculty, TAs, students • IRB: Exempt research • Utilized needs assessment focus group process • Expanded team: staff from UW Technology, Libraries, and Office of Educational Assessment • Provided focus group training sessions for staff

  26. Challenge: Getting campus to listen • How can the survey results best inform technology-related decisions on campus? • Our response: • Survey focus on goals and opinions, rather than simple snapshot of use • Extensive collaboration • Units who can implement changes are part of team • Practical reports and presentations for UW audience • Academic publications for larger audience

  27. Discussion • Examples of other similar projects or challenges? • What strategies can we try for establishing research partnerships across campus? • What strategies can we use to share research processes and findings between institutions?

  28. Contact Information • Cara Lanecgiacomi@u.washington.edu • Catalysthttp://catalyst.washington.edu

More Related