1 / 21

Seven Dangerous Questions: Transforming Academic Programs through IT and Assessment

This presentation explores the transformative power of assessment and IT in improving academic programs. It discusses the importance of asking the seven dangerous questions and provides examples of how data and technology can be used effectively. The presentation also delves into the challenges and costs associated with assessment and offers strategies for overcoming them.

satlas
Download Presentation

Seven Dangerous Questions: Transforming Academic Programs through IT and Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Seven Dangerous QuestionsUsing IT and Assessment to Transform Academic Programs Stephen C. Ehrmann, Ph.D. March 15, 2002

  2. Thanks! • 250+ institutional subscribers to TLT Group services • Annenberg/CPB and AAHE • Washington State Univ., St. Edward’s University, Indiana University • TLT Group Founding Sponsors • Blackboard, Compaq, Microsoft, SCT, WebCT • EDUCAUSE/NLII, CNI, AAC&U, League, AASCU • TLT Group Program Funders • FIPSE, Mellon Foundation, NSF, APS, PT3, LAAP...

  3. Outline • What’s at Stake- 7 Dangerous Questions • “Transformative Assessment?” “Alignment”? • What are you trying to improve • Criteria • Examples • What power could assessment have? • Answer the 7 dangerous questions: What sorts of data would be most useful • Assessment Infrastructure

  4. The Vision (1960 – present) • Technology, properly used, can improve education and its outcomes (the Triple Challenge: who learns, what they learn, what it costs) • Technology improves continually (Moore’s Law, etc.) • Therefore with every passing decade we ought to be able to make further improvements in education and its outcomes.

  5. Assessment isn’t for Wimps • “Dangerous questions” are: • Risky to ask! • Even riskier not to ask! (especially if other institutions or your constituents are beginning to ask) [hypothesis]

  6. Seven Dangerous Questions • As educators, how are we trying to improve results (from what baseline)? • Early in our effort to improve, is our behavior changing enough? • Where it isn’t, what’s standing in the way? • Is technology proving useful in changing what we’re doing? • Where it hasn’t been used that way, why not? • Costs and stresses – what are they? Can they be controlled? • Now that years have passed, have we moved enough (compare with #1)

  7. Programmatic Outcome (who learns, what they learned, etc.) or Strategy Mission Statement Budget Process Program Dev Staff Dev Admissions Legislative/Alumni Capital plans (etc.) Assessment – Use of Data TA & Alignment

  8. What You Can Study: Triads • The smallest number of elements to study if you’re trying to understand and shape the use of technology to improve outcomes Activity – structure in use Goal - Outcome I.T.

  9. Def’n of Transformation? { • Important, qualitative change in: Activity – structure in use Goal - Outcome I.T.

  10. Criteria for Picking Triad • Worth pursuing long enough to improve outcomes (5 years? More?) • Some aspect of this triad is already a strength and perhaps also a worry • Improving IT can play a cumulative role in improving activity in this triad • Caution: IT is just the yeast, not the whole recipe

  11. Example of a Triad

  12. Not This

  13. Another Triad

  14. And Another

  15. And Another

  16. *Triad: Yours?

  17. How to Focus (A Triad and Transformative Assessment) • Top down – commission, strategic plan, accreditation self-study • Bottom up- build momentum, coalition • Voluntary effort aimed at silent start, slow growth; • Brown bags • Workshops and talks; debates of the issues, warts and all • Voluntary studies • Grants and gifts • Community surveys, meetings • Recognize the political side of the process (in best, worse senses of the term)

  18. Assessment’s Powers • The discussion of how to collect data can help to make discussion of goals and strategies less abstract, agreements more meaningful • Discussion of alignment can help, too. • Periodic findings can refocus attention • Spotlight problems as well as achievements in order to attract resources • Steering, e.g., grab hidden opportunities, reduce or remove barriers

  19. *Seven Dangerous Questions • (Sooner): baseline of outcome • Is the activity on the increase? • Where it isn’t what’s standing in the way? • Technology helping? • Barriers to IT effectiveness? • Costs and stresses from the effort? • (Later): Outcome improving?

  20. Assessment Costs, Too! • People to do programmatic studies? Offices and budgets? Missions? • Current activities (helping others do studies? Training others?) • Assessment linked to mission and strategy? Autonomous? Adrift? • Reward system for people who take the risk of spending time, money on studies? • Training for them? Expert assistance? Tools for doing studies? ‘Dogwork’ assistance?

  21. We’re Done! • Questions or comments? • Give me a card or send me e-mail if you want these slides, information about how Flashlight tools or Network membership might help your institution move on these issues Ehrmann@tltgroup.org

More Related