Assessing the Aesthetic: Learning from Arts Assessment in Humanities and Sciences

1. From Abstract to Authentic: What the Humanities and Sciences Can Learn from Assessment in the Arts Jill L. Ferguson, J. Joseph Hoey IV The Learner Conference – Madrid, Spain July 11, 2015

2. Session Content • Clarify challenges of assessing the aesthetic • Present models, tools, and examples of assessment appropriate to creative disciplines and others, including the humanities and sciences • Draw from Assessment in Creative Disciplines: Quantifying and Qualifying the Aesthetic, by David Chase, Jill L. Ferguson, and Joseph Hoey (Common Ground, 2014) and Reframing Quality Assurance: Evidence from Practice by J Joseph Hoey IV and Jill L. Ferguson

3. What Are Your Objectives? What issues are important to you and your program in the assessment of student learning? Where do you think approaches from the arts might help?

4. Intended Learning Outcomes • Articulate the challenges of and opportunities for assessment in creative disciplines; • Employ concrete examples of how institutions are using assessment techniques in creative disciplines and improving student learning; • Apply the principles of quantifying and qualifying the aesthetic to assess learning outcomes in the humanities and STEM disciplines; and • Create an assessment blueprint for assessing our own programs.

5. Higher Education Context • Increasing demand to prove educational effectiveness • We are being asked to develop a new approach that incorporates expert-based judgment as well as providing transparent, replicable assessment across students. • Criteria for arts assessment need to understand student performance on an individual, collective, and on an aggregate basis.

6. The Best Assessment Methods Enable Us to Connect The Dots Graphic Credit: Leo Blanchette (2013)

7. Learning Domain Taxonomies – Expanded • Cognitive • Affective • Psychomotor • Intrapersonal • Interpersonal • Conative • (See handout)

8. Learning Outcomes in the Arts • Craft Outcomes • Technique and technical skills and competency • More amenable to traditional evaluation and assessment methods • Aesthetic Outcomes • Issues of artistic development – expression and voice • Introduces additional nuance and complexity compared to traditional evaluation and assessment methods

9. Metacognitive Learning Outcomes: The Self-Aware Artist • Introspection, self-critique and metacognition are basic to the arts and the development of the artist • Critique, Juries, Panels • Student learning assessment in the arts is facilitated by assessment within the professional context or close approximation. Examples: • Charrettes and design competitions • External projects

10. Learning Outcome: Building Creative Expression • Creativity: Fundamental to arts assessment • Creativity in Other Disciplines: Five Colleges of Ohio Creative and Critical Thinking Project: http://www3.wooster.edu/teagle/ • Creative Institutions: European University Association Creativity Project, http://www.eua.be/eua-work-and-policy-area/quality-assurance/projects/creativity.aspx

11. Big-C and Mini-C Creativity • Big C: • Who will be the next Beethoven, Michelangelo, or Einstein? • New ideas and contributions that have broad impact to a domain of learning • Mini-C: • “Coming up with fresh ideas for changing products, services and process so as to better achieve the organization’s goals” (Amabileet al., 2005).

12. 4Ps Model of Creativity (Adapted from Rhodes, 1961)

13. Unpacking the 4Ps – PersonIsaksen’s (2008) ‘Eight Pack’ of Personality Factors Cognitive Affective • Flexibility • Fluency • Originality • Elaboration • Curiosity • Complexity • Risk-taking • Imagination • Questions to ask: • How creative is the student? • How is the student creative?

14. Unpacking the 4Ps - Process • Wallas’ (1926) Stages in the Creative Process: • Stage 1: Preparation – Divergent Thinking • Stage 2: Incubation • Stage 3: Illumination • Stage 4: Verification

15. Unpacking the 4Ps - Product • Besemer (2006): Creative Product Semantic Scale • Novelty dimension: How new or original? • Resolution dimension: How well does it solve the relevant problem? • Style dimension: How far beyond the basic requirements needed?

16. Unpacking the 4Ps - Press • In what social context or environment does creativity thrive? • Is it conducive or hindering? Trusting or not? • Van Gundy (1985): • Internal (personal) perceptions of the environment • External: physical and other factors in the environment • Social interactions and relationships

17. Approaches to Teaching Creativity • Model it behaviorally! • Use high-impact instructional strategies shown to increase creativity – as we do in the arts • Inquiry-based and problem-based learning • Constructivist approaches • Peer learning • Sternberg and Williams’ 24 creativity teaching tips: http://www.cdl.org/articles/teaching-for-creativity-two-dozen-tips/

18. Course Design for Creativity Critical factors (Scott et al., 2004) • Realistic practice • Domain specific exercises • Depth and difficulty of material • Training in components of creativity • Amount of instructional feedback

19. Keys to Success in Teaching Creativity • Be explicit! • Provide guidance on cognitive and other domain-specific capacities that affect creativity • Offer students a clear set of heuristics or strategies for working on creative solutions, starting from what is currently known.

20. Types of Assessment • Direct • Based on professional judgment, guided by a decision facilitation tool • Of student work, performances, design • Indirect • Reporting about learning, attitudes, perceptions • Not a direct demonstration of competence • Passive • Derived from student records; online course observations

21. Direct Assessment Opportunities in Creative Disciplines: Examples • Performance assessments, such as a theatrical, musical, or dance performance • Portfolios and e-Portfolios • Individual and group projects/presentations, for example senior studio projects • Peer assessments and Critiques • Reflective journaling • Sketchbooks/Design Books • Externally juried design or performance competitions and charrettes • Licensure exams (for Interior Design and Architecture, for example)

22. Indirect Assessment Methods: Examples • Reflective journaling: Evidence of thought processes, evolution of ideas, development of critical reflection ability • Interviews with Individual Students • Focus groups • Surveys/Questionnaires • Employer feedback and satisfaction studies • Advisory board • Job/grad school placement data • Evidence of awards and/or recognitions in contests, competitions, or special programs

23. Unobtrusive or Passive Assessment • For online programs: All student actions and interactions are passively recorded, can be collected and analyzed, and then used for assessment purposes with other methods or the signs and traces of student learning. • Information derived from student records and agency files may also be important to overall program management when combined with other assessment data.

24. Triangulation of Assessment • For assessment results to be considered valid, a schema is necessary that relies on two out of the three methods of assessment (such as direct and indirect). (See diagram on page 74 of Assessment in Creative Disciplines: Quantifying and Qualifying the Aesthetic).

25. Triangulation Using data from different sources to answer a question about student learning.

26. Your Turn! How Would We Assess Creativity? • Get together in small groups. • Having defined a student learning outcome for creativity, how would we assess it? • How would we ladder creativity assessment within our curricula to ensure progressive student gains? • What assignments/opportunities for demonstration of student competence would we use? • What assessment tools/methods would we use and why?

27. Example: Theater and MBA Program • Collaboration between Drama and Management Departments at Royal Holloway, London. • Explores application of the ‘pediaof theatre’ to facilitate an embodied learning process on the MBA programme. • Drawing on the artistic encounter as a model of self-inquiry, the initial focus was on encouraging effective teamwork and communication. • At the heart of the course were ‘ensemble’ exercises which engaged students in personal and shared exploration within a playful environment.

28. Theatre and MBA Course Learning Outcomes and Assessment • Learning Outcomes • Experience a peer learning environment that fosters a sense of democratic collegiality • Develop management skills through creative activity • Develop self awareness • Encourage risk-taking • Develop the ability to communicate verbally and non-verbally • Assessment Methods • Observation of students’ interaction and flexible workshop design • Peer critique and tutor feedback • Final performance critiqued • Student follow-up questionnaire on meeting learning objectives

29. What Do We Document and At What Level? • For each degree program: • The student learning outcomes faculty consider most important, whether collective or individual • NOTE: If creativity is important in your program, is it stated as one of the program’s student learning outcome? • How these outcomes are defined in practice • How these outcomes are assessed • What results are gained (summary level) • Based on results gained, what actions are being taken. How are we using the results?

30. Levels of Assessment • Individual Student – Level Assessment in the Studio • Group – Level Assessment: Studio or Course Level • Program or Curriculum – Level Assessment • Institution – Level Assessment

31. Example: Individual Assessment in the Studio • n = 1: The artistic growth of the individual is the goal of instruction • Expert Evaluation: faculty set the reference for criteria and expected levels of achievement, • Referenced to departmental criteria for artistic development • Goal Setting and assessment of the individual: beginning of term, tracking through the term, progress is expressed as development of each student, and referenced to department criteria • Measurement – educational development can be rolled up for group-level assessment purposes.

32. Example: Assessment in the Applied Studio, Voice, University of the Pacific

33. Summary Points • Good assessment methods allow us to connect the dots in student learning, whether at the individual, course or program level. • Taxonomies of learning, important domains of learning, instructional design and assessment techniques used in the arts can be adapted to and can benefit student learning outcomes in other disciplines. • Primary among these is developing, operationally defining, and assessing creativity, an important learning domain for the 21st Century that varies by context and across disciplines.

34. Questions? If you are doing a promising practice that you would like to share, please e-mail us:joseph@assessmentincreativedisciplines.com or jill@assessmentincreativedisciplines.comThank You!