270 likes | 353 Views
Tools for Assessing Perceptions and Uncovering Influence. Jim Dearing Center for Health Dissemination and Implementation Research Kaiser Permanente Colorado Synergy Project for Research, Practice and Transformation January 10-12, 2010, Albuquerque NM.
E N D
Tools for Assessing Perceptions and Uncovering Influence Jim Dearing Center for Health Dissemination and Implementation Research Kaiser Permanente Colorado Synergy Project for Research, Practice and Transformation January 10-12, 2010, Albuquerque NM
Session Objective:To help you understand methods of data collection and measures for the assessment of perception and influence
Formative evaluation is a type of applied research that is conducted prior to the introduction of an innovation to increase the likelihood of achieving scale
Formative Evaluationof Two Types • Learning about the innovation • Learning about potential adopters
Learning about the Innovation • Is based in the assessment of perception • Whose perception most counts? • From what other types of stakeholders might we want to gather data about perception?
Learning about the Innovation • How can we learn of perceptions? • Two methods of data-collection • Interviews with open-ended response categories • Questionnaires with closed-ended response categories • Under what conditions might we prefer to collect data by interview vs. questionnaire?
Interviews are Preferable When • Interviewees are very high-ranking, few in number, when the topic is especially sensitive, or when we have reason to doubt that standard attributes will well-represent the characteristics of the innovation • Always pretest your interview protocol • Is this human subjects research? • IRB
Collection of Data via Interviews has Its Downside • Real-time jotting down of comments • 2nd person? • Digital recording • Transcription, training, coding into attribute categories, organizing of data, analysis of frequencies, reporting
A Questionnaire can be Brief • Compatibility • Cost • Simplicity • Adaptability • Effectiveness • Observability • Trialability
Simple Tallies of Responses can be Insightful • The more respondents the better • staff perception, staff portrayals, potential adopter perception • which of these three types of respondents is most important? • Use mean scores per attribute • Whether for creating attribute matrices or going straight to an attribute profile (1x7 data representation per innovation)
Tools for Assessing and Comparing Innovations • Attribute matrix, innovation profile, and potential for adoption (PAR) score are simple ways of organizing numerical data • purpose is standardization, diagnosis, improvement, comparison Dearing JW, Meyer G (1994). An exploratory tool for predicting adoption decisions. Science Communication 16(1):43-57.
Discussion • Do you have one priority innovation that you’re considering for scale-up? • what sorts of feedback have you gotten to-date about it, and from what type of stakeholders? • How do you decide who to collect formative evaluation data from? • Do you sample from the target population and if so, how?
Does All This Suggest Too Much Rigor? • Formative evaluation is usually done with small numbers of people (even questionnaires) • These means of learning about the innovation are meant to supplement other means of helping you decide which innovations are ready • You don’t need to overcome bias but you do want to understand it
Learning about Potential Adopters • Data-collection can be done in one of four ways: • informant interviews • observation • sociometric survey • self-report All for assessing social influence
What Difference Does It Make if We Work with Influentials? • Look at the degree of “reach” that people have in influence networks
A typical KP Colorado employee’s 2-step network neighborhood
An established KP Colorado employee’s 2-step network neighborhood
A KP Colorado bridging individual’s 2-step network neighborhood
Learning about Potential Adopters • Informant interviews (snowball process) • suited for large social systems • identify a few informants with broad knowledge of many others (1st round interviews) • ask their opinion of whom among potential adopters they think are looked to by others (2nd round interviews, beginning of data-collection) • 3rd round interviews, 4th round interviews, etc • stop when no new names are being generated
Learning about Potential Adopters • Observation • suited to small social systems • train knowledgeable observers to recognize the behaviors associated with social influence • what are these? • recording sheets (roster format)
Learning about Potential Adopters • Sociometric survey • who-to-whom questions • Well-suited to medium size social systems • Simple administration, simple data-entry • Wholly different form of relational data as a result • Best method for identifying and representing influence • Partner with a grad student?
Learning about Potential Adopters • Self-report survey • large social systems (marketing research) • lower inherent validity but validated instruments still available • less intrusive • more typical types of questions • Partner with a grad student?
Working Through Opinion Leaders • Does intervention with opinion leaders work to speed and spread the scale-up of innovations? • Yes, generally efficacious • Depends on how you recruit them • Depends on what you ask them to do
Recruitment & the Request • Your appeal should be normative in nature • Ask them to continue to act with the best interests of the network in mind • To evaluate the pros and cons of innovations communicated to them • To talk, to refer, to suggest, all within their own range of everyday behaviors
In Sum • I have married an outstanding woman. I owe all of my accomplishments to her – she is brilliant, patient, orderly and neat, and sexy beyond words. And as a mother, she sets the example for all professional, hard working women.
In Sum • Formative data-collection can be conducted to improve the likelihood that your innovation can reach more potential adopters, more rapidly, and produce positive perceptions among them • a range of tools exist for measurement of both personal perceptions and social influence
www.research-practice.orgjames.w.dearing@kp.orgDearing JW (2009). Applying diffusion of innovation theory to intervention development, Research on Social Work Practice 19: 503-518.