1 / 35

Perspectival Diversity and Consensus Analysis

Perspectival Diversity and Consensus Analysis. John Gatewood . . . . . . Lehigh University John Lowe . . . . . . . Cultural Analysis Group AAA Meetings, Philadelphia, Dec 5, 2009. Preview. INTRODUCTORY REMARKS Problem of “culture-sharing” (and non-sharing)

Download Presentation

Perspectival Diversity and Consensus Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Perspectival Diversity and Consensus Analysis John Gatewood . . . . . . Lehigh University John Lowe . . . . . . . Cultural Analysis Group AAA Meetings, Philadelphia, Dec 5, 2009

  2. Preview • INTRODUCTORY REMARKS • Problem of “culture-sharing” (and non-sharing) • Basic patterns of inter-informant agreement … adding“perspectival diversity” to the list • OUR CURRENT STUDY • Assessing effects of different distributional patterns on consensus analysis’s key indicators • Some initial findings • CONCLUSIONS • Methodological “lessons” for researchers • Future directions

  3. INTRODUCTORY REMARKS

  4. Problem of “Culture-Sharing” • By definition, culture is socially transmitted knowledge; hence, it must be “shared” … but sharing is always a matter of degree • Hence, two related issues for any given cultural domain: • How much knowledge is shared? (the AVERAGE “cultural competence”) • How is the knowledge socially distributed? (the DISTRIBUTIONAL PATTERN) • KEY INSIGHT = assess degree of culture-sharing by examining patterning of inter-informant agreement

  5. Basic Patterns of Agreement • Boster (1980, 1985) … four basic patterns of agreement (paraphrasing & expanding): • UNIFORM agreement – traditional view of culture • RANDOM agreement – free variation → no culture • EXPERTISE gradient – experts tend to agree with one another whereas non-experts deviate randomly • SUBCULTURAL variation – more than one ‘school of thought’ • Competing answer sets – different groups, different truths • Complementary knowledge – different groups systematically know different things • Romney, Weller & Batchelder (1986) … cultural consensus theory & consensus analysis

  6. Perspectival Diversity … a 5th Pattern • Pilot study of credit union employees (Gatewood & Lowe 2006) No consensus in sample * + No identifiable subcultural groups => Fish-scale overlappings of partial knowledge … “perspectival diversity” i.e., social interaction and knowledge among the employees was rather departmentalized … their understandings of ‘credit unions’ reflected what they needed to know to perform their own jobs, not necessarily what might be relevant to other people _______________________ * Pilot study’s conclusion about “no consensus” turned out to be an artifact of our failure to counter-balance items in the questionnaire form (see Gatewood & Lowe 2008) … but that’s another story

  7. To generalize, perspectival diversity occurs when… • All individuals have limited knowledge with respect to a given domain and to approximately the same degree • Each individual’s range of knowledge only partially overlaps with the ranges known by others And, consistent with this definition, different geometries of perspectival diversity are possible …e.g., circular pattern, linear pattern, taxonomic-hierarchical, overlapping polygons on a surface, etc.

  8. OUR CURRENT STUDY

  9. RESEARCH QUESTION: • Ceteris paribus, do the different distributional patterns affect the key indicators of consensus analysis? • As the average knowledge in a sample varies, do different distributional patterns “show” consensus more readily than other patterns? • Do some distributional patterns “mask” cultural consensus when other patterns “reveal” it? • If NO  … nothing to worry about [ yippee! ] • If YES  distributional patterning has an independent effect that needs to be taken into account when interpreting results of consensus analyses

  10. ITEM FORMAT: • Counter-balanced Likert-style questions, i.e., 6-point “strongly agree” to “strongly disagree” response scale … because these are so common in survey research ANALYSES: • Such data can be analyzed two ways: • INFORMAL MODEL of consensus analysis … i.e., input to factor analysis is a Resp x Resp correlation matrix (data treated as interval-scale) • FORMAL MODEL of consensus analysis … i.e., input to factor analysis is a chance-corrected agreement matrix (data treated as nominal-scale, e.g., dichotomized responses)

  11. Research Design … with average knowledge and distributional pattern as manipulated variables

  12. “Theoretical” Predictions

  13. Implementation • How to “experimentally manipulate” key parameters for different distributional models while holding others constant ?? … computer simulation to the rescue ! See: Excel “data-generating” file& Excel “findings” file

  14. SOME INITIAL FINDINGS

  15. Key Findings • Distributional pattern has an independent effect with respect to consensus indicators • w/r/to RATIO OF EIGENVALUES ( compared to the Uniform-to-Random model ) • Expertise patterns INCREASE the ratio • Subcultural patterns DECREASE the ratio • Perspectival patterns DECREASE the ratio • w/r/to MEAN 1st FACTOR LOADING • Distributional patterns have little effect on this indicator, AND consensus analysis estimates actual competence very well … with one exception: • Expertise (triangular) pattern INFLATES mean competence as well as the ratio of eigenvalues … (because it violates the “homogeneity of items” assumption?)

  16. Expertise (rectangular) pattern • The range of expertise about the same average competence also makes a difference: greater range  larger ratio of eigenvalues • Subcultural patterns • As expected, systematic differences in sub-group knowledge undermine consensus: • “By question” sub-groups may still show consensus overall, with the groups showing up on the 2nd factor • Different “answer keys” just destroy consensus • “Formal consensus model” (on dichotomized data) and “informal consensus model” yield very similar results

  17. CONCLUSIONS

  18. “Lessons” for Researchers • Since the ratio of eigenvalues is particularly sensitive to the distributional pattern of knowledge, REPORT MORE than just the ratio • Minimally, include: • Ratio of 1st to 2nd eigenvalues • Mean 1st factor loading (and st.dev. of those loadings) • Number of negative loadings • And, comparable “guidelines” should be established for evaluating these additional measures: e.g., 0.500 for mean loading; fewer than ~5% negative loadings in sample • These output statistics are necessary for more meaningful interpretations of one’s data

  19. IF your data show a hefty mean 1st factor loading but a low ratio of eigenvalues… DO NOT leap to the conclusion that either (a) subcultures exist or (b) there is free variation in the domain • You may be dealing with a case of PERSPECTIVAL DIVERSITY … which would warrant further investigation, such as examining the inter-person correlation matrix and the response-profiles of individuals one at a time to see if you can detect a subtle social patterning to who-knows-what

  20. Try to formulate questions that are “EQUALLY DIFFICULT” ( and ask lots of questions ) • Violations of Assumption 3 will inflate both the obtained ratio of eigenvalues & the mean 1st factor loading • e.g., Expertise (triangular) pattern INFLATES both indicators • So … ex post facto…if you notice that some questions were “much easier” than others, then either: (a) use higher threshold criteria before claiming the data conform to the cultural consensus model, and/or (b) remove the very easy questions and re-analyze

  21. Future Directions • Developing additional ‘geometries’ of perspectival overlapping • Analyzing relations between a variety of measures describing the initial Resp x Resp correlation matrix and the key indicators from consensus analysis • Exploring different instantiations of “guessing” (binomial, truncated-normal, beta distributions) • Exploring other possible measures from the factor analysis as predictors of culture-sharing, e.g., 1st eigenvalue divided by sample size

  22. Thank you … and we would be happy to continuetalking with interested folksafter the session

  23. Scalar data analyzed via Informal Method (vertical axis)VSDichotomized data analyzed via Formal Method (horizontal axis) Mean 1st factor loadings( r = .969 ) Ratios of eigenvalues( r = .933 )

  24. “Real” Survey Item “Real” Survey Item Simulated Item POTENTIAL PROBLEM… Frequency distributions of itemsfrom “real” surveys (top panels)are more graded than our“simulated” data (lower right) something we’re trying to resolve,but not there yet …

More Related