Variability of Likert scales Effects on data collection results

1. VariabilityofLikertscalesEffectsondata collectionresults Class 7 – 2011/2012 MIMED - FMUP

2. Questionnaires • Likert scale: what is it? • DifferentLikertscales • Study objective • Methods • - Studyparticipants • - Study design • Results • Conclusions

3. LikertScaleandItsVariables - A ComparativeStudy Currently, people’s opinion about an issue or service is a valuable way for researchers to acquire information in diverse areas of knowledge. • A questionnaire is an essential instrument that aims to collect this information based on the inquisition of the population in study; • It consists in a certain amount of questions covering a topic of interest to investigators; • Is useful in almost every kind of job. Questionnaries

4. LikertScaleandItsVariables - A ComparativeStudy Having in mind the need to measure and classify this information, one reaches the conclusion of the pressing necessity for scales, tools that permits measuring people’s attitudes and opinions[1]. Likert scale 1. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple treatment groups. Stat Med, 2010. 29(24): p. 2486-500.

5. LikertScaleandItsVariables - A ComparativeStudy Likertscale Whatisit? A scalethatconsistsinasking people to respond to a series of statements about a topic, in terms of the extent to which they agree or disagree with them. [2] Applications Measurepain, qualityoflife [3] 2. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple treatment groups. Stat Med, 2010. 29(24): p. 2486-500. 3.Huber JF, H.J., Zumstein MD, Ruflin G, Lüscher M., Visual circle scale (VCS)-a patient-friendly scale to measure pain compared to VAS and Likert scale. Z OrthopUnfall, 2007. 145(6): p. 795-7.

6. LikertScaleandItsVariables - A ComparativeStudy Different Likert Scales Different scales produce different results Then, different Likert scales may produce different results [4] Search: some articles were found confirming our idea 4 Dawes, J., Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 2008. 50(1): p. 61-77.

7. LikertScaleandItsVariables - A ComparativeStudy Influence of a central answer Does a neutral answer influence the results? [5] Already  studied Different Likert scales produce different results? Does a "I don't want to answer" option influence the results? Apparently not studied yet. Does both "I don't want to answer" and neutral options influence the results? Apparently not studied yet. 5. KulasJT. , S.A., Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings. Journal of Research in Personality, 2009. 43: p. 489–49

8. LikertScaleandItsVariables - A ComparativeStudy Objectives Discover if the addition of a neutral element, a “refusal to answer” option or both combined has repercussions in the results obtained. Evaluate the reproducibility of the scales used in the questionnaire. [6] 6. HanssonLM, G.M., Bergström R, Factorsaffectingreproducibilityofdietaryreportsusingfoodfrequencyquestionnaires. Eur J ClinNutr, 2000. 54(8).

9. LikertScaleandItsVariables - A ComparativeStudy Study Design: One questionnairewithtwoversions Numberofquestions in eachquestionnaire: 12 Questionnaire 1 Questionnaire 2 Same questions in bothquestionnaires Methods

10. LikertScaleandItsVariables - A ComparativeStudy • Our study is included in a Longitudinal Study type. • BECAUSE : • There is a follow-up period of the individuals in study, which can be longer or shorter; • Data is gathered at least in two points in time which allows us to observe changes in the population. Typeofstudy

11. LikertScaleandItsVariables - A ComparativeStudy StudyParticipants: All students that belong to the first year of the Integrated Master’s Degree in Medicine of Faculdade de Medicina da Universidade do Porto, distributed by classes 1 to 22 Except Students from classes 6 and 7 Methods

12. LikertScaleandItsVariables - A ComparativeStudy StudyParticipants: Questionnaire 1 These ten blocks where randomly divided in two groupswith SPSS. Questionnaire 2 Methods

13. LikertScaleandItsVariables - A ComparativeStudy Aboutthequestionnaires: • Themeofthequestionnaire: • Evaluationsystemsandstudymethods Why? Themewithlessopinionchangeprobability Methods

14. LikertScaleandItsVariables - A ComparativeStudy Aboutthequestionnaires: • Halfoftheparticipantsanswer to version 1 andtheotherhalf to version 2 • One monthlater, the first half answer to version 2 and the other half to version 1 Why? To avoid certain biases. Methods

15. LikertScaleandItsVariables - A ComparativeStudy About the questionnaires: • The questionnaires ought to be confidential. Why? How? Theparticipantswrotetheirnamesonthequestionnairesandweensuredconfidentiality . In order to receivefromtheparticipantstheirtrulyopinion Methods

16. LikertScaleandItsVariables - A ComparativeStudy About the questionnaires: • Comparisonscale: I totallyagree I disagree I totally disagree I agree • … plus a neutral answer: • … plusrefusal to answer: Neitheragreenordisagree I don’twant to answer Methods

17. LikertScaleandItsVariables - A ComparativeStudy About the questionnaires: Threetypesofcomparison Comparisonscale Vs Comparisonscaleplusrefusal to answeroption Comparisonscale Vs Comparisonscaleplus neutral andrefusal to answeroption Comparisonscale Vs Comparisonscaleplus neutral answer Comparisonscale: Likert scale with “I totally agree”, “I agree”, “I disagree” and “I totally disagree” Methods

18. LikertScaleandItsVariables - A ComparativeStudy Aboutthequestionaires: To evaluate the reproducibility Comparisonscaleplus neutral andrefusal to answeroption Vs Comparisonscaleplus neutral andrefusal to answeroption Comparisonscaleplusrefusal to answeroption Vs Comparisonscaleplusrefusal to answeroption Comparisonscale Vs Comparisonscale Comparisonscale: Likert scale with “I totally agree”, “I agree”, “I disagree” and “I totally disagree” Methods

19. Questionnaire 1 I 1. ComparisonScale 2.ComparisonScale 3.ComparisonScale + neutral answer 4.ComparisonScale 5.ComparisonScale + refusal to answer 6.ComparisonScale+ neutral answer + refusal to answer 7.ComparisonScale + refusalanswer II 1.ComparisonScale + refusal to answer 2. ComparisonScale + neutral answer + refusal to answer 3. ComparisonScale 4. ComparisonScale + neutral answer + refusal to answer 5. ComparisonScale Questionnaire 2 I 1. ComparisonScale + neutral answer 2.ComparisonScale 3.ComparisonScale 4.ComparisonScale + neutral answer 5.ComparisonScale 6.ComparisonScale+ neutral answer + refusal to answer 7.ComparisonScale II 1.ComparisonScale + refusal to answer 2. ComparisonScale 3. ComparisonScale + refusal to answer 4. ComparisonScale 5. ComparisonScale + neutral answer + refusal to answer

20. LikertScaleandItsVariables - A ComparativeStudy StudyParticipants: 250 234 158 Results

21. Reproducibility

22. Standard scale vs. Neutral optionscale

23. Standard scale vs. Refusal TO answeroptionscale

24. Standard scale vs. Both optionscombinedscale

25. LikertScaleandItsVariables - A ComparativeStudy Freedomofchoice VS Reproducibility Itdepends… Conclusion

26. LikertScaleandItsVariables - A ComparativeStudy Faculdade de Medicina da Universidade Do Porto 2º Semestre Março de 2012 João Dias José Carneiro Maria João Bastos Maria João Costa Pedro Barros Pedro Santos • Abel Trzan • Ana Fonseca • Ana Filipa Ferreira • Ana Isabel Maduro • Catarina Tinoco • Guilherme Amorim • Joana Antunes