70 likes | 170 Views
Quality and purpose: quality in quantitative methods. Angela Dale University of Manchester. Quality in survey methods. A survey is of adequate quality if it can answer specified question with an adequate degree of accuracy it is fit for purpose
E N D
Quality and purpose: quality in quantitative methods Angela Dale University of Manchester
Quality in survey methods • A survey is of adequate quality if it can answer specified question with an adequate degree of accuracy • it is fit for purpose • If it can provide an answer at a 1% level of confidence when only 5% is required, then it may not be cost effective
Standards for surveys • Random sample – best bet of avoiding bias in sample; but need adequate sampling fame • Adequate numbers for required subgroups – depends on level of accuracy needed • Good response rate – but extent of bias is crucial • Questionnaire must be well developed • well designed questions; good flow; salient
Can a scoring system work? • A scoring system can indicate overall strength on all these dimensions, but: • Higher quality costs more • Increased spend may have diminishing returns • But there will be a level below which a survey would have very low value/low credibility • Need to find a way of assessing quality against requirement
Analysis issues • Results based on analysis need to reflect the quality of the survey • Importance of making clear the level of accuracy of the results • The quality of the analysis is also of great importance • An excellent survey may be badly analysed , • methods may be used poorly; interpretations incorrect • inappropriate assumptions about causality may be made • Claims may go beyond what the data can support
Can a poor survey have value? • A very poor survey may still have value: • if analysed with care and weaknesses recognised • If only very limited conclusions, that can be justified, are drawn • Where there is no better alternative
Conclusions • It is of value to have a reference framework for assessing quality • But equally important to use it critically and with care • The key to quality lies not just in good data but in ensuring that claims made can be supported by the data