1 / 37

Common complications when analysing survey data

Common complications when analysing survey data. Module I3 Sessions 14 to 16. Objectives of these three sessions. You should be able to: Explain why weights are sometimes needed in analysing survey data Produce weighted tables of counts and other statistics

holmes-franks
Download Presentation

Common complications when analysing survey data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Common complicationswhen analysing survey data Module I3 Sessions 14 to 16

  2. Objectives of these three sessions You should be able to: • Explain why weights • are sometimes needed in analysing survey data • Produce weighted tables • of counts and other statistics • Suggest ways of adjusting analyses • when there are missing values • Analyse multiple response data • Cope with data containing zero values

  3. Contents • Review • Why these sessions? • There can be zero values • That may have to be analysed separately • Multiple responses are common • and are an example of data at multiple levels • Weights are often needed • Because observations represent different fractions of the population • Missing values can distort an analysis • Simple options are explored

  4. Review • Describe data well • can use Excel, or a statistics package • we repeat briefly, with a statistics package • Real data sets introduce surprises in analysis • That are not present with artificial training exercises • They need practice during training courses • Or they will be a problem to analyse later • But some complications are predictable • And very common • Like multiple response questions, or the need for weights • These are the complications we cover here

  5. How to describe data well – (repeat slide) • Look for oddities in the data • and be prepared to adapt the summaries that you calculate • Study the data as tables and graphs • Use frequencies and percentages • to summarize categorical variables • Use averages and measures of variability • to summarize numeric variables • Identify any structure in the data • and use it in producing your summaries

  6. Look at the data (repeat slide) The 2 types of variable are summarized in different ways

  7. Analysis to meet objectives (repeat slide) Simple objectives Not so simple objectives

  8. Meeting simple objectives (repeat slide) These summaries were made with Instat – see practical 1

  9. Answering more complicated objectives These were also with Instat AND explaining some of the variability

  10. Practicals 1 and 2 • Practical 1 • Reviews the construction of tables • Using a statistics package • Particularly to look at percentages • Because percentages have to be understood clearly • to analyse multiple response data • Practical 2 • Looks at the analysis of data containing zeros • And shows that calculating averages needs to be done carefully, when there is structure in the data • Both practicals give more practice • In the use of a statistics package

  11. Zero values • Zeros may be are a simple part of the data • For example: List the assets – radio, bicycle, etc • Some may have zero assets • Often however zero is a special value • And should be analysed in a special way • Examples: • How many livestock do you have? • What was your yield of maize? • How much rain fell yesterday? • What is different here?

  12. Obs. Value 3 8 0 0 5 6 0 7 0 1 Possible analysis Total = 30 n = 10 mean = 3 median = 2 etc Example This does nothing special The zeros are analysed with all the other values

  13. Obs. Value 3 8 0 0 5 6 0 7 0 1 Alternative analysis Total = 30 n = 10 number of zeros = 4 proportion of zeros = 0.4 (40%) n = 6 are non-zero mean = 5 of the non-zero values median = 5.5 etc Example continued

  14. Which is better? • As usual both are valid • It depends on the precise objective • And on the type of data • Often the 2-step analysis is appropriate • The data are split into 2 • For example: Do you have cattle? • Then (if you do) how many do you have? • Analysis • 60% of farmers owned cattle • Among the cattle owners, the mean was 5 per household

  15. Multiple response questions? These are NOT multiple responses because the question asks for the main source From Tanzania agricultural survey Ask for ALL sources used to make it multiple response

  16. Multiple responses? Not multiple response Multiple response You may own more than 1 asset

  17. Livestock survey examples

  18. For individual species it is easy What % keep cattle? What % keep sheep? Nothing special needed Looking at all species together Needs thought what % keep livestock does livestock keeping depend on type of household Analysis of multiple responses

  19. Practicals 3 and 4 • Multiple response analysis • Using a simple example • With three different layouts of the data • Then some real examples! • Using data from the Tanzania agriculture survey

  20. Introducing weights • Suppose a sample of 2 farmers • Farmer Yield A 1 t/ha B 2 t/ha • What is the mean? • Obviously it is (1 + 2)/2 = 1.5 t/ha! • But…

  21. Introducing weights - continued • Suppose a sample of 2 farmers • Farmer Area Yield Production A 5 ha 1 t/ha 5 tons B 0.5 ha 2 t/ha 1 ton • Now what is the mean? • It could still be (1 + 2)/2 = 1.5 t/ha • Or it could be (5 + 1)/5.5 = 1.1 t/ha

  22. But which is right? • They are both right, • but they answer different questions • Take food security • Are you interested in the farmer • Or the production • Or both • If the farmer is the unit of interest • Then there are 2 farmers • The mean is 1.5 • If the area is the unit of interest • Then there are 5.5 ha • And Farmer A is 10 times as important as farmer B • So a weighted mean is produced

  23. The weighted mean • So if the area is of interest – then with • Farmer Area Yield A 5 ha 1 t/ha B 0.5 ha 2 t/ha • Weight each yield by the area it represents • mean = (1*5 + 2*0.5)/5.5 = 1.1 • Here the areas are the “weights” • They are used when different observations • represent different proportions of the “population”

  24. Weights in the Tanzania agriculture survey The number of people in the population represented by each observation It was roughly a 1% sample, so the weights are about 100 The technical guide explains the calculations

  25. Practical 5 • Weights using a statistics package • First the rice survey • Weighting by the size of field • Then the Tanzania agriculture survey • Investigate ownership of radios • By sex of household head • And then by type of farming household

  26. Possession of radio by type of farming Unweighted analysis The observed numbers and percentages in the sample Look at livestock – but numbers small

  27. Possession of radio by type of farming Weighted analysis The estimated numbers and percentages in the region of Tanzania Look at livestock now – what do you conclude?

  28. Why such a large change with weighting? Examine the weights for these 2 groups Average weight = 60 Average weight = 20 So estimated % with radio = 100*(42*20)/(10*60+42*20) = 59%

  29. And always take care with small numbers Large sample overall But still a small sample of livestock-only farmers

  30. Missing values Survey of countries on principles of official statistics Non-response is one form of missing value Here 82 of the 194 countries did not respond

  31. More missing values This “non-response” is missing responses to questions within the 112 who responded overall

  32. Practical 6: Non-response and missing values • The data on the principles of official statistics • are re-analysed in a new way • Which adjusts for the missing values • The countries who did not respond • Then the missing values are considered • Within the responses that were available

  33. Coping with missing values • They should be stated in the reporting • Which they were in the report on the principles • Can they be ignored? • Often the missing values are simply ignored • The analysis of the principles ignores them • If their absence is uninformative • Then ignoring them is usually OK • Otherwise you could look to compensate • We show one way here • By using a weighted analysis • The main message is to think carefully • Don’t be quick to let the computer “impute” values

  34. Non-response in the Principles survey • The adjustment may present a fairer picture • Of the 194 countries • But it adds a worrying component • Would it be better to present the results separately • For each type of country? • And the 15 countries from the “Least Developed” group • Have a large weight • To compensate for those that are missing

  35. Missing values within the data • There are also a few missing values • For example Principle 4 has only 11 responses • Here there is much more information • From the other responses from this country • Possible actions are: • Do nothing • That was how the results that were reported • There are so few missing • Any adjustment will make very little difference • Change the weights • For the questions with missing values • Impute missing values • Simply, or using special software

  36. Can you now? • Cope with data containing zero values • Explain why weights • are sometimes needed in analysing survey data • Produce weighted tables • of counts and other statistics • Suggest ways of adjusting analyses • when there are missing values • Analyse multiple response data

  37. The next sessions are to practice in groups all you have covered here so far

More Related