1 / 41

Standard errors from audits

This study explores different methods for estimating total error in student loan audit data and suggests improvements for better accuracy and efficiency.

bremner
Download Presentation

Standard errors from audits

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Standard errors from audits Sumit Rahman, Department for Business, Energy and Industrial Strategy

  2. Student Loans Company (England) • 4 types of loan (tuition fee loans, maintenance loans, maintenance grants, part-time grants), and two different academic years falling in the same financial year – so 8 strata • Simple random sample of loans from each stratum (different sampling rates) • Administrative data with no non-response

  3. Estimating total error Step 1: estimate the stratum error rate

  4. Estimating total error Step 2: estimate the stratum error total

  5. Estimating total error Step 3: estimate the error total Step 4 (optional): estimate the overall error rate

  6. Estimating total error In one step

  7. Estimating total error In one step Step 1: estimate the stratum error rate

  8. Estimating total error In one step Step 1: estimate the stratum error rate Step 2: estimate the stratum error total

  9. Estimating total error In one step Step 3: estimate the error total Step 1: estimate the stratum error rate Step 2: estimate the stratum error total

  10. Estimating total error In one step

  11. Estimating total error In one step

  12. Standard errors

  13. Standard errors

  14. Standard errors • So our estimate of total error was £115m • And the standard error estimate is £31m • That’s really high – the coefficient of variation is 27% • A 95% confidence interval is £115m +/- £61m • Or £115m +/- 53%

  15. When is the ratio estimator better?

  16. When is the ratio estimator better?

  17. Improving the sample allocation

  18. Improving the sample allocation

  19. Improving the sample allocation

  20. Skills Funding Agency (England)

  21. Skills Funding Agency (England) census

  22. Skills Funding Agency (England) Separate ratio census

  23. Skills Funding Agency (England)

  24. Two-stage cluster sampling

  25. Two-stage cluster sampling

  26. Two-stage cluster sampling

  27. Two-stage cluster sampling

  28. Estimating the total error Step 1: estimate the college error rate

  29. Estimating the total error Step 2: estimate the stratum error rate

  30. Estimating the total error Step 3: estimate the stratum error total

  31. Estimating the total error Step 4: estimate the error total Step 5 (optional): estimate the overall error rate

  32. Estimating the total error In one step Step 1: college error rate

  33. Estimating the total error In one step Step 1: college error rate Step 2: stratum error rate

  34. Estimating the total error In one step Step 1: college error rate Step 2: stratum error rate Step 3: stratum error total

  35. Estimating the total error In one step Step 1: college error rate Step 4: error total Step 2: stratum error rate Step 3: stratum error total

  36. Using sampling weights

  37. Using sampling weights

  38. Using sampling weights

  39. Using sampling weights

  40. Using sampling weights

  41. Improvements to consider • Didn’t make use of ratio residuals. After the transformation I used the default formulas for the standard error in a clustered design – but using expansion residuals • In this set-up, PPS sampling is sensible. We could probably do this using funding as our measure of size rather than number of colleges in a stratum and number of learners in a college • Model based estimation?

More Related