University of South Australia

1. University of South Australia The Load Forecasting Dilemma – Factors influencing progression rates at Higher Education Institutions before and after the Bradley review. Andrea Matulick, Manager: Cognos 8 Migration Project, UniSA

2. Outline of presentation • Student Load forecasting overview • Predicting continuing student progression rate • CGS continuing load – major influencing factors • Why progression rates may vary • The future – what changes can we expect after the Bradley reforms

3. Student Load Forecasting Overview • Purpose of Load Forecasting: • Meeting DEEWR (or govt) Funding agreement arrangements • Budget allocations to academic areas • Determining tertiary admission centre intakes • Location and Level of Load Forecasting: • Mostly in planning, strategic planning or planning and quality offices • Mostly by staff at level HEO8 and above • (AAIR Load Management SIG Questionnaire 2009)

4. Student Load Forecasting Overview cont. • Formula to forecast total load for the next year – 2 step process: • Forecast the expected continuing load (using historic evidence based data) • Plan the correct commencing load to give the required total load (required to meet funding targets, strategic plans etc.)

5. Student Load Forecasting Overview cont. Factors making up each part of the total load equation: Planned Commencing Load = Number of offers * estimated Acceptance Rate* estimated EFTSL per Person Forecast Continuing Load = Previous Years Total Load * estimated Progression Rate WhereProgression Rate = current year continuing load / previous year total load

6. Student Load Forecasting Overview cont • Accuracy of Load Forecasting: • the more accurate the prediction of continuing load, the easier it is to plan for the commencing load required to achieve the desired total load. • it becomes difficult to predict and control load if the commencing load needs to vary significantly to cater for inaccuracy in forecasting continuing load or controlling commencing load

7. Student Load Forecasting Overview cont

8. Predicting Continuing Student Progression Rate • Progression Rate = current year continuing load / previous year total load • The analysis: • Look at how actual progression rates varied over the years • Use 2 different methods to predict a progression rate • Look at how accurate the predictions would have been • Look for reasons why progression rate may not be as predicted • The data used: • DEEWR aggregate student load data from 2004 to 2007 for 38 Australian Tertiary institutions with valid progression rates (PR from 0.1 to 1.0 in all 4 years)

9. Progression Rates for most institutions Predicting Continuing Student Progression Rate cont.

10. Progression Rates for 11 tagged institutions: Predicting 1ate cont.

11. Predicting Continuing Student Progression Rate cont. • Variation of actual student load progression rates over the 4 year period: • The average actual progression rate over all institutions was 63% • The minimum variation in actual progression rate for any institution was 0.88% • The maximum variation in actual progression rate for any institution was 15.59% • 71% of institutions had a progression rate that varied by less than 5% • 11% of institutions had variations of more than 10% • How accurately can we forecast progression rate?

12. Predicting Continuing Student Progression Rate cont. Comparison of accuracy using 2 different methods of forecasting: Method 1: Estimated Progression Rate is a 3 year average from the prior 3 years. Method 2: Estimated Progression Rate is set to the same as the most recent year.

13. Predicting Continuing Student Progression Rate cont. Actual Variation and Forecast Accuracy of progression rates over 4 years: • for 3 of the 4 funding groups, more universities would have achieved a better continuing load prediction using the 1YR method than the 3YR average • 84% of institutions could have made predictions of continuing progression rate to within 5% of the actual progression rate by using either of the two forecasting methods

14. Predicting Continuing Student Progression Rate cont.

15. Predicting Continuing Student Progression Rate cont. • Points to note so far: • Variation in actual progression rate • For most universities progression rate is fairly constant and estimating a value to use in their forecast continuing load is not difficult. For about 10% of institutions progression rate does vary significantly over years which may cause inaccuracies in forecasting continuing load. • Accuracy of 2 methods for forecasting progression rate • More universities would have achieved a better continuing load prediction using the 1YR method than the 3YR average. However, 84% of institutions could have made predictions of continuing progression rate to within 5% of the actual progression rate by using either of the two forecasting methods. • What are the influencing factors for variation and accuracy?

16. CGS Continuing Load – influencing factors • Use the Domestic UG load group as proxy because: • Large, relatively stable group that equates closely to CGS load • Expect less variation in commencing and continuing load • 11 universities with variation on actual PR > 5% over 4 year period (tagged) • Look atthese 11 universities to find out whichfactorsmayberelated to the largerthanusual variation in continuingstudent progression rate • How wellcouldthey have predictedtheircontinuingstudent progression rate and load? • Whichforecastingmethodwould have given the the best results?

17. CGS Continuing Load – influencing factors cont. 11 Unis with Dom UG PR varying >5% in 4 year period (tagged):

18. Do Total Load, State and Alliance Group relate to variation in progression rate?

19. CGS Continuing Load – influencing factors cont. • A closer look at why progression rate may vary: • Variation in commencing load • 6 of the 11 tagged institutions were in the top 8 institutions for variation in commencing load as a percent of most recent total load. All of the 6 had a variation of more than 22% in commencing load over the previous 4 years. • Difficulty in forecasting continuing load • 5 of the 11 tagged institutions were in the top 6 institutions being least able to predict their most recent year progression rate. All of the 5 could not have predicted their progression rate to within less than 2.2% for the most recent year using either of the 2 forecasting methods.

20. CGS Continuing Load – influencing factors cont. 10 Unis with largest variation in commencing load over 4 years:

21. CGS Continuing Load – influencing factors cont. 10 Unis with least accuracy in forecasting continuing load:

22. CGS Continuing Load – influencing factors cont. • More points to note: • Variation in commencing load • Large variations in commencing load as a percent of total load is a good indication that progression rate will also vary. Both of these factors mean it will be difficult to accurately predict and manage load. However, some institutions with large variations in commencing load and progression rate should not have difficulty predicting their continuing and total load • Accuracy in forecasting progression rate • Large variations in progression rate and commencing load are common factors in being unable to accurately predict and manage load. However they are not always present which means that in some cases other factors are more likely to be contributing – e.g. completion rate, major changes to programs or environmental factors

23. The future • What to look for - Changes in metrics • Large variations in progression rate over time (more than 5%) • Large variation in commencing load over a short period of time (more than 20% of total load) • Smaller institutions or those from regional areas • Plan to react to changes (expected or unexpected) by understanding the load modelling process used at the institution.

24. The future • What to expect - Changes in methods • Load forecasting will increase in strategic importance and become more integrated with other planning processes such as workforce and infrastructure • More universities will use sophisticated BI software to standardise and automate the load planning process • Load forecasting should be actively managed and reviewed as a key strategic process Questions?