Practical Forecasting for Juvenile Justice Agencies

1. Practical Forecasting for Juvenile Justice Agencies Sensible Methods for Anticipating Future Demands for Confinement Space in the Juvenile Justice System Jeffrey A. Butts, Ph.D.Daniel P. Mears, Ph.D.July 2002 A product from the Urban Institute’s Program on Youth Justice

2. Overview At some point, all youth justice agencies are asked to estimate the potential future demand for their services, whether they provide detention and correctional space, or treatment and community supervision. The techniques used to anticipate future demand are generally known as forecasting. When forecasting is viewed as a statistical exercise, the responsibility for generating forecasts will likely be delegated to researchers and technical staff. Yet, elected officials and administrators are accountable for the policy and budgeting decisions that depend on forecasting, and they have unique access to information about future trends in policy and practice. Information about policy and practice is critical for estimating future demands for confinement space and other programs. Even the best statistical models will never fully account for the complex forces that actually shape demand for space in the juvenile justice system. FORECASTS ARE MORE REALISTIC WHEN THEY INCLUDE THE VIEWS AND OPINIONS OF DECISION MAKERS AND MANAGERS. • The Urban Institute developed its “Practical Forecasting” web site to encourage policymakers and professionals in the juvenile justice system to participate more actively in forecasting and to depend less on analysts and consultants. • The various forecasting products available on the Urban Institute’s web site (http://youth.urban.org) were designed to help state and local agencies develop their own forecasting procedures and to ensure that those procedures are: • policy-oriented, • widely accessible, and • practical. The Urban Institute’s “practical forecasting” materials were prepared under grant number 98-JB-VX-K004 from the Office of Juvenile Justice and Delinquency Prevention (OJJDP), U.S. Department of Justice.

3. Basics • Forecasting is not the same as predicting-- forecasting is a systematic way to compare our previous predictions with actual developments, and then learn from the differences • Data analysis may be the easiest part of forecasting • A good forecasting process is 50% statistics, 50% human judgment • The Urban Institute’s “Practical Forecasting” web site makes the statistical part of forecasting easier • Agencies using the web site can focus their efforts on improving the judgment side of forecasting

4. Challenges in Bed Space Planning • “Need” for space is not an objective concept-- perceived need depends on the current mix of existing resources and how agency officials and policymakers view that mix • Too many agencies focus on accuracy -- all forecasts are wrong to some degree, but what did we learn?-- in forecasting, accuracy is less important than utility-- a statistical model is only as good as its weakest assumption • Agencies rely too much on outside consultants for forecasting and the responsibility for forecasting is too often delegated to technical staff • Policymakers and management must get actively involved and lead forecasting efforts

5. Better Methods of Forecasting A good forecasting process: • Relies on people first • Includes a diverse group of relevant stakeholders • Is parsimonious, focused on key factors • Is not constrained by data availability-- using an estimate is better than omitting an important factor • Focuses on identifying / understanding departures from expected trends • Is an ongoing effort, not a one-time exercise

6. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts Forecasting efforts are likely to take many twists and turns. Each jurisdiction needs to develop its own preferred method, its own way of running meetings and assigning tasks to individuals and agencies. In general, however, a practical, policy-oriented forecasting process should include at least these seven steps. < click on each step to review process

7. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 1. Identify stakeholders and set agenda • Stakeholder group is assembled through a consensus process (members of the group should bring a mix of authority & expertise to the table) • Stakeholders meet & set overall agenda • One agency may be chosen to coordinate the forecasting process, but the process itself must be owned by the group at large • Researchers selected to support forecasting process should respond directly to stakeholder group and not to one agency only or to one member of the stakeholder group

8. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts Possible members for stakeholder group • Detention & corrections agencies • Other juvenile justice practitioners (probation supervisors, prosecutors, defense attorneys) • Judges • Law enforcement • Governor/executive office • Legislators and legislative analysts • Child welfare and other social services • Mental health providers • Private youth services agencies • Community representatives • Outside researchers • Experts in programming and treatment options

9. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 2. Convene initial stakeholder meeting • In their first meeting, stakeholders focus on creating a shared framework for understanding the role of forecasting in agency planning, including its limitations • In group discussion, members identify the full range of factors thought to influence the use of confinement and how policy and practice may influence the demand for beds • Before adjourning, stakeholder group agrees on common agenda and the expected benefits of forecasting

10. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts Possible factors that could affect demand • Local crime rate and juvenile arrest rates • Availability of non-secure supervision options • Local sanctioning practices, placement preferences • Who controls admissions and releases from secure care? • Reimbursement incentives (e.g., state/local mix) • Reputation and availability of local service providers • Any other policies regarding the dispositions employed for young offenders

11. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 3. Assess data needs and organize information • Researchers organize available data on arrests, court filings, court processing, detention/corrections, etc. • Researchers take direction from the stakeholder group in searching for and analyzing data • Lack of data may be noted, but the scope of forecasting is not reduced to accommodate data problems • Researchers generate DRAFT forecast scenarios for various offender populations and program types

12. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts Type of data needed for forecasting offender groups Groups 1 & 2 (males committed on felony charges, by age)

13. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 4. Stakeholders meet to create forecasts • Stakeholders re-convene to review draft forecasts and work together to revise and refine projection scenarios • Data shortcomings are identified • Stakeholders create estimates for any missing data elements, noting those where group opinions may vary • Staff are assigned to obtain further data if possible, and to prepare revised versions of all critical forecast scenarios

14. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 5. Stakeholders meet again to approve forecasts • Following revision of forecasts by research staff, stakeholders meet to review forecast scenarios and make final changes when necessary • Stakeholders agree on final set of official forecasts and clarify the assumptions underlying each forecast scenario • Data shortcomings are noted, as are any assumptions that were controversial or that involved strongly divergent opinions among the stakeholder group

15. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 6. Lead agency disseminates official forecasts • Stakeholder group releases official forecasts, identifying critical assumptions and sources for each (stakeholder consensus, empirical analysis, etc.) • Forecasts are disseminated by lead agency in the forecasting effort, but the stakeholder group is clearly identified as author • Focus is on electronic dissemination and forecasts are distributed to relevant groups, agencies, & public officials

16. 1. Identify stakeholders 2. Hold first meeting 3. Assess data needs 4. Create forecasts 5. Review forecasts 6. Publish forecasts 7. Repeat process 7 Steps to Credible and Useful Forecasts 7. Process is repeated frequently • Stakeholders re-convene to repeat process (every 6 to 12 months for correctional populations, perhaps more often for detention or other programs with short lengths of stay) • On second and subsequent forecasting rounds, process begins with a critical review of previous forecasts • Stakeholder group identifies extent to which actual populations deviated from forecasts, and reasons for those departures are discussed and disseminated with the round of next official forecasts DONE

17. Summary of the Urban Institute’s Practical Forecasting Model Hold first meeting of stakeholders; review forecasting basics; plan meetings and design process Assemble available data and prepare summary information about key offender groups Compare previous forecasts with current conditions and set goals for next forecasting effort Convene stakeholders and use online forecasting tool to consider various forecast scenarios Repeat Every 6 to 12 Months Agree on most likely forecast scenarios and disseminate results to a broader audience for review and comment

18. Key Points to Remember • The process of generating and using forecasts may be as important as the content of forecasts • An effective forecasting process is not necessarily expensive and does not have to require a large investment of agency resources and personnel • Nominal improvements in forecasting may produce considerable improvements in the efficiency and effectiveness of detention and correctional planning • The most important improvement an agency can make in its forecasting process is usually not to increase its statistical sophistication, but to increase the number and diversity of key stakeholders involved directly in forecasting

19. The Program on Youth Justice URBAN INSTITUTE Justice Policy Center 2100 M Street, NW Washington, DC 20037 Practical Forecasting Web Site http://youth.urban.org