110 likes | 255 Views
ABTA Institute National 2009 Conference on Innovations in Government Accountability & Performance 5/18/2009 Melbourne, FL. Government Accountability – A State Level Perspective.
E N D
ABTA InstituteNational 2009 Conference onInnovations in Government Accountability & Performance5/18/2009Melbourne, FL Government Accountability – A State Level Perspective
John L. Lewis Research Manager Bureau of Research & Data Analysis Florida Department of Corrections • Our Bureau compiles most non-budget numbers or data associated with the operations of the Florida Department of Corrections. • Our section compiles output and outcome data related to inmate programs such as GED counts and ‘recidivism rates’ for various groups of inmates. • ‘Recidivism Rates’ are the standard by which most large correctional programs are judged (A way that the Florida Department of Corrections is ‘accountable’ for the dollars it spends on inmate programs)
Two Keys in Measurement • ‘Standardized’ definitions (What) – are needed in ABTA & all measurement efforts • A good corrections example of how difficult this can be is in the measurement of ‘recidivism rates’ • ‘Practicality’ versus ‘Power’ of the Measurement Tool (How Well) – a key consideration in ABTA & other measurement systems • The University of Maryland’s ‘5-Level Evaluation Scale’ presents a good example – later, time permitting, we can discuss in detail
Does a document truly reflect ‘accountability’? How good is an ‘evaluation’? The University of Maryland’s (UM) ‘5-Level Evaluation Rating Scale’ addresses these questions. Levels range from 1 – 5 With Level 5 involving the highest level of confidence/complexity, but such accountability may not be practical (may take too much time, effort, and resources for ‘smaller’ or relatively unimportant, minor programs).
Level 1 Accountability/Evaluation • Involves data or information where no comparison is used • Instead, the relationship between a program & an outcome, i.e., recidivism, is analyzed • This level of accountability is used in many agencies for program ‘performance measures’
Level 2 Accountability/Evaluation • Involves data or information that compares two or more groups -- one receiving and one not receiving the treatment/program • However, one group lacks comparability to the other • This is also frequently used in program ‘performance measures’ or reports
Level 3 Accountability/Evaluation • Involves a comparison between two or more groups, one that receives and one that does not receive the treatment/program • Same as Level 2, except groups are comparable • Level 3 reports often employ less rigorous statistical control procedures
Level 4 Accountability/Evaluation • Requires an analysis of comparable treatment and comparison groups, controlling with rigorous statistical methods for factors other than participation that may influence outcomes • A Level 4 report may also have a random assignment design that had problems in implementation
Level 5 Accountability/Evaluation • Involves an evaluation with well-implemented random assignment to a treatment group and a control group that does not receive the treatment/program. • The most confidence can be placed in ‘Level 5’ data/information • Of course, it’s also more complex/costly/time consuming and may not be practical • As we evaluate programs, many factors come into play to determine the applicable level of measurement to use
In Closing • Questions? • Comments? • Suggestions? Thanks