210 likes | 359 Views
using evidence-based outcomes to justify public sector programs—a practitioner model Hard Stuff Made Easy…Using a Lean Sigma Approach. The Six Sigma Approach. Six Sigma. LEAN Six Sigma. Focus on Waste Removal Rework Causes of Defect Not as Statistical Detailed
E N D
using evidence-based outcomes to justify public sector programs—a practitioner modelHard Stuff Made Easy…Using a Lean Sigma Approach
The Six Sigma Approach Six Sigma LEAN Six Sigma Focus on Waste Removal Rework Causes of Defect Not as Statistical Detailed Utilized More-so outside manufacturing Education Transportation Healthcare Financial Institutions • Business Management Strategy1 • Seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and minimizing variability in processes; • Uses a set of quality management methods • Statistical Methods, • Key Goal(s) = Cost Reduction and/or Profit Increase 1Generally Speaking
why do it? Funding Programmatic (policy / practice change) Narrow the Gap of Success Improve Outcomes Communicate Outcomes Increase Dialogue with Stakeholders Communicate Data Driven Decisions Best Practices Detect Patterns Control Process Create Competitive Advantage • Economic Crisis • Political Environment • Societal Demands MINDSET
General theme /concerns from those in the trenches about program evaluations • Unnecessarily Diverts Resources • Generally (10-20%) of the Program Budget to Conduct and Process Outcome Evaluation; what is gained will enable streamlining resources and focus on “what works” • Produce Negative Results • “what does not work” is as important as “what does work” • Too Complicated • Create Additional Burden(s) • Another Form of Program Monitoring
Benefits At All Levels[Federal/Local/Institutional/Individual] • Understanding & Accountability • Demonstrate Program Outcomes • Evidence of Effectiveness & Efficiency • Documentation for Reports • Foundation for Strategic Planning • Accountability, Flexibility, Sustainability
Essential Criteria to Effectively Argue for the Existence of Program/Funding? answer these questions? Can You Provide Evidence of Your Answers? • What do you do? • Why do you do it? • Have you been effective at doing what you do? • Do you measure what you do? • Have you improved what you do? • What did or did not work? • What have you changed? Why? • Are you best practice? • Who has benefited from what you do? How do you know? • If you didn’t do what you do, what would be the impact?
Traditional Methodology Mission Retain, Graduate, Enroll in Post-Secondary The “Black Box” Input is Converted into Output Inputs Outputs No Longer Good Enough! Annual Performance Report (APR)
Mind The “GAP”What Can and Cannot BE Controlled • Variation • Man • Machine • Method • Measurement • Materials • Mother Nature
Five-year End of Cycle Evaluation FTF Comparison of ULM Talent Search Graduates to Louisiana Public High School Graduates, by Percent 82% First Generation 91% Low Income
Four-Year Graduation Rates of ULM-ETS Target Schools National Average = 73.9% State Average = 65.9% Source: Louisiana Department of Education, District Composite Reports, revised January 2010
92% of Senior (College Ready) Participants Will Graduate Each Year 92%
70% of Senior (College Ready) Participants Will Apply for Post-Secondary Admission During the Budget Period 70%
60% of All Senior (College Ready) Participants Will Enroll in College 60%
PARETOULM Impact on Federal Mandatory Objectives Objective 1: 90% of Non-Seniors Will Be Promoted to Next Grade Objective 2: 92% of Senior (College Ready) Participants Will Graduate Each Year Objective 3: 70% of Senior (College Ready) Participants Will Apply for Financial Aid During the Budget Period Objective 4: 70% of Senior (College Ready) Participants Will Apply for Post-Secondary Admission During the Budget Period Objective 5: 60% of All Senior (College Ready) Participants Will Enroll in College Average Annual ∆% (Slopes) Identify Greatest Opportunities for Improvement
Percent Difference/Percent Total Student Contribution Post-Secondary Application for Admission Exceeded Student Contribution
Objective 5Percent Difference/Percent Total Student Contribution Post-Secondary Enrollment Exceeded Student Contribution Did Not Meet Contribution
obstacles • Defects / Variation • Rework • Overproduction • Waste • Scope Drift / Creep • Continued Sustainablity
Opportunities & Limitations obstacles benefits Proactive vs. Reactive Empowerment Depersonalizes Decision Making Analysis of Results (Reporting is NOT Analysis) Sense of Ownership Manage Unexpected Challenges Transparency & Accountability • Upper Level Management • Budgeting & Forecasting • Staff Buy-in
Recommendations Do not wait until you are being asked for data • Mandated at Executive Level • Individual must be trained in Evaluation Studies • Matter of Necessity in Today’s Climate Develop your improvement plans • Ability to Distinguish Your Program from Others Track your accomplishments • Summative Evaluations are Just ONE part of the Equation • Formative Summative Results Publish them