1 / 16

Evaluation on shifting ground

Evaluation on shifting ground. Sonia Barnes, NZ Police Alison Chetwin, Independent Evaluator (formerly NZ Police). Presentation overview. Context Neighbourhood Policing – evaluation approach and changes to it, specific learning and contributions

rosine
Download Presentation

Evaluation on shifting ground

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation on shifting ground Sonia Barnes, NZ Police Alison Chetwin, Independent Evaluator (formerly NZ Police)

  2. Presentation overview • Context • Neighbourhood Policing – evaluation approach and changes to it, specific learning and contributions • Mobility – evaluation approach and changes to it, specific learning and contributions • What we learnt

  3. Context Policing Excellence • Suite of 9 initiatives aiming to improve efficiency and effectiveness of NZ Police and shift more effort into prevention • Implementation of change programme began in 2010 with formal project management processes and evaluation • Implementation completed in 2013, targets to be met by 2014/15

  4. Context • 2014/15: achievement of Policing Excellence aims • Increase preventative policing by 4% • Reduce crime and crash by 13% • Reduce prosecutions by 19% • 2017: contribute to Better Public Services targets • Reduce crime by 15% • Reduce violent crime rate by 20% • Reduce youth crime by 5%

  5. Neighbourhood Policing • Neighbourhood Policing Teams: • Sergeant + 4-6 • Vulnerable neighourhoods • Up to 9,000 residents • Community engagement, partnership working, problem solving • Aim to: • Reduce crime and crash and re-victimisation • Increase trust and confidence, satisfaction, feeling safe

  6. Original evaluation approach • South Auckland – to inform national roll-out • Desire for outcome evaluation • Two years before measure • Crime reduction • Perceptions of Police • Formative evaluation • Identification of outcomes and theory of action • Activities based on good practice • Early operation of teams

  7. Adapting the evaluation to implementation changes • Accelerated implementation • Implementation in South Auckland late 2010 – early 2011 • Evaluation commenced early 2011 • Directive mid-2011 for 32 NPTs by end 2011 • Adaptation of evaluation approach • Emphasis on formative evaluation and learning to inform further implementation • Advisory role on implementation team • Responsive to ad hoc evaluative requests

  8. What the evaluation contributed • Wider awareness and utility of findings • Evidence based development of initiative: • International research evidence • Learning from local implementation • Implementation team sought advice from evaluation team on aspects of development • Education/ awareness raising of utility of evaluative work – raised profile of value of evaluation

  9. Mobility initiative • Mobile devices to access and record into police systems while out of station • Aim to save time, improve safety, make police work more effective • Trial in four police areas to be evaluated • Emphasis on summative evaluation

  10. Mobility trial Model of evaluator/implementer collaboration • Early involvement as independent expert adviser • Input into trial design to ensure evaluability • Reporting to allow time for outcomes to emerge • Integration between project management and evaluation • Formal agreement to evaluation framework • Investment decision to draw on evaluation evidence

  11. Mobility – outcomes framework

  12. Shifting timeline Oct 2012 Business case for national roll-out Jan 2013 National roll-out commences

  13. Options – changed timeframe • Continue with agreed evaluation plan • Seek reversion to agreed timeframe • Abandon evaluation • Adapt evaluation to new needs

  14. Adapted design • Need to refine the trial • Formative real time information – surveys at completion of each roll-out • Informal verbal reports – surveys and observations • Need to inform national roll-out • Strengthen and bring forward qualitative component • Formal interim report and presentation • Need to inform investment decision • Retain summative evaluation

  15. What we learnt • Importance and value of working with project management methodologies • Flexibility expands evaluation uses • Being prepared to circulate emerging findings leads to greater utility • Being willing to assist in developing the initiative results in initiative being more grounded in evidence • Summative evaluation needed for large investment decisions • Accelerated timeframes not always achievable • Demonstrate value of formative evaluation

  16. Contact details Sonia Barnes Senior Evaluator, NZ Police sonia.barnes@police.govt.nz Alison Chetwin Independent Researcher & Evaluator alison.chetwin@ clear.net.nz http://www.police.govt.nz/new-zealand-police-evaluation-reports

More Related