1 / 35

Making Public Spaces Safer

Making Public Spaces Safer. Towards Policing Perception Maps automatically extracting quantitative analysis of qualitative survey data. Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster University/ Brunel University.

elwyn
Download Presentation

Making Public Spaces Safer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making Public Spaces Safer Towards Policing Perception Maps automatically extracting quantitative analysis of qualitative survey data Dr B. Nasa, Prof J. Binner Dr M. A. Ferrario, Dr W. Simm, Prof J. Whittle, Dr B. Lam; Sheffield Management School Lancaster University/ Brunel University

  2. Background: Derry/NI SELF POLICING & FREE DERRY: 1960s on going clashes between Protestants and Catholics. By the end of 1971, barricades prevented access to Free Derry (Bog Side Area) . No police forces allowed. BLOODY SUNDAY: 30th January 1972, 27 civilians were shot by the British Army Parachute Regiment during a Civil Rights march. 13 died on the scene. HOW TO build trust in the police after 30+ years of Conflict (The Troubles)?

  3. Derry District Policing Partnership Derry DPP is one of the 26 DPPs in NI • Responsibilities: • Monitoring local police performance • Giving a voice to community views on policing • Gaining the public's cooperation to prevent crime. DDPP Household survey 2009 Confidence and Satisfaction

  4. DDPP Household Satisfaction & Confidence Survey’09 • All-household survey to inform next year local policing plan (2010/11) • First-ever DPP blanket consultation by Neighbourhood • 46,000 households: 465 responses received to the survey (1% response rate)

  5. Verbatim vs Tick-Box

  6. Making Sense of Qualitative Data • The survey contained questions with multiple choice answers and the opportunity for the respondent to expand upon their selection in free textform. • Traditionally, this free text is difficult and time consuming to analyse by theme and sentiment. • Automatically extracting ‘quantitative summaries’ could help identify themes which are not part of the multiple choice options and would allow comparison across results from different districts.

  7. Factual vs Perception Mapping Perceptions maps Map of actual crime statistics

  8. Challenge: How do people feel... • 36% of respondents** answered ‘don’t know’ to the satisfaction question => How can the police ‘win’ the undecided? • Understanding ‘Why’people are undecidedmay help… ** diagram and figures from DDPP survey

  9. Data Set: Original Source 2 3 1 4 5 6 DDPP ID 1 LOCALE 2 CONFIDENCE LEVEL 3 CONFIDENCE VERBATIM 4 SATISFACTION LEVEL 5 SATISFACTION VERBATIM 6

  10. Extracted – Full Data Set First Step: 449 Records in total Second Step: Sentence Level Split + Clean Data+ Remove Blank Verbatim = 379 Sentence level comments (Satisfaction) = 485 sentence level comment (Confidence)

  11. Sample Data Set – 95% CL; CI 5% • 200 comments (Satisfaction) • 221 comments (Confidence) • Manual analysis (three researchers) • Theme, Actionability, Sentiment (TAS) • Automated Theme Extraction – Accuracy Testing • VYV • Bayes Classifier

  12. TOP THEMES (Satisfaction & Confidence) Approach: Initial Identification of themes (Cross-checked with DDPP) Independent Theme Review (by three researchers) Up to 4 themes per comments Calibration and Compilation of final theme List Emerging Results:

  13. THEMES BY LOCATION Second Step: Match Themes to Neighbourhoods Neighbourhood analysis of Satisfaction with Policing in Derry

  14. 3-WAY ANALYSIS: themes v. sentiments v. location

  15. SATISFACTION v. SENTIMENTS

  16. Is stated level of satisfaction reflected in verbatim Null: SL is independent of VS Chi square statistics : χ2 = 211.04 (0.000 ) Reject null, so SL is associated with VS

  17. Example VERBATIM ANSWERS** Do you think the PSNI are doing a good job in the Foyle area? Don’t know – PSNI take too long to respond to a reported incident. Don’t Know - They certainly don’t appear to be able to curb the problems in my area of Culmore. Don’t Know -Because they let the young joy riders straight out again also they know all the drug dealers in the one area still they turn a blind eye.

  18. Automated Theme Extraction VYV Bayes Classifier

  19. Accuracy of Automated VYV Theme Extraction Approach: VYV automatically assigned (two) themes Accuracy assessed on a 3-point Likert scale Independent assessment by 2 reviewers, followed by a ‘calibration exercise’ to address differences 69-72% of the themes were acceptable or accurate for both sets Reviewers agreed on both themes 93-94% of the time

  20. “Very helpful and friendly staff” <w pos="NN" lemma="staff" sem="I3.1/S2">staff</w> Noun picked from POS = “Staff” sem tag I3.1 Secondary sem tag S2

  21. VYV tags vs Manual (Satisfaction) VYV identified a total of 90 unique themes Manual tagging identified 17 themes Some Interesting Parallels and Challenges

  22. Theme Assignment Alternative • Classifier methods can be used to classify texts: • The classifier is trained using manual data • A number of exclusive classifications are defined • As subset of data is selected and manually classified • This subset is used to train the classifier on the features (in this case the words) that make up each classification • New comment are classified by the classifier based on features

  23. Bayes Classifier applied to DDPP Data • Naive Bayes Classifier was applied to “Confidence” data: • Training subset used previous 83 manually tagged comments • classifier accuracy increases with number of training data • 20 further comments were extracted at random and classified by the Bayes classifier to test accuracy. • Preliminary Results: • In a manual review by 2 researchers (method described previously) the classifier returned an “Acceptable” or “Accurate” classification for 60% of the 20 comments.

  24. Towards Perception Maps

  25. Interactive Map Display Safer Streets Plot the information derived from automatic analysis of survey verbatim on an interactive map Allows interaction with, and evaluation of the data by stakeholders and possibly public

  26. Implementation • Design Storyboard by Information Architecture student Zoe Zhao (Brunel University) • Development Progress using web technologies • Google maps • Web 2.0 technology including Ajax and JQuery

  27. StoryBoard A • Frontpage, which contains: • Map of the whole area • Overall impression – e.g. Themes, Number of People Reporting, etc

  28. StoryBoard B Click on the map to select the area that you want more information The selected area will be highlighted The information of the specific area replaces the overall impression

  29. StoryBoard C Select the information that you want to explore further Place the cursor on the theme for actual figures You also can rate your satisfaction level in this category The sentiment charts will be updated automatically

  30. StoryBoard D Click on the themes for actual comments You will see both positive and negative comments under this theme You can enter your comments or respond to other people’s comments You will see your comment appears on the list straightaway

  31. Challenges: • Database design • Google map plots • Ajax for realtime updates • Jquery interface animation DDPP Survey Data 2009

  32. Conclusion and Next Step • Empirical Analysis • Test specific hypothesis such as • whether there is more crime/offences where police are sectarian • Is response related with police behaviour • Automatic TAS • Train bayes classifier using larger dataset with fewer categories • Interactive Map • Put the plans into action

  33. Partners & Collaborators www.voiceyourview.com

More Related