Ministero della Giustizia

1. Ministero della Giustizia Direzione Generale di Statistica

2. Italian Ministry of JusticeOrganization, Human Resources and Services DepartmentDepartment of Statistics Customer Satisfaction Surveys conducted in the pilot courts of Turin and Catania Tbilisi, February 10, 2012

3. Background The idea of conducting a satisfaction survey in Italy aimed to Court users takes its stands from the guidelines prepared by the Quality Working Group of the CEPEJ. The idea was proposed and enthusiastically accepted by the Heads of the Courts of Turin and Catania. How satisfied or dissatisfied are you with your personal experience of the court today?

4. Objectives • Main objectives of the survey are: • To evaluate the overall satisfaction level of Justice amongst thefinal users • To establish Key Performance Indicators (KPIs) to be tracked throughout time • To pinpoint the areas of improvement and (propose to) take action accordingly • To analyze the satisfaction amongst specific sub-samples (e.g. males/ females, age classes, …) • To plot Performance vs Importance diagram Performance Maintain Focus Importance Low priority Improve

5. Working group • A key success factor towards the success of such initiative is to involve as much as possible the stakeholders of the world-justice. • Thus, the working group to design and conduct the survey was composed by: • Ministry of Justice representatives(and as members of the CEPEJ) • Statisticians • Heads of the Courts of Appealand Tribunals • Judges and Lawyers • Professors of the University(Law, Politics and Economic Sciences)

6. Media Attention The initiative was reported on main national newspapers and local news broadcast

7. Courts under survey Turin Catania • Members of the CEPEJ were responsible for managing and coordinating the initiative in both cities • Last year we conducted the survey in Turin and Catania • Depending on the goodness of the results this kind of survey might be extended to other courts across the Country • In Turin the survey was conducted only with users while in Catania it was done also with other two targets: Lawyers and Court staff

8. Constraints and methodology • Since this Satisfaction Survey is carried out without a budget, we have decided to: • maximize the use of internal resources and free external resources • guarantee the quality of the project through involvement of internal professional statisticians and university professors • select a group of around 25 students per city to conduct the interviews Additional slides are reported in the second part of this presentation on how to conduct interviews

9. The sample Interview Citizens visiting the Courts of Turin and Catania (preferably people who have already got a contact with the service provided) during February-March 2011. • Preferably a “random” sample of 600 usersper Court • Alternatively a sample composed of atleast 380 users per Court • Statistical note  within an unlimited population: • A sample of 600 users guarantees that the standard error is lower than 4% • A sample of 380 users guarantees that the standard error is lower than 5%

10. The target The target has been defined according to the following criteria: Criteria of inclusion Criteria of exclusion * • Parties • Witnesses • Interpreters, experts • Relatives of the Parties, of witnesses, etc. • Lawyers • Judges, Prosecutors and staff of the Court • Policemen, Bailiffs, etc. (*) In Catania two specific satisfaction surveys aimed at Lawyers and Court staff have been carried out separately.

11. The questionnaire The questionnaire has been designed according to the CEPEJ guidelines, taking into account specific needs of the Courts in Turin and Catania. For this reason we have used two slightly different questionnaires. Both questionnaires are divided into 3 different sections: • Screening questions • Main Questions. Satisfaction of: • Court premises, organization, clean environment • Punctuality of hearings, length of proceedings • Judge professionalism, competence, cordiality • Information points within the Courtand web-sites • Court location and ways of transport • Demographic Questions

12. Promoting the initiative Before the interviews started, many posters promoting this initiative were stuck to the walls of the Courts premises. This simple action had a great contribution on the respondents’ willingness to take part into this survey. • Additional publicity has been obtained through: • Lawyers in Turin and Catania: they were asked to talk about this initiative among their clients. • Judges: they were all formally informed about this initiative. Catania has requested the CEPEJ to provide some gadgets to be distributed to the people who participated to the survey

13. Data acquisition and results The data collection is performed through an on-line application. At the end of the day all questionnaires are being loaded-up into a central database by the interviewers using an on-line form. The results (charts and graphs) are available in real-time to the members of the working group. Statistical Note: In order to ensure good quality standards, a selection of random paper questionnaires will be double-checked against the online data to verify that the questionnaires have been transmitted correctly.

14. Project timing The whole project lasted about 8 months, from very beginning to presentation of results

15. The results: Overall Satisfaction • Overall satisfaction with the court experience TURIN Score: 3.16 Score: 2.48 CATANIA Note: Scores are calculated applying a weighted mean. Weights range from 1 (Very Dissatisfied) to 5 (Very satisfied).

16. Satisfaction vs Importance diagram TURIN

17. Satisfaction vs Importance diagram CATANIA

18. Italian Ministry of JusticeOrganization, Human Resources and Services DepartmentDepartment of Statistics Key suggestions on how to conduct interviews and collect the data Tbilisi, February 10, 2012

19. Interviewers training Prior to the fieldwork a number of motivational sessions have been carried out. Moreover, in order to ensure good quality standards, interviewers were carefully briefed on the following: • Phases of the interview (approach, questionnaire, closure) • Behavior and good conduct • What to do and what to avoid • Questionnaire flow (screening, main and demographic questions)

20. Behavior and good conduct • Kindness, discretion, self-confidence, inspiring trust • Neutral behavior, attention to non-verbal communication • Read the questions in the exact order they are reported into the questionnaire • Availability to read questions and all possible answers even more than once • Take note of all the comments coming from the interviewed person

21. The questionnaire - Introduction by the interviewer INTRODUCTION Good morning / afternoon, my name is ___________________ and I’m here today because the Ministry of Justice is currently conducting a survey aimed at court users such as parties, witnesses, experts, interpreters, etc The Ministry of Justice is interested in your own opinion and in your experience of this court in order to improve the overall quality of the service. This is a statistical survey and therefore your answers will be handled with care. At the end of this survey data will be analyzed in the utmost anonymous way. Can I ask you a few questions? Strict confidentiality of your answers is guaranteed. This survey is anonymous.

22. The questionnaire - structure Conducting one interview takes approximately 15 – 18 minutes (it is important to notice the person about the actual duration of the interview) • Introduction by the interviewer • Screening questions • Core questionnaire • Demographic questions • Closing

23. SW utilized for data collection – the tool • Data has been collected using free tools on the web, called Google Docs • Google Docs is a collection of web-based documents that let users edit the same file at the same time so you always have the latest version • In particular, an online form was designed in order to collect the data submitted by the interviewers • The main advantages of using an online form for collecting data are: • Quality of data is guaranteed • Answers can be accessed in real-time • Data is ready for analysis at any time • The workgroup can easily track the interviewing process TURIN CATANIA

24. SW utilized for data collection – the steps 1 • Designing the questionnaire A form (i.e. questionnaire) can be easily designed using single and multiple choice questions, scales, grids, text boxes etc. 2 • Collecting the data Once the form is online, interviewers can load their questionnaires via web. The data submitted is stored in an online spreadsheet document. 3 • Summary report A summary report with charts and graphs is available in real-time. 4 • Exporting data into Excel for further analysis. Data can be exported into excel format (amongst the others) and eventually into spss format for further analysis.

25. Taking care of all details in the organization The workgroup must consider all the details related to the organization, in this slide, for example, we report the map shown to students to make them aware of the courts premises and location. 3 1 2 PIAZZA GIOVANNI VERGA, Palazzo di Giustizia (1) VIA FRANCESCO CRISPI, ex Pretura (2) VIA VERONA, Sezione Lavoro (3)

26. Timing and other practical informations • In our experience, in a working morning, e.g. from 9.00 to 14.00, an interviewer can conduct up to 6 interviews • Interviewers were organized into sub-groups of 6-8 people with an agreed calendar over a period of 6-8 weeks • It is important to recommend the interviewers that the choice of the person to be interviewed must be absolutely random • At the end of each day, each interviewer was requested to report filled-in questionnaires into the on-line application

27. Ministero della Giustizia Direzione Generale di Statistica