1 / 80

Parliamentary Reports: A Limited Usefulness Analysis

This annotated version of a playful presentation challenges the effectiveness of evaluation reports for Parliament. Despite 80 slides, the reading time is under five minutes.

berick
Download Presentation

Parliamentary Reports: A Limited Usefulness Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. I think it’s something internal, doc! This is an annotated version of a playful presentation given at the 2019 conference of the Canadian Evaluation Society. It is intended to ignite thinking about the limited usefulness to Parliament of reports generated by the evaluation units that are embedded in federal departments and agencies. Although there are 80 slides in the show, the reading time should be less than five minutes. Michael Obrecht CE

  2. I think it’s something internal, doc!

  3. Hmm. May I see your health card, please?

  4. ` …………………………………………………………………………………………………………………………………………….. ` ………………………………………………………………………………………………………………………………………..…… Patient Interview Notes Name: Sure, doc. Here it is. Age: Sex: Clinical observations: ``

  5. ` …………………………………………………………………………………………………………………………………………….. ` ………………………………………………………………………………………………………………………………………..…… Patient Interview Notes Hope it’s OK. I haven’t renewed it since I fell asleep! Name: Age: Sex: Clinical observations: ``

  6. ` …………………………………………………………………………………………………………………………………………….. ` ………………………………………………………………………………………………………………………………………..…… Patient Interview Notes Name: Rip Van Winkle Age: Sex: Clinical observations: ``

  7. ` …………………………………………………………………………………………………………………………………………….. ` ………………………………………………………………………………………………………………………………………..…… Patient Interview Notes Name: Rip Van Winkle Age: Elderly Sex: Clinical observations: ``

  8. It’s OK, ah… Rip. So, why don’t you tell me how it all began.

  9. Doc, it started when I asked myself a question…

  10. How many evaluation reports are there in the federal government database?

  11. v Fascinating.

  12. Audit and Evaluation Database I found the database at the Treasury Board site. Established in 2003

  13. Each department has its own virtual bin... Department 21 Property of Treasury Board

  14. A filled with audits… Department 21 Property of Treasury Board

  15. Ev evaluations… Department 21 Property of Treasury Board

  16. ? and reports that do not have “audit” or “evaluation” in the title. Department 21 Property of Treasury Board

  17. There are 70 departments and agencies in the database. Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treas Board

  18. That’s a lot of bins, doc! Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treas Board

  19. So rather than going through each one, I decided to do an estimate. Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treas Board

  20. I organized them by size of department. Over 8 billion $ 1 to 8 billion $ Under .1 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board

  21. Watch the second row, doc, while I explain how I got the estimates. Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  22. I located the department near the mean budget… RCMP Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  23. And counted the reports in its bin. RCMP Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  24. Next I located the department near the median budget… DFO’s bin Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  25. and counted the reports. DFO’s bin Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  26. I added the counts for the RCMP and DFO then divided by two to get a mean for bins in the row… Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  27. That produced an average of 54 Audits, 33 Evaluations and 10 Other Reports. Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  28. Then I multiplied each mean by 23. Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  29. Result? An estimated 1,242 Audits, 759 Evaluations and 230 Other Reports in the second row. Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board (23 bins)

  30. I did the same for the other three rows to get the grand total. Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board

  31. Doc, I estimate that there are 1,816 Audits, 1,454 Evaluations and 584 Other Reports in the government database! Under .1 billion $ Over 8 billion $ 1 to 8 billion $ .1 to .9 billion $ Property of Treasury Board Property of Treasury Board Property of Treasury Board

  32. I see. And you did this because…?

  33. I felt obsessed by the question, doc!

  34. I was compelled to find those numbers!

  35. Sex: M ` Clinical observations: Narcolepsy? OCD?

  36. Hmmm. So why you think it is something internal?

  37. Well doc, when I was doing the estimate, I had a vision of Parliament.

  38. An MP rose and said…

  39. Mister speaker, we demand an audit of all government research programs!

  40. The Speaker agreed, and soon the call went out from Parliament.

  41. We want an audit of all government research programs!

  42. That night, the internal auditors went through their bins to find relevant audit reports. A A Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treasury Board Property of Treas Board

  43. Read this! Next morning, they appeared on Parliament Hill, waving their reports and calling out…. Read these! Read this!

  44. “That is kind of you, dear internal auditors,” said Parliament. “But no thanks!”

  45. Those audits are history! We want the current situation!

  46. We want a broad, cross-government perspective. Your reports are too narrowly focused.

  47. You are not independent! You work for a department!

  48. We don’t even know who actually did each audit!

  49. The internal auditors were disappointed, but eventually they all left Parliament Hill…

  50. …and returned to their departments.

More Related