120 likes | 136 Views
This audit evaluates the pre-clearance process performance, consistency levels, and operational efficiency of PAAB from June 2013 to May 2014. Results show insights on inconsistencies, operational performance, revisions, escalation data, and promoting consistency efforts. Learn about the factors affecting delays, client-representative interactions, and the measures taken to ensure consistency.
E N D
PAAB Pre-Clearance Mechanism Audit Summary
Audit Purpose • Assess the overall performance of the process • Evaluate the level of consistency Methodology • Audit Period: June 1st 2013 to May 31st 2014 • Elements from Six Sigma & Standard Audit Practices • Quantitative deviations – Performance • Qualitative deviations – Consistency
Context Variability (Audit Period) • Submissions: 6987 • Manufacturers: 96 • Agencies: 136 • Products: 600+ • Therapeutic Areas: 25 • APS Categories: 26 • Reviewers: 13
Operational Performance Turn Around • 99.38 % of Response Letters are sent within 10 days • 0.62% of Response Letters send after 10 days
Operational Performance: Revisions • 36.82% of Submissions were Approved after the 1st revision • 61.22% of Submissions were Approved by the 2nd revisions • 78.16% of Submissions were Approved by the 3rd revisions
Consistency How do we define “Inconsistency”? • When PAAB makes a mistake? • When PAAB is inconsistent in the pre-clearance process? • When PAAB makes a ruling that doesn’t align with previous ruling? • When PAAB makes different ruling for two current submissions? • When reviewer and the submitter disagree on an issue?
Results There were no clear and definitive examples of inconsistencies in Reviewer based decisions. • There were differing interpretations and perspectives • PAAB Code • Scientific data • Examples of suspected inconsistencies are needed
Results Escalation Data: • Total Escalations as a percentage of the population: 0.95% • PAAB ruling upheld: 49% • PAAB/Client found a solution: 33% • PAAB ruling overturned: 13% • Client withdrew: 5%
Results • In 6 samples – differing opinion between the Client and the Reviewer • 4 submissions – Client provided APS reference backup Approved • 1 submission – Client provided APS reference backup Denied • 1 submission – Client, 1 issue Approved and 1 issue Denied • 14% of Samples had 2 or more Reviewers • 14% of samples had 2 or more Client representatives • Reviewers missing issues during their first review: • 14% of samples – Reviewer missed identifying an issue(s) • 24% of samples – Change made by the Client, new reference documents or translation
Results • Factors causing delays to the processing of submissions: • 4% of samples – waiting for updated or new Product Monographs or Formulary Listing Letters. • 2% of samples – waiting for another APS to be approved. • The majority of initial submissions omitted the layout and/or French versions • 22% of samples – Immediately approved • 2% of samples – Client forgot to update the Copy Deck • 1 sample – Client couldn’t decide how they would submit the APS (branded or unbranded)
Promoting Consistency What is PAAB doing to support Consistency? • Defined Therapeutic Teams • OneNote Database • PAAB efile System • Reviewer Training • Team Meetings