120 likes | 128 Views
PAAB Pre-Clearance Mechanism Audit Summary. Audit. Purpose Assess the overall performance of the process Evaluate the level of consistency Methodology Audit Period: June 1 st 2013 to May 31 st 2014 Elements from Six Sigma & Standard Audit Practices
E N D
PAAB Pre-Clearance Mechanism Audit Summary
Audit Purpose • Assess the overall performance of the process • Evaluate the level of consistency Methodology • Audit Period: June 1st 2013 to May 31st 2014 • Elements from Six Sigma & Standard Audit Practices • Quantitative deviations – Performance • Qualitative deviations – Consistency
Context Variability (Audit Period) • Submissions: 6987 • Manufacturers: 96 • Agencies: 136 • Products: 600+ • Therapeutic Areas: 25 • APS Categories: 26 • Reviewers: 13
Operational Performance Turn Around • 99.38 % of Response Letters are sent within 10 days • 0.62% of Response Letters send after 10 days
Operational Performance: Revisions • 36.82% of Submissions were Approved after the 1st revision • 61.22% of Submissions were Approved by the 2nd revisions • 78.16% of Submissions were Approved by the 3rd revisions
Consistency How do we define “Inconsistency”? • When PAAB makes a mistake? • When PAAB is inconsistent in the pre-clearance process? • When PAAB makes a ruling that doesn’t align with previous ruling? • When PAAB makes different ruling for two current submissions? • When reviewer and the submitter disagree on an issue?
Results There were no clear and definitive examples of inconsistencies in Reviewer based decisions. • There were differing interpretations and perspectives • PAAB Code • Scientific data • Examples of suspected inconsistencies are needed
Results Escalation Data: • Total Escalations as a percentage of the population: 0.95% • PAAB ruling upheld: 49% • PAAB/Client found a solution: 33% • PAAB ruling overturned: 13% • Client withdrew: 5%
Results • In 6 samples – differing opinion between the Client and the Reviewer • 4 submissions – Client provided APS reference backup Approved • 1 submission – Client provided APS reference backup Denied • 1 submission – Client, 1 issue Approved and 1 issue Denied • 14% of Samples had 2 or more Reviewers • 14% of samples had 2 or more Client representatives • Reviewers missing issues during their first review: • 14% of samples – Reviewer missed identifying an issue(s) • 24% of samples – Change made by the Client, new reference documents or translation
Results • Factors causing delays to the processing of submissions: • 4% of samples – waiting for updated or new Product Monographs or Formulary Listing Letters. • 2% of samples – waiting for another APS to be approved. • The majority of initial submissions omitted the layout and/or French versions • 22% of samples – Immediately approved • 2% of samples – Client forgot to update the Copy Deck • 1 sample – Client couldn’t decide how they would submit the APS (branded or unbranded)
Promoting Consistency What is PAAB doing to support Consistency? • Defined Therapeutic Teams • OneNote Database • PAAB efile System • Reviewer Training • Team Meetings