1 / 13

WG5 P02 Proposal 2014 Qualification of Standard Scripts

WG5 P02 Proposal 2014 Qualification of Standard Scripts. Proposal through CSS 2014 http:// www.phusewiki.org/wiki/index.php?title=File:FDA_Scrips.ppt. Anyone should be able to submit a script, according to a check list Categorize scripts according to complexity

druce
Download Presentation

WG5 P02 Proposal 2014 Qualification of Standard Scripts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. WG5 P02 Proposal 2014Qualification of Standard Scripts

  2. Proposal through CSS 2014http://www.phusewiki.org/wiki/index.php?title=File:FDA_Scrips.ppt • Anyone should be able to submit a script, according to a check list • Categorize scripts according to complexity • Complexity: low, medium, high, software • Output: tabulated data, analysis data, table, figure, listing • Metadata for script should indicate • Type of output: tabulated data, analysis data, table, figure, listing • Study design: parallel, crossover, etc • State of qualification: ?

  3. Proposal through CSS 2104 • Test data • Overall project should have minimum test data (SDTM & ADaM) • Scripts can propose new test data, must pass (Data fit? Open CDISC?) • Share program to produce test data, never binary test data • 2 levels of qualification to match script complexity/output • Light vs. Heavy qualification • Common elements include • header • good programming practices • clearly declared scope of script (e.g., study design(s)) • test data matches scope & passes "FDA Data Fit" assessment (?) • documentation inputs/outputs/dependencies/usage

  4. Proposal through CSS 2104 • Heavy qualification • Beta package includes minimal elements for contribution • Specification & Documentation (could be in pgm header) • Test data (Data Fit? or Open CDISC or other, as appropriate) • Tests & Expected results defined • Peer Review: GPP, Specs & Docn reviewed, Tests reproduced • Draft • Write qualification plan, Review tests for completeness/suitability (e.g., Branch testing – are all conditional blocks/combos tested?) • Test • Peer Review: Write qualification report, incl. log/output from tests • Final

  5. Proposal through CSS 2104 • Light qualification • Beta package includes skip if >1 yr production usewithout ERROR • Draftminimal elements for contribution • Specification & Documentation (could be in pgm header) • Test data (Data Fit? or Open CDISC or other, as appropriate) • Tests & Expected results defined • Peer Review: GPP, Specs & Docn reviewed, Tests reproduced • Write qualification plan, Review tests for completeness/suitability (e.g., Branch testing – are all conditional blocks/combos tested?) • Test • Peer Review: Write qualification report, incl. log/output from tests • Final

  6. Proposal through CSS 2104

  7. Proposalmeaningful terms in blueQualificationhttp://www.phusewiki.org/wiki/index.php?title=File:WG5_P02_Proposal_-_2014.pptx • Qualification Instructions (see embedded template ð) • Certificationphase of Qualification applies to new scripts and tests • Confirmation phase applies to updates of existing scripts • States: Contributed, Development, Testing, Qualified • Roles • Contributor: Anyone with appropriate skills & interests • Developer: CSS Working Group 5 volunteer familiar with objectives** • Tester: CSS WG 05 volunteer familiar with objectives** • Environment Tester: Anyone in industry community able to set up automatic test replication in their work environment • Reviewer: Author of white papers, designers of script targets ** suggests a quick-start onboarding page in CSS Phusewiki

  8. ProposalQualification • Metadata for script should indicate • Whitepaper & output ID • Programming language & version (e.g., R 3.1.1) • Type of output: tabulated data, analysis data, table, figure, listing • Study design: parallel, crossover, etc • State of qualification: Contributed, Development, Testing, Qualified • Test Data requirements • available as a program or script (text, not binary) • based on expected standards (SDTM, ADaM) • data requirements clearly & accurately specified for each script • include expected results from these data for defined tests/checks

  9. ProposalQualification • End-user Objectives • Clear overview of purpose and resources • Inspire confidence from first sight • Ease of use, clear messaging from first run • Reproducible results • Consistency of scripts, learning first one makes remaining familiar • Ease of converting users to contributors • Contributor Objectives • Standardize routine steps • Modularize routine components • Automate testing, issue identification • Centralize & consolidate information & results

  10. ProposalQualification • TransitionsContributed is the original State of all scripts • to Development, checklist includes by Developer & Reviewer • R & D confer on suitability of contribution. Suitable starting point?[ may require analysis details, specs, from contributor ] • D reviews components • D works with Contributor to complete minimum components[ including Test Data and Coverage of defined tests ] • D adds standard parameter, dependency checking • D writes Qualification instructions .docx (see template, above) • to Testing, checklist includes by Tester • Review Qualification instructions, consider coverage of tests • Execute Qualification instructions • Work with Developer to complete execution successfully

  11. ProposalQualification • Transitionscontinued • to Qualified by Tester & Environment Tester & Reviewer • T updates posted test outputs from certification/confirmation • E updates local tests and executes (posting PASS/FAIL results) • R confirms script output matches intention & qualification process covers important elements and considerations. Also provides user (rather than technical) feedback? • Achieve "Qualified" state when all tests in all test environments PASS (i.e., match outputs that T has certified and/or confirmed) and that R agrees that target is achieved

  12. ProposalQualification • Efforts Required • Finalize Qualification states, roles, workflow, checklists, and templates • Design test structure in google code • Develop scripts that will allow Environment Testing • Develop general components (e.g. parameter, dependency checking) • Identify Environment Testers based on • Host environment • SAS or R version • Identify opportunities to automate qualification. E.g., • Docx format for Qualification instructions is not easily machine readable • Environment Testers to post results back as machine readable • Script green-light/red-light qualification matrix on Phusewiki

  13. Doing now what patients need next

More Related