1 / 13

WG5 Subgroup2 Creation and validation of scripts

This document outlines proposed actions, questions, and issues related to creating and validating scripts. It aims to produce a flowchart and raise points for other working groups. The scope includes other subgroups, required scripts, system to house scripts, leveraging existing standards, and more. This is the start of a framework, with much work to be done in the future.

rshoemaker
Download Presentation

WG5 Subgroup2 Creation and validation of scripts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WG5 Subgroup2Creation and validation of scripts Kevin Kane, PHASTAR Joy Li, FDA

  2. Aims • To create a list of proposed actions • To start to produce a flowchart that documents the process of creating and validating scripts • To raise list of questions, issues and points to note for other WG5 subgroups and other WGs

  3. Scope • Other subgroups • What scripts are required • System to house scripts • Leverage existing standards (CDISC) • Try not to limit to specific scripting languages • Well documented and validated • Can be for outputs or data transformation

  4. Scope This is only a start of a framework. Much work will need to be done in the future. We hope you will continue to work with us. Let’s make as much progress as we can – we won’t be able to finalise all details.

  5. Script Creation • Once a script is loaded, the original author is stored as metadata but does not have any further rights or responsibilities • Basic set of programming standards would be useful. If they are too detailed, may conflict with individual organisations • We should develop standard templates for specifications and user guides etc • Investigate “V Model” further for development process • For minor changes, this should not be a separate script – should be added as an option • Encourage backward compatibility but not an absolute requirements

  6. Metadata • Program name • Language • Program version (auto?) • Platform • Purpose • SDTM/ADaMversion/NA (dropbox) • Keywords • Original Author (auto) • Usage counts • Ratings/feedback • Validation status • Assumptions • Inputs • Outputs • Requirements • Comments/notes • Reason for change • Bug flag (DB table?) • Current author • Language version • Validation documentation

  7. Definition of validated script • Script does what it says in specification • Specifications are required • Design • Inputs • Outputs • Test under various scenarios: these scenarios become assumptions • Code review • Validation by experience is not enough • Website/wiki needs a disclaimer • ISSUE: What documentation is required for unvalidated scripts

  8. Process for scripts to be validated • Upload all validation documentation • Approval by moderator (committee?) • Meets all requirements on validation checklist Can we learn from SAS Online help web pages?

  9. Script governance - functions • Approve scripts • Draft specs • Call for Scripts • Template specs • Guidelines • Validation checklist • Library management • Ratings management • Define metadata • Change management • Incentive management

  10. Script governance – documentation required • Guidelines for creating specs • Define metadata • Overlap between specs and metadata • Web based database? • Template for user guide • Basic programming standards • Checklist for approval to validated state • Definition of requirements to consider a script validated

  11. Issues to pass to platform group • Need to be able to review and comment on scripts. Ideally with quality rating • Create and store multiple versions • Need scripts to be able to have different states: e.g. validated; unvalidated; in development • Metadata e.g. program name; language; parameters; bug flag; variables; outcomes; version number (need to decide list of metadata variables) • Check-in check-out (not 100% defined- what happens if one person checks out for long time) • Ability to have multi-person multi-function teams • Can we have a metadata database on a Wiki

  12. Notes • CDISC CTO suggested we find out about FDA Open Toolbox project. Norman Stockbridge and George Rochester from the FDA (ask Mat) • We noted that different user groups may have different requirements e.g. FDA reviewers data exploration

  13. Incentives:Results from brainstorm • Maybe we don’t need any incentive • Encourage people to get a top rating leading to enhanced reputation • Platform records downloads – “most cited script” • Messages to “market”:- • Reputation factor • This system can save organisations money • This is the same code that the FDA will use • Could offer a PhUSE discount or award • FDA recommendation to use scripts • Airmiles/points system – bronze/silver/gold • Academic encouragement : get your methodology adopted • Confirm if we need any money. Ask PHARMA???

More Related