1 / 11

Rigorous & Consistent Evaluation of Biomarker/Subgroup Identification

Rigorous & Consistent Evaluation of Biomarker/Subgroup Identification. Biomarker/Subgroup Analysis/Identification Sub-team QSPI Multiplicity Working Group October 1, 2013. Objectives. Short term: Test the platform Compare an initial set of methods Collaborate on a publication Longer term:

abiba
Download Presentation

Rigorous & Consistent Evaluation of Biomarker/Subgroup Identification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rigorous & Consistent Evaluation of Biomarker/Subgroup Identification Biomarker/Subgroup Analysis/Identification Sub-team QSPI Multiplicity Working Group October 1, 2013

  2. Objectives • Short term: • Test the platform • Compare an initial set of methods • Collaborate on a publication • Longer term: • Build knowledge base of good methods

  3. Predictive Biomarker Project Three components: Data generation (consistency) Analysis methods (openness) Performance measures (consistency)

  4. Available Files What is provided: • notes.docx • pbpsfolder • pbpe folder • Read “notes.docx” • Just “how”, not “why” • Run examples in “pbps_calls.R” • Use help files if needed • Run examples in “pbpe_calls.R” • Use help files if needed

  5. “pbps” Options Dimensions: • n_datasets • n_subj (overall number) • n_pred Random number seeds for reproducibility: • pred_seed • rando_seed • Not needed of rando_method is “ordered” or “alternating” • predmarkers_seed • resp_seed

  6. “pbps” Options (cont.) • pred_method (“genetic”) • Leave blank for uncorrelated X’s • gen_n_genes • gen_n_blocks_gene • gen_n_snps_block • rando_method (“random”, “alternating”, “ordered”)

  7. “pbps” Options (cont.) • n_predmarkers • gen_maf • predmarkers_method • “unrestricted”, “one per gene”, “one per block”, “one per gene (unique)”, “one per block (unique)” • predmarkers_etype • A vector, e.g. “c(dominant, recessive, dominant)” • predmarkers_esize (also a vector) • resp_trt0_mneg (mean response) • resp_trt1_mneg (mean treatment effect) • resp_sd • subgrp_def(optional)

  8. “pbps” Options (cont.) • out_dataset_format • “concatenated”, “separate” • out_dataset • out_info • out_truth(used as a prefix) Not planning to use initially: • Custom X’s using pred_file • Custom randomization scheme using rando_file • Custom list of predictive markers: predmarkers_method is • “user-specified” (also need predmarkers_list) • Or “partitioned”

  9. “pbps” Output • Datasets • CSV file • id, trt, y, X's • Info file • TXT files listing selected options • New info: correlations, predictive markers, seeds • “Truth” files: CSV files, one column per dataset • Marker “truth” file (0/1 for each marker) • Effect “truth” file (expected TE for each subject) • Subject “truth” file (0/1 for each subject)

  10. Analysis Any analysis method, as long as it: • Uses standard datasets as input • Produces these standard results • Marker “result” file • Same format as marker “truth” file • Subject “result” file • Same format as subject “truth” file • Estimate “result” file (optional) • One row only, one number per dataset

  11. Next Steps • Test programs • Another discussion on performance measures • Improve programs as necessary • Scenarios of interest • e.g. therapeutic area, phase, dimensions • Initial list of BSID methods to be evaluated • Collaborate on method evaluation • Collaborate on a publication

More Related