E N D
A friendly clinician has asked you to go about setting up a test for a genetic disorder previously characterised by a research laboratory. Discuss how you would go about designing a robust diagnostic assay that could be offered as a service to other laboratories. Concentrate on factors that you would need to take into account when setting up a service. Stephanie Batey RCPath: Session 3(am) 06/11/2008
ACCE framework Major framework for genetic test evaluation: Analytical validity Clinical validity Clinical utility Ethical, legal and social implications (ELSi) How cost effective is the test?
Test development strategy • Determine clinical utility. • Background research: molecular and clinical aspects of this disorder • Communication with research laboratory • Design testing strategy for diagnostic approach • Validation – robust and reproducible assay - Analytical and clinical validity • Establish the service: test new samples • Review the test/ evaluation • Advertise service at conferences and submit gene dossier to UKGTN.
Clinical utility Clinical utility: the likelihood that the test will lead to an improved outcome • Clinical benefit:diagnosis, treatment, prognosis & management, family planning, impact on health economy • Results must have clear implications. Consider ELSi. • Severity and prevalence of the disease • Sufficient demand – local and national • Local clinical expertise: clinical interpretation, gatekeeping issues • Appropriate pre- and post-counselling available?
Research information • What is known about disorder and gene(s) involved? • Is the relevant research laboratory reputable/ accessible? • Current testing strategy in research laboratory and detection rate? • Any known common mutations/ mutation hotspots in the appropriate gene(s). • Homology to other genes. • Access to positive samples for validation? - Ethical considerations.
Designing the assay • Decide on method of mutation analysis: Mutation scanning vs targeted mutation detection • Consider: mutation type and distribution, size and nature of the gene • High GC content resulting in difficult melt profiles or high frequency of SNPs in a gene may restrict mutation scanning methods. • Pseudogenes: nested PCR approach • Influenced by method used in research laboratory? • Best use of resources – combine with other services in the laboratory? • > 1 assay required eg. MLPA & direct sequencing?
Test efficiency • Test needs to be efficient to meet reporting deadlines and match referral levels. • Dependent on: • Method itself – techniques differ in speed • Referral level: employ pre-screen? • Mutation spectrum: target hot-spots, sequence specific exons • Batching – dependent on referral level and reporting times • Pick-up rate: if high referral rate and pick-up rate low – test becomes inefficient • Is test amenable to automation? Is a high throughput approach justified?
Practical aspect • Design and order relevant primers/ probe/ kits? • SNP checks, BLAST searches. • Gain information about any new kits • Communication with other diagnostic laboratory/ research laboratory. • Optimisation of reactions/ analysis software
Test validation • Once decided on method of test and testing strategy: Validation of assay: access samples with known genotypes: informed consent. Reference materials? Consider target population. • Reference standards for assay. • Test large number of samples with confirmed mutations or normal genotype as part of blind study.100% concordance • Reproducibility: within runs and between runs. Use same sample as control. • Low failure rate: saves money and time • Assess false positive and false negative rates • If feasible – screen large number of anonymised normal controls: gain data on polymorphisms
Analytical validity Ability of a test to measure accurately and reliably the genotype of interest. Analytical sensitivity : the probability of a positive result in the presence of a mutation. Analytical specificity: the probability of a negative result when a mutation is not present. Ideally 100% • Method and mutation specific • Will depend on target population and prevalence of the disease/ mutations in that population Be aware of test limitations – make clear on reports
Clinical validity Clinical validity: the ability of a test to detect or predict the presence or absence of a phenotype or clinical disease. Reflects: • Clinical sensitivity: proportion of positive results in the presence of the disease in question • Clinical specificity: proportion of disease-free individuals who test negative. • Penetrance of the mutations, which the test identifies • Genetic heterogeneity • Allelic heterogeneity • Variable expressivity • Inappropriate clinical referrals?
Establish service • Test new samples with unknown genotypes • Include appropriate positive and normal controls: ongoing IQC critical • Gain further data to demonstrate the test is robust and reproducible • Reporting templates • Issue reports for retrospective patients? Ethical considerations. Has testing strategy changed? • Documentation: validation, IQC, SOP UKGTN/ EQA • Review after set period of time – relevant audit • UKGTN proposal • Relevant EQA scheme available • Sample swap QA for rare disorders – recent development.
Financial considerations • Where will funding come from? • Budget: staff, consumables, equipment, overheads • Anticipated workload: - how long will test take to set up, including validation? - existing resources/ expertise? - new equipment/ training needed. - integration into current services • Cost dependent on: - number of referrals - method of testing. • Price test will be offered at?
References • Evaluation of genetic tests: the experience of the UK Genetic Testing Network: Dr Mark Kroese, UKGTN Public Health Advisor, 2008 • Melzer, D.et al (2008) Genetic tests for common diseases; new insights, old concerns BMJ 336:590-3 • Zimmern, R.L & Kroese, M. (2007) The evaluation of genetic tests J Public Health (Oxf) 29:246-50 • www.phgfoundation.org The evaluation of clinical validity and clinical utility of genetic tests: summary of an expert workshop (Kroese, M. Elles, R.,Zimmern), 2007. CETT program mentioned. • Best Practice guidelines and UKGTN website • Prence (1999) Validation of genetic tests Genetic testing