1 / 22

Q-Vals (and False Discovery Rates) Made Easy

Q-Vals (and False Discovery Rates) Made Easy. Dennis Shasha Based on the paper "Statistical significance for genomewide studies" by John Storey and Robert Tibshirani PNAS August 5, 2003 9440-9445. Challenge. You test plants/patients/… in two settings (or from different populations).

tate
Download Presentation

Q-Vals (and False Discovery Rates) Made Easy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Q-Vals (and False Discovery Rates) Made Easy Dennis Shasha Based on the paper "Statistical significance for genomewide studies" by John Storey and Robert Tibshirani PNAS August 5, 2003 9440-9445

  2. Challenge • You test plants/patients/… in two settings (or from different populations). • You want to know which genes are differentially expressed (alternate) • You don’t want to make too many mistakes (declaring a gene to be alternate when in fact it’s null – not differentially expressed).

  3. First Idea • You take p-vals of the differences in expression. • P-val(g) is the probability that if g is null, it would have a difference at least this large. • You choose a cutoff, say 0.05. • You say all genes that differ with p-val <= 0.05 are truly different. • What’s the problem?

  4. Thought Experiment • Suppose that no genes are truly differentially expressed. • You will conclude that about 5% of those you called significant really are. • Your false discovery rate (number null among those predicted to be alternate/number predicted to be alternate) = 100%. • Bad.

  5. A Fundamental Insight • All truly null genes (i.e. not truly differentially expressed) are equally likely to have any p-val. • That is by construction of p-val: under the null hypothesis, 1% of the genes will be in the top 1 percentile, 1% will be in percentile between 89 and 90th and so on. P-val is just a way of saying percentile in null condition.

  6. What Do We Do With That? • Mixture model: imagine null genes as light blue marbles and truly different genes as red ones. • If the assay is decent, red marbles should be concentrated at the low p-values.

  7. 0 …. Pval …………………………………………………1

  8. Method We Can Use • We don’t of course know the colors of the marbles/we don’t know which genes are true alternates. • However, we know that null marbles are equally likely to have any p-value. • So, at the p-value where the height of the marbles levels off, we have primarily light blue marbles/null genes. • Why?

  9. Flat region starts here Level of flat region 0 …. Pval …………………………………………………1

  10. Answer • Because if all genes/marbles were null, the heights would be about uniform. • Provided the reds are concentrated near the low p-vals, the flat regions will be primarily light blues.

  11. Example: all null • Consider the all null case. • All marbles are light blue. • False discovery rate in region to left of flat region is estimated number of white marbles (based on flat region)/number of marbles to left of flat region. • This will be close to 100%

  12. Flat region starts here Level of flat region 0 …. Pval …………………………………………………1

  13. Example: all non-null • Consider the all non-null case. • All marbles are red and they are highly skewed. • Flat region is essentially zero. • False discovery rate in region to left of flat region is estimated number of white marbles (based on flat region)/number of marbles to left of flat region. • This will be close to 0.

  14. Flat region starts here 0 …. Pval …………………………………………………1

  15. Example: mixed case • Get a distribution of p-values. • Find flat region. • Estimate number of nulls in the left-of-flat region by extending the flat line. • This gives the false discovery rate.

  16. Number of genes having pval Possible p-value threshold Flat line; base level of nulls 0 …. Pval ……………………………………………1

  17. Example: mixed case • What would you estimate the false discovery rate to be in the case that we declare the entire area to the left of the possible p-value threshold to be significant? • 10%, 25%, 50%?

  18. Number of genes having pval Possible p-value threshold Flat line; base level of nulls 0 …. Pval ……………………………………………1

  19. Obtaining q-values from False Discovery Rate • Suppose we order genes from least p-value to greatest. • That corresponds to one of these cartesian graphs. • The q-value of a gene having p-value p is exactly the False Discovery Rate if the declared significance region had a threshold of p.

  20. Number of genes having pval Q-value of a gene having this p-val is the FDR if this is the significance threshold. Flat line; base level of nulls 0 …. Pval ……………………………………………1

  21. Lessons for Research • Mushy p-values (large error bars/few replicates) may force us to the far left in order to get a low False Discovery Rate. • This may eliminate genes of interest. • If testing out a gene is not too expensive, then we can accept a higher False Discovery Rate – nothing magical about 0.01.

  22. Number of genes having pval Better p-values avoid loss of genes, for small FalseDiscovery Rate. Flat line; base level of nulls 0 …. Pval ……………………………………………1

More Related