1 / 31

EGASP 2005 Evaluation Protocol

EGASP 2005 Evaluation Protocol. Paul Flicek EBI. Basics. The evaluations are probably wrong GTF is not standard There are hidden assumptions Filters, overlaps, clusters Terminology varies Genes, exons, etc. Evaluation Measures. Exons and introns Sensitivity (Sn) Specificity (Sp)

ginger-mcdowell
Download Presentation

EGASP 2005 Evaluation Protocol

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EGASP 2005EvaluationProtocol Paul Flicek EBI

  2. Basics • The evaluations are probably wrong • GTF is not standard • There are hidden assumptions • Filters, overlaps, clusters • Terminology varies • Genes, exons, etc. EGASP2005Evaluations

  3. Evaluation Measures • Exons and introns • Sensitivity (Sn) • Specificity (Sp) • Exon length • Exons per transcript • Transcript • Sn / Sp • Overlap • Gene • Sn / Sp EGASP2005Evaluations

  4. Definitions EGASP2005Evaluations

  5. Definitions • Positive Transcript • Correct translation start • Correct translation stop • Every splice site correct • Positive Gene • At least one positive transcript EGASP2005Evaluations

  6. Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0 Gene Sp = 1.0 Prediction EGASP2005Evaluations

  7. Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0 Gene Sp = 1.0 Prediction EGASP2005Evaluations

  8. Examples Annotation Trans Sn = 0.0 Trans Sp = 0.0 Gene Sn = 0.0 Gene Sp = 0.0 Prediction EGASP2005Evaluations

  9. Examples Annotation Trans Sn = 1.0 Trans Sp = 1.0 Gene Sn = 1.0 Gene Sp = 1.0 Prediction EGASP2005Evaluations

  10. Examples Annotation Trans Sn = 0.5 Trans Sp = 0.5 Gene Sn = 1.0 Gene Sp = 1.0 Prediction EGASP2005Evaluations

  11. Examples Annotation Trans Sn = 1.0 Trans Sp = 0.67 Gene Sn = 1.0 Gene Sp = 1.0 Prediction EGASP2005Evaluations

  12. The winners are…(there are clear trends) • The most successful programs use expressed sequences • Programs using evolutionary conservation are more successful than those that do not • Exon and nucleotide measures are similar • We are improving EGASP2005Evaluations

  13. Spear Catching Time EGASP2005Evaluations

  14. EGASP 2005EvaluationsBlock 1 Paul Flicek EBI Expressed Sequence Methods

  15. Nucleotide EGASP2005Evaluations

  16. Exon EGASP2005Evaluations

  17. Intron EGASP2005Evaluations

  18. Gene EGASP2005Evaluations

  19. Number of Genes 1027 1389 EGASP2005Evaluations

  20. Unique Exons EGASP2005Evaluations

  21. Summary EGASP2005Evaluations

  22. EGASP 2005EvaluationsBlock 2 Paul Flicek EBI Evolutionary Conservation (Dual/Multiple Genome) Methods

  23. Summary EGASP2005Evaluations

  24. EGASP 2005EvaluationsBlock 3a Paul Flicek EBI Ab initio (single genome) and Exon only Methods

  25. Summary EGASP2005Evaluations

  26. EGASP 2005EvaluationsBlock 3b Paul Flicek EBI Open (Any) Methods

  27. Summary EGASP2005Evaluations

More Related