1 / 19

Sequence encoding, Cross Validation Morten Nielsen BioSys, DTU

Sequence encoding, Cross Validation Morten Nielsen BioSys, DTU. Outline. Sequence encoding How to represent biological data Overfitting cross-validation Method evaluation. Sequence encoding. Encoding of sequence data Sparse encoding Blosum encoding Sequence profile encoding

raven-turner
Download Presentation

Sequence encoding, Cross Validation Morten Nielsen BioSys, DTU

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sequence encoding, Cross ValidationMorten NielsenBioSys, DTU

  2. Outline • Sequence encoding • How to represent biological data • Overfitting • cross-validation • Method evaluation

  3. Sequence encoding • Encoding of sequence data • Sparse encoding • Blosum encoding • Sequence profile encoding • Reduced amino acid alphabets

  4. Sparse encoding Inp Neuron 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 AAcid A 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 R 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 N 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 C 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Q 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 E 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0

  5. BLOSUM encoding (Blosum50 matrix) A R N D C Q E G H I L K M F P S T W Y V A 4 -1 -2 -2 0 -1 -1 0 -2 -1 -1 -1 -1 -2 -1 1 0 -3 -2 0 R -1 5 0 -2 -3 1 0 -2 0 -3 -2 2 -1 -3 -2 -1 -1 -3 -2 -3 N -2 0 6 1 -3 0 0 0 1 -3 -3 0 -2 -3 -2 1 0 -4 -2 -3 D -2 -2 1 6 -3 0 2 -1 -1 -3 -4 -1 -3 -3 -1 0 -1 -4 -3 -3 C 0 -3 -3 -3 9 -3 -4 -3 -3 -1 -1 -3 -1 -2 -3 -1 -1 -2 -2 -1 Q -1 1 0 0 -3 5 2 -2 0 -3 -2 1 0 -3 -1 0 -1 -2 -1 -2 E -1 0 0 2 -4 2 5 -2 0 -3 -3 1 -2 -3 -1 0 -1 -3 -2 -2 G 0 -2 0 -1 -3 -2 -2 6 -2 -4 -4 -2 -3 -3 -2 0 -2 -2 -3 -3 H -2 0 1 -1 -3 0 0 -2 8 -3 -3 -1 -2 -1 -2 -1 -2 -2 2 -3 I -1 -3 -3 -3 -1 -3 -3 -4 -3 4 2 -3 1 0 -3 -2 -1 -3 -1 3 L -1 -2 -3 -4 -1 -2 -3 -4 -3 2 4 -2 2 0 -3 -2 -1 -2 -1 1 K -1 2 0 -1 -3 1 1 -2 -1 -3 -2 5 -1 -3 -1 0 -1 -3 -2 -2 M -1 -1 -2 -3 -1 0 -2 -3 -2 1 2 -1 5 0 -2 -1 -1 -1 -1 1 F -2 -3 -3 -3 -2 -3 -3 -3 -1 0 0 -3 0 6 -4 -2 -2 1 3 -1 P -1 -2 -2 -1 -3 -1 -1 -2 -2 -3 -3 -1 -2 -4 7 -1 -1 -4 -3 -2 S 1 -1 1 0 -1 0 0 0 -1 -2 -2 0 -1 -2 -1 4 1 -3 -2 -2 T 0 -1 0 -1 -1 -1 -1 -2 -2 -1 -1 -1 -1 -2 -1 1 5 -2 -2 0 W -3 -3 -4 -4 -2 -2 -3 -2 -2 -3 -2 -3 -1 1 -4 -3 -2 11 2 -3 Y -2 -2 -2 -3 -2 -1 -2 -3 2 -1 -1 -2 -1 3 -3 -2 -2 2 7 -1 V 0 -3 -3 -3 -1 -2 -2 -3 -3 3 1 -2 1 -1 -2 -2 0 -3 -1 4

  6. Sequence encoding (continued) • Sparse encoding • V:0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 • L:0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 • V.L=0 (unrelated) • Blosum encoding • V: 0 -3 -3 -3 -1 -2 -2 -3 -3 3 1 -2 1 -1 -2 -2 0 -3 -1 4 • L:-1 -2 -3 -4 -1 -2 -3 -4 -3 2 4 -2 2 0 -3 -2 -1 -2 -1 1 • V.L = 0.88 (highly related) • V.R = -0.08 (close to unrelated)

  7. Sequence encoding (continued) • Each amino acids is encoded by 20 variables • This might be highly ineffective • Can this number be reduced without losing predictive performance? • Use reduced amino acid alphabet • Charge, volume, hydrophobicity, Chemical descriptors • Appealing, but in my experience it does not work -)

  8. Evaluation of predictive performance • A prediction method contains a very large set of parameters • A matrix for predicting binding for 9meric peptides has 9x20=180 weights • Over fitting is a problem Temperature years

  9. Evaluation of predictive performance ALAKAAAAM ALAKAAAAN ALAKAAAAR ALAKAAAAT ALAKAAAAV GMNERPILT GILGFVFTM TLNAWVKVV KLNEPVLLL AVVPFIVSV MRSGRVHAV VRFNIDETP ANYIGQDGL AELCGDPGD QTRAVADGK GRPVPAAHP MTAQWWLDA FARGVVHVI LQRELTRLQ AVAEEMTKS • Train PSSM on raw data • No pseudo counts, No sequence weighting • Fit 9*20 parameters to 9*10 data points • Evaluate on training data • PCC = 0.97 • AUC = 1.0 • Close to a perfect prediction method Binders None Binders

  10. Evaluation of predictive performance AAAMAAKLA AAKNLAAAA AKALAAAAR AAAAKLATA ALAKAVAAA IPELMRTNG FIMGVFTGL NVTKVVAWL LEPLNLVLK VAVIVSVPF MRSGRVHAV VRFNIDETP ANYIGQDGL AELCGDPGD QTRAVADGK GRPVPAAHP MTAQWWLDA FARGVVHVI LQRELTRLQ AVAEEMTKS • Train PSSM on Permuteddata • No pseudo counts, No sequence weighting • Fit 9*20 parameters to 9*10 data points • Evaluate on training data • PCC = 0.97 • AUC = 1.0 • Close to a perfect prediction method AND • Same performance as one the original data Binders None Binders

  11. Repeat on large training data (229 ligands)

  12. Cross validation Train on 4/5 of data Test/evaluate on 1/5 => Produce 5 different methods each with a different prediction focus

  13. Method evaluation • Use cross validation • Evaluate on concatenated data and not as an average over each cross-validated performance • And even better, use an external evaluation set, that is not part of the training data

  14. Method evaluation • How is an external evaluation set evaluated on a 5 fold cross-validated training? • The cross-validation generates 5 individual methods • Predict the evaluation set on each of the 5 methods and take average

  15. Method evaluation

  16. Method evaluation

  17. 5 fold training Which PSSM to choose?

  18. 5 fold training. Use them all

  19. Summary • Define with encoding scheme to use • Sparse, Blosum, reduced alphabet, and combinations of these • Deal with over-fitting • Evaluate method using cross-validation • Evaluate external data using method ensemble

More Related