1 / 22

Seminar Topics and Projects

Explore various Deep Learning projects in NLP including tokenizer, POS tagging, sentiment analysis, relation extraction, morphological analysis, and more. Learn valuable techniques and resources for implementing advanced NLP models.

daney
Download Presentation

Seminar Topics and Projects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Università di Pisa Seminar Topics and Projects Giuseppe Attardi Dipartimento di Informatica Università di Pisa

  2. Deep Learning Tokenizer • Depling 2016 challenge requires tokenizer for any of the Universal Dependency TreeBank • Build a DL tokenizer using Keras based on the approach of: • Basile, Valerio and Bos, Johan and Evang, KilianA General-Purpose Machine Learning Method for Tokenization and Sentence Boundary Detection (2013), http://gmb.let.rug.nl/elephant/

  3. Deep Learning POS for UD • Depling 2016 challenge requires tokenizer for any of the Universal Dependency TreeBank • Build a DL POS using CNN, for example a LSTM that uses word embeddings and possible charcater embeddings.

  4. Deep Learning Morph Analyzer • Depling 2016 challenge requires tokenizer for any of the Universal Dependency TreeBank • Build a DL morphological analyzer that copmutes the morphology of each word, using Keras and charcaher embeddings.

  5. UD extensions • Write scripts to extract additional relations from the analysis of UD parse trees

  6. Convolutional Networks for Sentiment Analysis • Annotated Data: SemEval training set • Unannotated Data: 50 million tweets • Code: DeepNL, https://github.com/attardi/deepnl • Article: A. Severyn, A. Moschitti.UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification

  7. POS tagging using Word Embeddings • Data: Evalita 2016 • Embeddings: http://tanl.di.unipi.it/embeddings/ • Article: Stratos, M. Collins. Simple Semi-Supervised POS Tagging.http://www.cs.columbia.edu/~stratos/research/naacl15semipos.pdf

  8. Negation/Speculation Extraction • Determine the scope of negative or speculative statements: • The lyso-platelet had no effect • MnlI-AluIcould suppress the basal-level activity • Approach: • Classifier for identifying cues • Classifier to determine scope • Data • BioScope collection

  9. Corpus of Product Reviews • Download reviews from online shops • Classify as positive/negative according to stars • Train classifier to assign score

  10. Relation Extraction • Exploit word embeddings as features + extra hand-coded features • Use the Factor Based Compositional Embedding Model (FCM)http://www.cs.jhu.edu/~mrg/publications/finere-naacl-2015.pdf • SemEval 2014 Relation Extraction data

  11. Entity Linking with Embeddings • Experiment with technique: R. Blanco, G. Ottaviano, E. Meiji. 2014. Fast and Space-Efficient Entity Linking in Queries. labs.yahoo.com/_c/uploads/WSDM-2015-blanco.pdf

  12. Extraction of Semantic Hierarchies • Use word embeddings as measure of semantic distance • Use Wikipedia as source of text • http://ir.hit.edu.cn/~jguo/papers/acl2014-hypernym.pdf Organism Plant Ranuncolacee Aconitum

  13. Suggested Topics for Seminars

  14. Neural Reasoning • B. Peng, Z. Lu, H. Li, K.F. WongToward Neural Network-based Reasoning • A. Kumar et al.Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

  15. Question Answering • Bowl Competition (QANTA vs Jennings) • https://www.youtube.com/watch?v=kTXJCEvCDYk • Iyyer et al. 2014: A Neural Network for Factoid Question Answering over Paragraphs • IBM Watson: • http://www.aaai.org/Magazine/Watson/watson.php • TAC: • http://www.nist.gov/tac/2008/qa/index.html

  16. Image Understanding • H. Y. Gao et al. Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering, NIPS, 2015.

  17. Deep Learning Applications • Character RNNs on text and code • http://karpathy.github.io/2015/05/21/rnn-effec8veness/ • Morphology • Better Word Representations with Recursive Neural Networks for Morphology – Luong et al. • Polysemous words • Improving Word Representa8ons Via Global Context And Multiple Word Prototypes by Huang et al. 2012 • Natural language Inference (Logic) • Question Answering • Image – Sentence mapping

  18. Entity Linking • Entity Kierarchy Embeddings • http://www.cs.cmu.edu/~zhitingh/data/acl15entity.pdf

  19. Deep Learning tsunami over NLP • C. Manning. 2015. http://www.mitpressjournals.org/doi/pdf/10.1162/COLI_a_00239

  20. Opinion Mining • B. Liu. Sentiment Analisis and Subjectivity. 2010. Handbook of NLP. http://www.cs.uic.edu/~liub/FBS/NLP-handbook-sentiment-analysis.pdf

  21. Semantic Role Labeling • http://ufal.mff.cuni.cz/conll2009-st/task-description.html

  22. DL for NLP • Neural Machine Translation • D. Bahdanau, K. Cho, Y. Bengio. Neural machine translation by jointly learning to align and translate.http://arxiv.org/pdf/1409.0473v6 • Natural Language from scratch • Zhang, X., & LeCun, Y. (2015). Text Understanding from Scratch.http://arxiv.org/abs/1502.01710

More Related