1 / 19

Word sense disambiguation and information retrieval

Word sense disambiguation and information retrieval. Chapter 17 Jurafsky, D. & Martin J. H. SPEECH and LANGUAGE PROCESSING Jarmo Ritola - jarmo.ritola@hut.fi. Lexical Semantic Processing. Word sense disambiguation which sense of a word is being used non-trivial task robust algorithms

ulla
Download Presentation

Word sense disambiguation and information retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Word sense disambiguation and information retrieval Chapter 17 Jurafsky, D. & Martin J. H. SPEECH and LANGUAGE PROCESSING Jarmo Ritola - jarmo.ritola@hut.fi

  2. Lexical Semantic Processing • Word sense disambiguation • which sense of a word is being used • non-trivial task • robust algorithms • Information retrieval • broad field • storage and retrieval of requested text documents • vector space model

  3. Word Sense Disambiguation • (17.1) “..., everybody has a career and none of them includes washing DISHES” • (17.2) “In her tiny kitchen at home, Ms. Chen works efficiently, stir-frying several simple DISHES, including braised pig’s ears and chicken livers with green peppers” • (17.6) “I’m looking for a restaurant that SERVES vegetarian DISHES”

  4. Selectional Restriction • Rule-to-rule approach • Blocks the formation of representations with selectional restriction violations • Correct sense achieved as side effect • PATIENT roles, mutual exclution • dishes + stir-fry => food sense • dishes + wash => artifact sense • Need: hierarchical types and restrictions

  5. S.R. Limitations • Selectional restrictions too general • (17.7) … kind of DISHES do you recommend? • True restriction violations • (17.8) …you can’t EAT gold for lunch… • negative environment • (17.9) … Mr. Kulkarni ATE glass … • Metaphoric and metonymic uses • Selectional association (Resnik)

  6. Robust Word Sense Disambiguation • Robust, stand alone systems • Preprocessing • part-of-speech tagging, context selection, stemming, morphological processing, parsing… • Feature selection, feature vector • Train classifier to assign words to senses • Supervised, bootstrapping, unsupervised • Does the system scale?

  7. Inputs: Feature Vectors • Target word, context • Select relevant linguistic features • Encode them in a usable form • Numeric or nominal values • Collocational features • Co-occurrence features

  8. Inputs: Feature Vectors (2) • (17.11) An electric guitar and BASS player stand off to one side, not really part of the scene, just as a sort of nod to gringo expectations perhaps. • Collocational [guitar, NN1, and, CJC, player, NN1, stand, VVB] • Co-occurrence [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0] fishing, big, sound, player, fly, rod, pound, double, runs, playing, guitar, band

  9. Supervised Learning • Feature-encoded inputs + categories • Naïve Bayes classifier • Decision list classifiers • case statements • tests ordered according to sense likelihood

  10. Bootstrapping Approaches • Seeds, small number of labeled instances • Initial classifier extracts larger training set • Repeat => series of classifier with improving accuracy and coverage • Hand labeling examples • One sense per collocation • Also automatic selection from machine readable dictionary

  11. Unsupervised Methods • Unlabeled feature vectors are grouped into clusters according to a similarity metric • Clusters are labeled by hand • Agglomerative clustering • Challenges • correct senses may not be known • heterogeneous clusters • Number clusters and senses differ

  12. Dictionary Based Approaches • Large-scale disambiguation possible • Sense definitions retrieved from the dictionary • The sense with highest overlap within context words • Dictionary entries relative short • Not enough overlap • expand word list, subject codes

  13. Information Retrieval • Compositional semantics • Bag of words methods • Terminology • document • collection • term • query • Ad hoc retrieval

  14. The Vector Space Model • List of terms within the collection • document vector: presence/absence of terms • raw term frequency • normalization => direction of vector • similarity is cosine of angle between vectors

  15. The Vector Space Model • Document collection • Term by weight matrix

  16. Term Weighting • Enormous impact on the effectiveness • Term frequency within a single document • Distribution of term across a collection • Same weighting scheme for documents and queries • Alternative weighting methods for queries • AltaVista: di,j contains 1’000’000’000 words • average query: 2.3 words

  17. Recall versus precision • Stemming • Stop list • Homonymy, polysemy, synonymy, hyponymy • Improving user queries • relevance feedback • query expansion, thesaurus, thesaurus generation, term clustering

  18. Summary • WSD: assign word to senses • Selectional restriction • Machine learning approaches (small scale) • supervised, bootstrapping, unsupervised • Machine readable dictionaries (large scale) • Bag of words method, Vector space model • Query improvement (relevance feedback)

  19. Exercise - Relevance Feedback The document collection is ordered according to the 'raw term frequency' of words "speech" and "language". The values and ordering is shown in the table below. • You want to find documents with many "speech" words but few "language" words (e.g. relation 8 : 2). Your initial query is {"speech", "language"}, i.e. they have equal weights. • The search machine always returns three most similar documents. • Show that with relevance feedback you • get the documents you want. • How important is the correctness of • feedback from the user?

More Related