1 / 36

Information Retrieval

This lecture introduces the concepts of Boolean search and ranking techniques in information retrieval. It explores the strengths and weaknesses of Boolean search and discusses the importance of ranking documents based on their relevance to a query. The lecture also covers term weighting techniques such as term frequency and inverse document frequency.

rdarlene
Download Presentation

Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Information Retrieval For the MSc Computer Science Programme Lecture 2 Introduction to Information Retrieval (Manning et al. 2007) Chapter 6 & 7 Dell Zhang Birkbeck, University of London

  2. Boolean Search • Strength • Docs either match or not. • Good for expert users with precise understanding of their needs and the corpus. • Weakness • Not good for (the majority of) users with poor Boolean formulation of their needs. • Applications may consume 1000’s of results, but most users don’t want to wade through 1000’s of results – cf. use of Web search engines.

  3. Beyond Boolean Search • Solution: Ranking • We wish to return in order the documents most likely to be useful to the searcher. • How can we rank/order the docs in the corpus with respect to a query? • Assign a score – say in [0,1]– for each doc on each query.

  4. Document Scoring • Idea: More is Better • If a document talks about a topic more, then it is a better match. • That is to say, a document is more relevant if it has morerelevant terms. • This leads to the problem of term weighting.

  5. Bag-Of-Words (BOW) Model • Term-Document Count Matrix • Each document corresponds to a vector in ℕv, i.e., a column below. The matrix element A(i,j) is the number of occurrences of the i-th term in the j-th doc.

  6. Bag-Of-Words (BOW) Model • Simplification • In the BOW model, • the doc • John is quicker than Mary. • is indistinguishable from the doc • Mary is quicker than John.

  7. Term Frequency (TF) • Digression: Terminology • WARNING: In a lot of IR literature, “frequency” is used to mean “count”. • Thus term frequency in IR literature is used to mean the number of occurrencesof a term in a document not divided by document length (which would actually make it a frequency). • We will conform to this misnomer: in saying term frequency we mean the number of occurrences of a term in a document.

  8. Term Frequency (TF) • What is the relative importance of • 0 vs. 1 occurrence of a term in a doc, • 1 vs. 2 occurrences, • 2 vs. 3 occurrences, ……? • Can just use raw tf . • While it seems that more is better, a lot isn’t proportionally better than a few. • So another option commonly used in practice:

  9. Term Frequency (TF) • The score of a document d for a query q • 0 if no query terms in document • wfcan be used instead of tf in the above

  10. Term Frequency (TF) • Is TF good enough for weighting? • Ignorance of document length • Long docs are favored because they’re more likely to contain query terms. • This can be fixed to some extent by normalizing for document length. [talk later]

  11. Term Frequency (TF) • Is TF good enough for weighting? • Ignorance of term rarity in corpus • Consider the query ides of march. • Julius Caesar has 5 occurrences of ides , while no other play has ides . • march occurs in over a dozen. • All the plays contain of . • By this weighting scheme, the top-scoring play is likely to be the one with the most ofs.

  12. Document/Collection Frequency • Which of these tells you more about a doc? • 5 occurrences of of? • 5 occurrences of march? • 5 occurrences of ides? • We’d like to attenuate the weight of a common term. But what is “common”? • Collection Frequency (CF) • the number of occurrences of the term in the corpus • Document Frequency (DF) • the number of docs in the corpus containing the term

  13. Document/Collection Frequency • DF may be better than CF WordCFDF try10422 8760 insurance10440 3997 So how do we make use of DF?

  14. Inverse Document Frequency (IDF) • Could just be the reciprocal of DF (idfi = 1/dfi). • But by far the most commonly used version is:

  15. Inverse Document Frequency (IDF) • Prof Karen Spark Jones 1935-2007

  16. TFxIDF • TFxIDFweighting scheme combines: • Term Frequency (TF) • measure of term density in a doc • Inverse Document Frequency (IDF) • measure of informativeness of a term: its rarity across the whole corpus

  17. TFxIDF • Each term i in each document d is assigned a TFxIDF weight • Increases with the number of occurrences within a doc. • Increases with the rarity of the term across the whole corpus. What is the weight of a term thatoccurs in allof the docs?

  18. Term-Document Matrix (Real-Valued) The matrix element A(i,j) is the log-scaled TFxIDF weight. Note: can be > 1.

  19. Vector Space Model • Docs Vectors • Each doc j can now be viewed as a vector of TFxIDF values, one component for each term. • So we have a vector space • Terms are axes • Docs live in this space • May have 20,000+ dimensions • even with stemming

  20. Vector Space Model • Prof Gerard Salton 1927-1995 The SMART information retrieval system

  21. t3 d2 d3 d1 θ φ t1 d5 t2 d4 Vector Space Model • First application: Query-By-Example (QBE) • Given a doc d, find others “like” it. • Now that d is a vector, find vectors (docs) “near” it. Postulate: Documents that are “close together” in the vector space talk about the same things.

  22. Vector Space Model • QueriesVectors • Regard a query as a (very short) document. • Return the docs ranked by the closeness of their vectors to the query, also represented as a vector.

  23. Desiderata for Proximity • If d1 is near d2, then d2 is near d1. • If d1 near d2, and d2 near d3, then d1 is not far from d3. • No doc is closer to d than d itself.

  24. Euclidean Distance • Distance between dj and dk is • Why is this not a great idea? • We still haven’t dealt with the issue of length normalization: long documents would be more similar to each other by virtue of length, not topic. • However, we can implicitly normalize by looking at angles instead.

  25. Cosine Similarity • Vector Normalization • A vector can be normalized (given a length of 1) by dividing each of its components by its length – here we use the L2 norm • This maps vectors onto the unit sphere. • Then longer documents don’t get more weight.

  26. Cosine Similarity • Cosine of angle between two vectors The denominator involves the lengths of the vectors. This means normalization.

  27. t 3 d 2 d 1 θ t 1 t 2 Cosine Similarity • The similarity between dj and dk is captured by the cosine of the angle between their vectors. No triangle inequality for similarity.

  28. Cosine Similarity - Exercise • Rank the following by decreasing cosine similarity: • Two docs that have only frequent words (the, a, an, of) in common. • Two docs that have no words in common. • Two docs that have many rare words in common (wingspan, tailfin).

  29. Cosine Similarity - Exercise • Show that, for normalized vectors, Euclidean distance measure gives the same proximity ordering as the cosine similarity measure.

  30. Cosine Similarity - Example • Docs • Austen's Sense and Sensibility (SaS) • Austen's Pride and Prejudice (PaP) • Bronte's Wuthering Heights (WH) cos(SaS, PaP) = 0.996 x 0.993 + 0.087 x 0.120 + 0.017 x 0.000 = 0.999 cos(SaS, WH) = 0.996 x 0.847 + 0.087 x 0.466 + 0.017 x 0.254 = 0.889

  31. Vector Space Model - Summary • What’s the real point of using vector space? • Every query can be viewed as a (very short) doc. • Every query becomes a vector in the same space as the docs. • Can measure each doc’s proximity to the query. • It provides a natural measure of scores/ranking – no longer Boolean. • Docs (and queries) are expressed as bags of words.

  32. Vector Space Model - Exercise • How would you augment the inverted index built in previous lectures to support cosine ranking computations? • Walk through the steps of serving a query using the Vector Space Model.

  33. Efficient Cosine Ranking • Computing a single cosine • For every term t, with each doc d, Add tft,d to postings lists. • Some tradeoffs on whether to store term count, term weight, or weighted by IDF. • At query time, accumulate component-wise sum.

  34. Efficient Cosine Ranking • Computing the k largest cosines • Search as a kNN problem • Find the k docs “nearest” to the query (with largest query-doc cosines) in the vector space. • Do not need to totally order all docs in the corpus. • Use heap for selecting top k docs • Binary tree in which each node’s value > values of children • Takes 2n operations to construct, then each of k“winners” read off in with logn steps. • For n=1M, k=100, this is about 10% of the cost of complete sorting.

  35. Efficient Cosine Ranking • Heuristics • Avoid computing cosines from query to each of n docs, but may occasionally get an answer wrong. • For example, cluster pruning.

  36. Take Home Messages • TFxIDF • Vector Space Model • docs and queries as vectors • cosine similarity • efficient cosine ranking

More Related