1 / 7

Exercising these ideas

Exercising these ideas. You have a description of each item in a small collection. (30 web sites) Assume we are looking for information about boxers, a type of dog. List the items that are relevant to this information need. (If it is impossible to tell, as with #9, mark it not relevant.)

ronan-austin
Download Presentation

Exercising these ideas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exercising these ideas • You have a description of each item in a small collection. (30 web sites) • Assume we are looking for information about boxers, a type of dog. • List the items that are relevant to this information need. (If it is impossible to tell, as with #9, mark it not relevant.) • Rq = { } • Assume that an unspecified search system has returned a ranked result as follows: • (1, 4, 6, 8, 10, 12, 15, 16, 17, 18, 20, 23,26, 28, 30) • Show the ranking for the query as done on page 76. • Plot the precision vs. recall curve

  2. Exercise, continued • We do a second query against the same collection. This time we are interested in the Boxer rescue organizations. • Rq = { } • Ranked result = (4, 6, 7, 10, 11, 15, 16, 17, 18, 19, 22, 23, 25, 28, 29) • Calculate the average precision at recall level 3 for these two queries.

  3. More exercise • We redo the first query, using a new search algorithm. The ranked result is now this: • (4, 5, 6, 7, 8, 10, 11, 15, 16, 17, 18, 20, 23, 28, 29) • Produce the average recall vs. precision figures for these two algorithms. How would you describe the performance? • We redo the second query using the new search algorithm. The ranked result is this: • (7, 10, 11, 15, 17, 18, 19, 21, 23, 24, 26, 28, 29, 30)

  4. Precision Histogram

  5. For each of the four searches, what is the R-precision? • Use a precision histogram to compare these two algorithms for the two queries shown. • Calculate the harmonic mean at document 5 in the rankings, using the first query and repeating for each algorithm. • Calculate the E measure for algorithm 1 using query 1 and a moderate preference for recall rather than precision.

  6. |Rk| Coverage = ------ |U| |Ru| Novelty = --------------- |Ru| + |Rk| Collection Answer Set |A| Relevant docs |R| Relevant docs unknown to the User which were retrieved |Ru| Relevant docs known to the User |U| Relevant docs known to the User which were retrieved |Rk|

  7. Assume that the even number elements in the collection are known to the user. • Calculate the coverage ratio for algorithm 1 on search 1 • Calculate the novelty ratio for the same algorithm and search

More Related