Textual Spatial Cosine Similarity

1. Textual Spatial Cosine Similarity Giancarlo CrocettiPace University Seidenberg School of CSIS

2. Introductin Similarity is a quantifiable measure of how similar two objects are We have many document similarity measures today Cosine Similarity is widely used and is considered a standard in search engines. Cosine Similarity has a serious drawback: does not consider word placement

3. An Example • Compare “John loves Mary” with “Mary loves John” simcosine(“John loves Mary”, ”Mary loves John”) = 1.0 • Definitely similar, but they are not the same • Methods based on NLP exists, but computationally intensive I will introduce a Textual Space Similaritythat provides Semantic-Quality resultswithout the overhead of semantic approaches.

4. Textual Space Similarity • Different from other positional-based methods. Given two documents i and j, let’s define the Spatial Difference for the term t the quantity: • Each term can be 0 when in the same position • 1 when the first instance of a term is at the beginning of one document. • Otherwise 0<<1

5. Textual Space Similarity (continued) Finally, we define the Textual Space Similarity of two documents di and dj the quantity: With l the number of matching terms in the two documents. • Numerator is the summation of quantities [0,1] appearing no more than l times, therefore TSS [0,1] • In order for TSS to have the same direction of other document similarities: [0,1]

6. Back to the Example = TSS(“John loves Mary”, “Mary loves John”) = This result is quite different from the cosine similarity of 1.0

7. Textual Spatial Cosine Similarity In order to both spatial and term features in documents we combine the cosine similarity with TSS deriving the Textual Spatial Cosine Similarity defined as: • a=1: = sim • a=0: = TSS • In general a=0.5 works well in most cases =  * sim+(1- )*TSS

8. TSCS and Corpus Size • Similarity is a value [0,1] and it is not clear what is the threshold to use to assert two documents are “similar” • Cosine similarity varies with changes in corpus size • We ran an experiment to see how the similarity of two seeded document varies with changes in corpus size (a=0.5) Similarity variations with different corpus sizes using Cosine Similarity variations with different corpus sizes using TSCS

9. TSCS and Paraphrasing • The dataset consisted of 734 English pairs drawn from publicly available datasets: • Microsoft Research Paraphrase Corpus • Microsoft Research Video Description Corpus • WMT2008 development dataset • We analyzed the TSCS performance in detecting paraphrases, by using different values of alpha

10. TSCS and Paraphrasing (continued) • Number of correct detection maximized with a=0 • TSCS recognized a total of 649 paraphrases • TSCS (in its degenerate case of a=0) achieved an accuracy of 649/734 = 0.8842 • TSCSa=0 = TSS can be adopted in the detection of paraphrasing

11. Conclusions • Textual Space Cosine Similarity (TSCS) adds a spatial dimension without computational intensive, semantic approaches • TSCS is minimally sensitive to changes in the corpus size • In its degenerative case can be used as a model for paraphrasing detection with accuracy levels close to 90%. • TSCS can be used by search engines in: • Detection of plagiarism • Content recommendation • Content Discovery

12. Thank You Giancarlo Crocetti – gcrocetti@pace.edu