150 likes | 303 Views
Given two strings S 1 of length m and S 2 of length n over the same alphabeth. The Longest Common Substring problem is to find the longest substring of S 1 that is also a substring of S 2 . A generalization is the k-common substring problem .
E N D
Given two strings S1 of length m and S2 of length n • over the same alphabeth. The Longest Common • Substring problem is to find the longest substring of S1 • that is also a substring of S2. • A generalization is the k-common substring problem. • Given the set of strings S={S1,S2,……………,Sk}. • where • |Si|=ni . • Σ ni=N. • Find for each 2 ≤ k ≤ K , the longest string which occur • as substring of all strings. LCS
1. Brute Force Technique 2. Dynamic Programming 3. Suffix Tree LCS
ABABC BABCA ABCBA LCS:= ABC Time Complexity =O(n^3) LCS
LCS(S[1…m-1],T[1…n-1])+1 If S[m]=T[n] LCS(S[1...m],T[1…n]) = 0 Otherwise LCS
j i Time Complexity = O( m n) Longest Common Substring Longest Common Substring LCS
String 1=abbc$ String 2=babc# $ a c # b b a b b b c c c c c # # # $ $ $ LCS
Cost Effective Way to Determining Two or More Documents are near Duplicates of one another LCS compares two strings and finds the longest run of characters that occurs in both of them. We can then declare the two documents as near duplicates if the ratio of the common substring length to the length of the documents exceeds some threshold. Consider the Example BelowSelling a beautiful house in California.Buying a beautiful chip in California.The longest common substring is " in California." (it is 15 characters long, whereas " a beautiful " comes in second at 13 characters long). The first string is 40 characters long. So, you could assess how similar the strings are by taking the ratio: 15/40 = 0.375. Best part about this application is that user can decide the threshold level interactively Target Audience of this Application *Ideal for Universities which do not have access to turn it in. *Students who do not have access to turn it . LCS
Patient Matching Medical record linkage is becoming increasingly important as clinical data is distributed across independent sources. Two corresponding fields within a record are said to agree only if all characters match; otherwise the fields are considered as mismatches. LCS score for the names ‘TAMMY SHACKELFORD’ ‘TAMMIE SHACKLEFORD’ The total length of the common substrings is [5 (SHACK) + 4 (TAMM) + 4 (FORD)] = 13. The length of the shorter name string (ignoring white space) is 16, therefore the LCS score is (13÷16) = 0.8125 LCS
1000 Inputs 10000 Inputs LCS
In Dynamic programming following changes can be done to exiting algorithm to reduce the memory usage of an implementation :- • Keep only the last and current row of the Dynamic Programming table to save memory O(min(m, n)) instead of O(n m)). • Store only non-zero values in the rows. This can be done using hash tables instead of arrays. This is useful for large alphabets. • Exiting Ukkonen’s suffix-tree implementation of longest common substring problem can be modified using McCreight and Weiner to see marginal improvement in time and space complexities. • Hybrid algorithm's performance can be compared with exiting performance results and see if there is any significant change in time and space complexity using rolling hash and suffix arrays LCS
Longest common substring problem http://en.wikipedia.org/wiki/Longest_common_substring_problem#See_also On–line construction of Suffix trees http://www.cs.helsinki.fi/u/ukkonen/SuffixT1withFigs.pdf Generalized suffix tree http://en.wikipedia.org/wiki/Generalized_suffix_tree Real World Performance of Approximate String Comparators for use in Patient Matching http://www.cs.mun.ca/~harold/Courses/Old/CS6772.F04/Diary/5604Grannis.pdf LCS