1 / 63

Learning Semantic String Transformations from Examples

Learning Semantic String Transformations from Examples. Rishabh Singh and Sumit Gulwani. FlashFill. Transformations. Syntactic Transformations Concatenation of regular expression based substring “VLDB2012”  “VLDB” Semantic Transformations More than just characters

kylene
Download Presentation

Learning Semantic String Transformations from Examples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning Semantic String Transformations from Examples Rishabh Singh and SumitGulwani

  2. FlashFill

  3. Transformations • Syntactic Transformations • Concatenation of regular expression based substring • “VLDB2012”  “VLDB” • Semantic Transformations • More than just characters • “1/5/2010”  “May 1st 2010”

  4. Semantic Transformations • Semantic information as relational tables • 1  January, 2  February • Learn table lookup queries • VLOOKUP macro 2nd most problematic

  5. Outline • Lookup Transformations • Lookup + Syntactic Transformations • Case Studies

  6. Demo Table Lookup Transformations

  7. Learning Framework Input Strings Output String F … F1 Fn 1. Domain-specific Language L 2. Algorithm to learn all Fs from (i,o)

  8. Lookup Transformation Language

  9. Example - Lookup Select(Name, EmpRecord, (SSN = v1))

  10. Example – Transitive Lookup Select(Price, PriceRec, (ItemId = Select(ItemId, ItemRec, Item = v1))

  11. Learn Query Select(Price, PriceRec, (ItemId = Select(ItemId, ItemRec, Item = v1))

  12. Synthesis Algorithm : • Input: (input state , output string ) • Output: all conforming expressions • Reachability algorithm from input strings

  13. Strings reachable from input row

  14. ) strings in table rows of visited nodes

  15. …….. Repeat until k steps or fixpoint

  16. ……..

  17. Sound and k-complete • t: number of reachable strings • p: number of candidate keys • m: maximum size of a candidate key

  18. Data structure • Maintains tree structure • share common sub-expressions • CNF of Boolean Conditionals • independent column predicates

  19. Synthesize Procedure Synthesize((i1,o1), …, (in,on)) P = GenerateStrt(i1,o1) for j = 2 to n: P’ = GenerateStrt(ij,oj) P = Intersectt(P’, P) return P

  20. Demo Semantic String Transformations

  21. Syntactic String Language [GulwaniPOPL11]

  22. Combined Language Syntactic manipulations over lookup outputs Syntactic manipulations before indexing

  23. Synthesis Algorithm: • Reachability based on syntactic string matches • Boolean conditionals

  24. { “SSN: 044-58-3429”, “044-58-3429”, “1125”, “Steve Russell” } Set of reachable strings

  25. { “SSN: 044-58-3429”, “044-58-3429”, “1125”, “Steve Russell” } and in paper

  26. Experiments • 50 benchmark problems • 12 , 38 • ~1020 consistent expressions • Size of data structure: ~2000 • Performance: 96% less than 1 second • Ranking: at most 3 examples (95% 2 examples)

  27. Related Work • Matching strings for table joins • Record Matching [Elmagarmid et. al. 07, Koudas et. al. SIGMOD06] • Schema Matching [Dhamankar et. al. SIGMOD04, Warren & Tompa VLDB06] • Query Synthesis • from representative view [Das Sharma et.al. ICDT10, Tran et.al. SIGMOD09] • Text-editing by example • QuickCode[Gulwani POPL11] • SMARTedit[Lau et.al. ML03], Simulatenous Editing[Miller et.al. USENIX01]

  28. Thanks! Algorithm Designers Software Developers End-Users Large potential

  29. Backup slides

  30. Semantic String Transformations =TEXT(C,”00 00”)+0

  31. Semantic String Transformations

  32. Idea 1: Share sub-expressions e Select(C2, T1, C1=v1) Select(C3, T2, C1=e) Select(C2, T3, C1=Select(C2,T2,C1=e)

  33. Youtube Videos French Polish Urdu German Serbian Russian http://bit.ly/flashfill

  34. Idea 2: CNF conditionals

  35. No. of Consistent Expressions

  36. Succinct Representation

  37. Performance

  38. Ranking

  39. Idea 2: CNF conditionals

  40. : string value : set of lookup programs to generate

  41. Related Work • Record Matching • Similarity functions for matching [Elmagarmid et. al. 07, Koudas et. al. SIGMOD06] • Customizable similarity function [Arasu et. al. VLDB09] • Learning Schema Matches • iMAP [Dhamankar et. al. SIGMOD04] concat. of column strings using domain-specific knowledge • [Warren & Tompa VLDB06] concatenation of column substrings, single table

  42. Related Work • Query Synthesis [Das Sharma et.al. ICDT10, Tran et.al. SIGMOD09] • Infer relation from large representative example view • no joins or projections • Text-editing using examples • QuickCode[Gulwani POPL11] string transformations • SMARTedit[Lau et.al. ML03], Simulatenous Editing[Miller et.al. USENIX01] programming by demonstration

  43. General Framework • A Domain-specific Transformation Language L • Expressive and succinct • Efficient Data structures for set of expressions • Version-space algebra • GenerateStr • All sets of expressions from I-O example • Intersect • Intersect two sets of expressions

  44. Example - Lookup Select(Name, EmpRecord, (SSN = v1))

  45. Example – Transitive Lookups Select(Price, PriceRec, (ItemId = Select(ItemId, ItemRec, Item = v1))

More Related