1 / 49

S é mantique lexicale Vecteur conceptuels et TALN

S é mantique lexicale Vecteur conceptuels et TALN. Mathieu Lafourcade LIRMM - France www.lirmm.fr/~lafourca. Objectifs. Analyse sémantique Word Sense Disambiguation Text Indexing in IR Lexical Transfer in MT Conceptual vector Reminiscence Vector Models (Salton)

hubert
Download Presentation

S é mantique lexicale Vecteur conceptuels et TALN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sémantique lexicaleVecteur conceptuels et TALN Mathieu Lafourcade LIRMM - France www.lirmm.fr/~lafourca

  2. Objectifs Analyse sémantique Word Sense Disambiguation Text Indexing in IR Lexical Transfer in MT Conceptual vector Reminiscence Vector Models (Salton) Conceptual Models (Sowa) Concepts (et non des termes) Ensemble E choisi a priori (petit E) / par emergence (grand) Concepts interdépendants Propagation sur arbre d’analyse morpho-syntaxique (pas analyse de surface)

  3. Conceptual vectors An idea = a combination of concepts = a vector The Idea space = vector space A concept = an idea = a vector = combination of itself + neighborhood

  4. Conceptual vectors • Set of k basic concepts • Thesaurus Larousse = 873 concepts • A vector = a 873 uple • Encoding for each dimension C = 215 • Sense space = vector space + vector set

  5. Conceptual vectors • Example : cat • Kernel c:mammal, c:stroke <… mammal … stroke …> <… 0,8 … 0,8 … > • Augmented c:mammal, c:stroke, c:zoology, c:love … <… zoology … mammal… stroke … love …> <… 0,5 … 0,75 … 0,75 … 0,5 … > • + Iteration for neighborhood augmentation • Finer vectors

  6. Vector space • Basic conceptsare not independent • Sense space = Generator Space of a real k’ vector space (unknown) = Dim k’ <= k • Relative position of points 

  7. Our conceptual vectors Thesaurus • H : thesaurus hierarchy — K concepts Thesaurus Larousse = 873 concepts • V(Ci) : <a1, …, ai, … , a873> aj = 1/ (2 ** Dum(H, i, j)) 1/16 1/16 1/4 1 1/4 1/4 1/64 1/64 4 2 6

  8. Conceptual vectors Conceptc4: ‘PEACE’ peace conflict relations hierarchical relations society The world, manhood

  9. Conceptual vectors Term “peace” c4:’PEACE’

  10. exchange profit finance

  11. Conceptual vector distance • Angular Distance DA(x, y) = angle (x, y) • 0 <= DA(x, y) <= π • if 0 then collinear - same idea • if π/2 then nothing in common • if π then DA(x, -x) with -x as anti-idea of x x’ x  y

  12. Conceptual vector distance • Distance = acos(similarity) DA(x, y) = acos(((x1y1 + … + xnyn)/|x||y|)) DA(x, x) = 0 DA(x, y) = DA(y, x) DA(x, y) + DA(y, z)  DA(x, z) DA(0, 0) = 0 and DA(x, 0) = π/2 by definition DA(x, y) = DA(x, y) with . 0 DA(x, y) = π - DA(x, y) with .< 0 DA(x+x, x+y) = DA(x, x+y)  DA(x, y)

  13. Conceptual vector distance • Example • DA(tit, tit) = 0 • DA(tit, passerine) = 0.4 • DA(tit, bird) = 0.7 • DA(tit, train) = 1.14 • DA(tit, insect) = 0.62 tit = kind of insectivorous passerine …

  14. Conceptual lexicon • Set of (word, vector) = (w, )* • Monosemy word  1 meaning  1 vector (w, ) tit

  15. Conceptual lexiconPolyseme building • Polysemy word n meanings n vectors {(w, ), (w.1, 1) … (w.n, n) } bank

  16. Conceptual lexicon Polyseme building • (w) = (w.i)　 = .i • bank = • 1. Mound, • 3. river border, … • 2. Money institution • 3. Organ keyboard • 4. … bank Squash isolated meanings against numerous close meanings

  17. 1:DA(3,4) & (3+2) 3: DA(4,5) & (4+5) 2:(bank4) 5: DA(6,7)& (6+7) 4:(bank3) 6:(bank1) 7:(bank2) Conceptual lexicon Polyseme building • (w) = classification(w.i) bank

  18. 1:D(3,4), (3+2) 3:D(4,5), (4+5) 2:4 4:3 5:D(6,7), (6+7) 6:1 7:2 Lexical scope (w) = • LS(w) = LSt((w)) LSt((w)) = 1 if is a leaf LSt((w)) = (LS(1) + LS(2)) /(2-sin(D((w))) otherwise • (w) = t((w)) t((w)) = (w) if is a leaf t((w)) = LS(1)t(1) + LS(2)t(2) otherwise Can handle duplicated definitions

  19. Vector Statistics • Norm () • [0 , 1] * C (215=32768) • Intensity () • Norm / C • Usually  = 1 • Standard deviation (SD) • SD2 = variance • variance = 1/n * (xi - )2with as the arith. mean

  20. Vector Statistics • Variation coefficient (CV) CV = SD / mean No unity - Norm independent Pseudo Conceptual strength If A Hyperonym B  CV(A) > CV(B) (we don’t have  ) • vector « fruit juice » (N) MEAN = 527, SD = 973 CV = 1.88 • vector « drink » (N) MEAN = 443, SD = 1014 CV = 2.28

  21. Vector operations • Sum • V = X + Y  vi = xi + yi • Neutral element : 0 • Generalized to n terms : V =  Vi • Normalization of sum : vi /|V|* c Kind of mean

  22. Vector operations • Term to term product • V = X  Y  vi = xi * yi • Neutral element : 1 • Generalized to n terms V =  Vi Kind of intersection

  23. Vector operations • Amplification • V = X ^ n  vi = sg(vi) * |vi|^ n  V = V ^ 1/2 and n V = V ^ 1/n V  V = V ^ 2 if vi  0 • Normalization of ttm product to n terms V = nVi

  24. Vector operations • Product + sum • V = X  Y = ( X  Y ) + X + Y • Generalized n terms : V = n Vi +  Vi • Simplest request vector computation in IR

  25. Vector operations • Subtraction • V = X - Y  vi = xi- yi • Dot subtraction • V = X  Y  vi = max (xi- yi, 0) • Complementary • V = C(X)  vi = (1 - xi/c) * c • etc. Set operations

  26. Intensity Distance • Intensity of normalized ttm product • 0 ( (X  Y))  1 if |x| = |y| = 1 DI(X, Y) = acos(( X  Y)) • DI(X, X) = 0 and DI(X, 0) = π/2 DI(tit, tit) = 0 (DA = 0) DI(tit, passerine) = 0.25 (DA = 0.4) DI(tit, bird) = 0.58 (DA = 0.7) DI(tit, train) = 0.89 (DA = 1.14) DI(tit, insect) = 0.50 (DA = 0.62)

  27. Relative synonymy • SynR(A, B, C) — C as reference feature SynR(A, B, C) = DA(AC, BC) • DA(coal,night) = 0.9 • SynR(coal, night, color) = 0.4 • SynR(coal, night, black) = 0.35

  28. Relative synonymy • SynR(A, B, C) = SynR(B, A, C) • SynR(A, A, C) = D (A  C, A  C) = 0 • SynR(A, B, 0) = D (0, 0) = 0 • SynR(A, 0, C) = π/2 • SynA(A, B) = SynR(A, B, 1) = D (A 1, B 1) = D (A, B)

  29. Subjective synonymy • SynS(A, B, C) — C as point of view SynS(A, B, C) = D(C-A, C-A) 0  SynS(A, B, C)  π normalization: 0  asin(sin(SynS(A, B, C)))  π/2 C” ” C’ ’ C A  B

  30. Subjective synonymy When |C|   then SynS(A, B, C)  0 SynS(A, B, 0) = D(-B, -A) = D(A, B) SynS(A, A, C) = D(C-A, C-A) = 0 SynS(A, B, B) = SynS(A, B, A) = 0 • SynS(tit, swallow, animal) = 0.3 • SynS(tit, swallow, bird) = 0.4 • SynS(tit, swallow, passerine) = 1

  31. Semantic analysis • Vectors propagate on syntactic tree P GN GVA Les termites GV rapidement GNP attaquent les fermes GN du toit The white ants strike rapidly the trusses of the roof 68

  32. Semantic analysis P GN GVA Les termites GV rapidement GNP attaquent les fermes GN to start to attack to Critisize du toit farm business farm building truss roof anatomy above

  33. Semantic analysis • Initialization - attach vectors to nodes P GN GVA Les termites GV rapidement • 1 • 5 GNP attaquent • 2 les fermes GN • 3 du toit • 4

  34. Semantic analysis • Propagation (up) P GN GVA Les termites GV rapidement GNP attaquent les fermes GN du toit

  35. Semantic analysis • Back propagation (down) • (Ni j) = ((Ni j) (Ni)) + (Ni j) P GN GVA Les termites GV rapidement • 1’ • 5’ GNP attaquent • 2’ les fermes GN • 3’ du toit • 4’

  36. Semantic analysis • Sense selection or sorting P GN GVA GV Les termites rapidement GNP attaquent les fermes GN to start  to attack to Critisize du toit farm business farm building  truss  roof anatomy above

  37. 1:D(3,4) & (3+2) 3:D(4,5) & (4+5) 2:(bank4) 5:D(6,7)& (6+7) 4:(bank3) 6:(bank1) 7:(bank2) Sense selection • Recursive descent • on t(w) as decision tree • DA(’, i) Stop on a leaf Stop on an internal node

  38. Vector syntactic schemas • S: NP(ART,N) • (NP) = V(N) • S: NP1(NP2,N) • (NP1) =  (NP1)+ (N) 0<<1 (sail boat) = (sail) + 1/2 (boat) (boat sail) = 1/2 (boat) + (sail)

  39. Vector syntactic schemas • Not necessary linear • S: GA(GADV(ADV),ADJ) • (GA) = (ADJ)^p(ADV) • p(very) = 2 • p(mildly) = 1/2 (very happy) = (happy)^2 (mildly happy) = (happy)^1/2

  40. Iteration & convergence • Iteration with convergence Local D(i, i+1)  for top  Global D(i, i+1)  for all Good results but costly

  41. Lexicon construction • Manual kernel • Automatic definition analysis • Global infinite loop = learning • Manual adjustments synonyms Ukn word in definitions iterations kernel

  42. Applicationmachine translation • Lexical transfer • sourcetarget • Knn search that minimizes DA(source, target) • Submeaning selection Direct Transformation matrix cat chat

  43. ApplicationInformation Retrieval on Texts • Textual document indexation • Language dependant • Retrieval • Language independent - Multilingual • Domain representation horse  equitation • Granularity Document, paragraphs, etc.

  44. ApplicationInformation Retrieval on Texts • Index = Lexicon = (di ,i )* Knn search that minimizes DA((r), (di)) doc1 request docn doc2

  45. Search engine Distances adjustments • Min DA((r), (di)) may pose problems • Especially with small documents • Correlation between CV & conceptual richness • Pathological cases « plane » and « plane plane plane plane … » « inundation »« blood » D = 0.85 (liquid)

  46. Search engine Distances adjustments • Correction with relative intensity • Request vs retrieved doc (r and d) D =  (DA(r , d) * DI(r , d)) • 0 (r , d)  1  0  DI(r , d)  π/2

  47. Conclusion • Approach • statistical (but not probabilistic) • thema (and rhema ?) • Combination of • Symbolic methods (IA) • Transformational systems • Similarity • Neural nets • With large Dim (> 50000 ?)

  48. Conclusion • Self evaluation • Vector quality • Tests against corpora • Unknown words • Proper nouns of person, products, etc. • Lionel Jospin, Danone, Air France • Automatic learning • Badly handled phenomena? • Negation & Lexical functions (Meltchuk)

  49. End • 1. extremity • 2. death • 3. aim • …

More Related