1 / 45

CMPS 561 Fuzzy Set Retrieval

CMPS 561 Fuzzy Set Retrieval. Ryan Benton September 1, 2010. Agenda. Fuzzy Sets Fuzzy Operations Fuzzy Set Retrieval Processing Fuzzy Queries Set Issues l -level Sets. Fuzzy Sets. Issue. Problem Boolean assumes documents Exist in 1 of 2 states States are Non-Overlapping CRISP!

aderyn
Download Presentation

CMPS 561 Fuzzy Set Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMPS 561Fuzzy Set Retrieval Ryan Benton September 1, 2010

  2. Agenda • Fuzzy Sets • Fuzzy Operations • Fuzzy Set Retrieval • Processing Fuzzy Queries • Set Issues • l-level Sets

  3. Fuzzy Sets

  4. Issue • Problem • Boolean assumes documents • Exist in 1 of 2 states • States are Non-Overlapping • CRISP! • Other crisp states may exist • Grades: A, B, C, D, F • Can be in one and not the other. • Often, exact boundaries are not known • Hot, Warm, Luke-warm, Cool, Chilly, Cold • When does Hot become Warm?

  5. Issue • Fuzzy Sets • Informal Definition • Fuzzy Set is a class with fuzzy boundaries. • Alternatively, fuzzy set is a class with non-crisp boundaries.

  6. Fuzzy Sets – Slightly More Formal • Regular (Crisp) set • Let X = {xi} be the universe of objects of interest. • Fuzzy Subset of X  A. • A is a set of order pairs • A = { < x, mA(x)> } • mA(x)  Grade of membership of x in A • mA : X  [0,1]

  7. Example 1 • Assume Crisp Set is • X = {x1, x2, x3, x4} • Fuzzy Subsets • A = {<x1,1.0>,<x2,0.5>< x3,0.8>} • B = {<x2,0.8>}

  8. medium tall short 1 m 0 1 .75 m .2 0 4’ Example 2Membership Functions Crisp 4’6” 6’ Fuzzy mtall(4’)=0, mmedium(4’)=0.2, mshort(4’)=.75

  9. Fuzzy Operations

  10. Operations 1 • Let • A = { < x, mA(x)> } • B = { < x, mB(x)> } • Equality • A = B iff • mA(x) = mB(x), for all x Î X.

  11. Operations 2 • Containment • A Í B iff • mA(x) £mB(x), for all x Î X. • Complement • Ā = { < x, mĀ(x)> } iff • mĀ(x) = 1 - mA(x), for all x Î X.

  12. Operations 3 • Union • A È B = {< x, mAÈB(x)> } iff • mAÈB = max(mA(x), mB(x)), for all x Î X. • Intersection • A Ç B = {< x, mAÇB(x)> } iff • mAÇB = min(mA(x), mB(x)), for all x Î X.

  13. Fuzzy Set Retrieval Model

  14. Document Representation Boolean • Relation • D = { < da, ti, mD(da, ti)> } • mD: D x T  {0,1} • mD(da, ti) • 1, if da contains ti • 0, otherwise • Dt = {daÎ D | mD(da, t) = 1} • d º Dd = {tiÎ T | mD(d, ti) = 1}

  15. Document RepresentationFuzzy • D = { < da, ti, mD(da, ti)> } • mD: D x T  [0,1] • mD(da, ti) • The degree of membership of ti to da • Dt = {<da, mD(da, t)> | daÎ D >} • Dd = {<ti, mD(d, ti)> | tiÎ T>}

  16. Processing Boolean Query (Method 2) • Dt1Ùt2 = {daÎ D | mD(da, t1) Ù mD(da, t2) = 1} • Dt1Út2 = {daÎ D | mD(da, t1) Ú mD(da, t2) = 1} • Dt = set of Documents containing term t • T = {a,b,c,d,e} • Da, Db, Dc, Dd, De,

  17. Processing Fuzzy Query (Method 2) • Dt1Ùt2 = {<da , mD(da, t1) Ù mD(da, t2) > | daÎ D } • Dt1Út2 = {<da , mD(da, t1) Ú mD(da, t2) > | daÎ D } • Dt = set of <Document, membership value> pairs containing term t • T = {a, b, c, d, e} • Da, Db, Dc, Dd, De

  18. Retrieval FunctionBoolean • RSV º F • RSVt(da) = mD(da, t) • RSVØe(da) = 1 - RSVe(da) • RSVe1Ùe2(da) = RSVe1(da) Ù RSVe2(da) • RSVe1Úe2(da) = RSVe1(da) Ú RSVe2(da)

  19. Retrieval FunctionFuzzy • RSVt(da) = mD(da, t) • RSVØe(da) = 1 - RSVe(da) • RSVe1Ùe2(da) = min(RSVe1(da),RSVe2(da)) • RSVe1Úe2(da) = max(RSVe1(da),RSVe2(da))

  20. Processing Fuzzy Queries

  21. Term-Document Incidence Matrix

  22. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4))

  23. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example(Method 1) q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4)) 0.7 0.5 0.6 0.7 0.3 0.2 0.1 0.5 0.6 0.1 0.5 0.8 0.8 0.8 0.5 d3 0.6 d6 0.6 d8 0.6 0.9 0.5 0.2 0.9 0.5 0.2 0.2 0.2 0.5 0.7 0.3 0.9

  24. Inverted File Processing(Method 2) • Dt = {<da, mDt(da)> | mDt(da) > 0} • DØe = {<da, 1-RSVe(da)> | 1-RSVe(da) > 0} • De1Ùe2 = {<da, RSVe1Ùe2(da)> | RSVe1Ùe2(da) > 0} • De1Úe2 = {<da, RSVe1Úe2(da)> | RSVe1Úe2(da) > 0}

  25. Output Dt De1 De2 De1 Ç De2 De1 È De2 D\De1 Query t e1 e2 e1Ùe2 e1Úe2 Øe1 Processing Fuzzy Query (Method 2)

  26. Set Issues

  27. A È B = B È A (AÈ B) È C = AÈ (B È C) AÈ (B Ç C) = (AÈ B) Ç (AÈ C) AÈ Æ = A A È A’ = S AÇ B = BÇ A (AÇ B) Ç C = AÇ (B Ç C) AÇ (B È C) = (AÇ B) È (AÇ C) A Ç S = A A Ç A’ = Æ Set Identities

  28. ℇ · + + e g g Ç È U U v v Definitions of Union and Intersection

  29. ℇ + e È Ç Operations – Union & Intersection • a · b = ab • a b = a + b – ab • a b = (a + b) / (1 + ab) • a b = (ab) / ( 1 + (1-a)(1-b) • a b = min(a + b, 1) • a b = max(0, a + b - 1)

  30. · + g g U U v v Operations – Union & Intersection • a b = (a + b – (1 – g)ab) / (g + (1-g)(1- ab)) • g  [0, +∞) • a b = (ab) / (g + (1-g)(a+b)) • g  [0, +∞) • a b = min(1, (av + bv)(1/v)) • v= [1, +∞) • a b = 1 - min(1, [(1-a)v + (1-b)v](1/v)) • v= [1, +∞)

  31. T p Operations – Union & Intersection a b =

  32. T p Operations – Union & Intersection a b =

  33. Example of Set Operations • Assume • Union = Probabilistic Sum • Ú = a b = a + b – ab • Intersection = Product • Ù = a · b = ab

  34. Term-Document Incidence Matrix

  35. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4))

  36. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example (Method 1)Max, Min q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4)) 0.7 0.5 0.6 0.7 0.3 0.2 0.1 0.5 0.6 0.1 0.5 0.8 0.8 0.8 0.5 d3 0.6 d6 0.6 d8 0.6 0.9 0.5 0.2 0.9 0.5 0.2 0.2 0.2 0.5 0.7 0.3 0.9

  37. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example (Method 1)probabilistic sum and product q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4)) 0.5316 0.384 0.4998 0.504 0.12 0.09 0.06 0.3 0.48 0.1 0.5 0.8 0.8 0.8 0.5 d3 0.6 d6 0.6 d8 0.6 0.9 0.5 0.2 0.9 0.5 0.2 0.2 0.2 0.5 0.7 0.3 0.9

  38. Differences

  39. l-level Sets

  40. l-level Sets - Definitions • l-level Set • D(l*) = {<d,t> | mD (da, t) ≥ l*} • Dt(l*) = { d | <d, t> Î D (l*)} • Then meaning of Dt(l*) of descriptor tÎTÎT* • Dt(l*) = {<d, mDt(l*)(d) = mDt(l*)(d) > | d Î Dt(l*)} Note: T* has same meaning as E used in Boolean IR • If t’ÎT*, then meaning of Dt(l*) of descriptor t =Øt’ • Dt(l*) = {<d, mDt(l*)(d)=1-mDt’(l*)(d)> | 1-mDt’(l*)(d) > l*,d Î Dt’(l*) }

  41. l-level Sets - Definitions • If t’, t’’ÎT*, then meaning of Dt(l*) of descriptor t = t’ Ù t’’ • Dt(l*) = {<d, mDt(l*)(d)=min(mDt’(l*)(d), mDt’’(l*)(d))> |d Î Dt’(l*) Ç Dt’’(l*) } • If t’, t’’ÎT*, then meaning of Dt(l*) of descriptor t = t’ Ú t’’ • Dt(l*) = {<d, mDt(l*)(d)=max(mDt’(l*)(d), mDt’’(l*)(d))> |d Î Dt’(l*) È Dt’’(l*) }

  42. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4))

  43. Ú Ù Ù Ø Ø t4 t2 t1 t2 t3 Fuzzy example Max, Min q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4)) 0.7 0.5 0.6 0.7 0.3 0.2 0.1 0.5 0.6 0.1 0.5 0.8 0.8 0.8 0.5 d3 0.6 d6 0.6 d8 0.6 0.9 0.5 0.2 0.9 0.5 0.2 0.2 0.2 0.5 0.7 0.3 0.9

  44. 0.7 0.5 0.6 Ú 0.7 0.3 0.0 0.0 0.5 0.6 Ù Ù 0.1 (0) 0.5 1.0 Ø Ø 1.0 1.0 0.5 t4 t2 t1 t2 d3 0.6 d6 0.6 d8 0.6 0.9 0.5 0.2 (0) 0.9 0.5 0.0 t3 0.2 (0) 0.2 (0) 0.5 0.7 0.3 0.9 Fuzzy example Max, Min, l*=0.3 q = (t1ÙØt2) Ú (t2Ù Øt3Ù t4))

  45. Questions?

More Related