1 / 22

Extended SQL for Multimedia Queries

Extended SQL for Multimedia Queries. select <whats> from <whatr> where <preds> orderby <whato) groupby <whatg> <whats> can be alphanum, images, functions <functions> can be numeric-valued, vector- valued (such as histograms),

conor
Download Presentation

Extended SQL for Multimedia Queries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Extended SQL for Multimedia Queries select <whats> from <whatr> where <preds> orderby <whato) groupby <whatg> <whats> can be alphanum, images, functions <functions> can be numeric-valued, vector- valued (such as histograms), image-valued (such as extractors), aggregates,…. <preds> can be hard constraints or similarity functions <whato) can be alphanum or similarities <whatg> can be alphanum or similarities

  2. Queries from Current System select A.id, A.img, simfMRI(A.img, B.img) from fMRI A, fMRI B where B.id = 1 orderby simfMRI(A.img,B.img) fMRI simfMRI select A.id, I.img, S.prob from fMRI A, simfMRI S where S.img = A.img orderby S.prob id img img prob similarities of all images to a selected one with id = 1 *Assumes simfMRI is a global comparison of 2 fMRI images.

  3. Queries from Current System User query: select A.id, A.img from fMRI A, fMRI B where B.id = 1 and simfMRI(A.img, B.img) MystiQ query: fMRI simfMRI select distinct A.id, A.img from fMRI A, simfMRI S where S.fid = A.fid fid id img img prob similarities of all images to a selected one with id = 1 *Assumes simfMRI is a global comparison of 2 fMRI images.

  4. Potential Queries from Very Simple and Beyond Return the similarity between the fMRI images of patient 12 and patient 37. select simfMRI(A.img,B.img) from fMRI A, fMRI B where A.id = 12 and B.id = 37 fMRI simfMRI id img prob single similarity value

  5. Q3 Find pairs of patients in the database having the same age and similar fMRI images. select A.id, B.id, P.age, simfMRI(A.img, B.img) from patient P, patient Q, fMRI A, fMRI B where P.ID = A.id and Q.ID = B.id and A.age = B.age orderby simfMRI(A.img, B.img) patient fMRI simfMRI id img img img prob id age pairs of database images in order of similarity

  6. Q3 Find pairs of patients in the database having the same age and similar fMRI images. User query: select A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI B where P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI(A.img, B.img) patient fMRI simfMRI pairs of databaseimages id age fid id img fid1 fid2 prob MystiQ query: select distinct A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI B, simfMRI where P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI.fid1 = A.fid and simfMRI.fid2=B.fid Note: twopatients A.id,B.id may havemultiple similarimages. Theprobability is non-trivial.

  7. Q4 Given a query image called thisimg, return the id’s and similarities for database images that are similar enough. select A.id, P.age, A.img, simfMRI(thisimg, A.img) where P.ID = A.id and simfMRI(thisimg,A.img) > thresh orderby simfMRI(thisimg, A.img) thisimg thresh patient fMRI simfMRI id age id img img prob

  8. Q4 Given a query image called thisimg, return the id’s and similarities for database images that are similar enough. select A.id, P.age, A.imgfrom patient P, fMRI A where P.ID = A.id and simfMRI(thisimg,A.img) > thresh and simfMRI(thisimg, A.img) User query: thisimg patient fMRI simfMRI id age img fid id img fid prob MystiQ query: select A.id, P.age, A.img, simfMRI(thisimg, A.img) from patient P, fMRI A, simFMRI S where P.ID = A.id and simfMRI(thisimg,A.img) > thresh and A.fid = S.fid

  9. Q4 Given a query image called thisimg, return the id’s and similarities for database images that are similar enough. Better user query: select A.id, P.age, A.imgfrom patient P, fMRI A where P.ID = A.id andand HIGH(simfMRI(thisimg,A.img)) [weight=7] and simfMRI(thisimg, A.img) [weight=0] thisimg patient fMRI simfMRI id age img fid id img fid prob

  10. Weights I’m proposing extending the probabilistic predicatesand/or probabilistic tables with weights E.g. FROM simfMRI S [weight=3.5]WHERE simfMRI(img1,img2) [weight=-0.5] Default weight = 0.0 Positive weight means the predicate is more important Negative weight means it is less important.

  11. High/Low Predicates I’m also proposing introducing two newprobabilistic predicates: High(value) = more “true” the higher “value” is Low(value) = more “true” the lower “value” is These predicates make sense only within a query: Different values are compared and the higher (lower)receive higher probability.

  12. Q5 Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order by similarity at that particular location. select A.id, A.img, activation(A.img,loc), simsignal(signal(A.img,loc), signal(thisimg,loc)) from fMRI A where activation(A.img,loc) > thresh orderby simsignal(signal(A.img,loc), signal(thisimg,loc) thisimg loc thresh fMRI simsignal activation signal (, ) (,) fid id img signal prob activation is a function that returns the mean activation level at a given location. signal is a function that returns the signal at a given location. locations denote particular voxels.

  13. Q5 Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order by similarity at that particular location. User query: select A.id, A.img, activation(A.img,loc) from fMRI A, thisimg T where activation(A.img,loc) > thresh and simsignal(signal(A.img,loc), signal(T,loc)) thisimg loc thresh fMRI simsignal activation signal (, ) (,) fid id img signal prob activation is a function that returns the mean activation level at a given location. signal is a function that returns the signal at a given location. locations denote particular voxels.

  14. Q5 Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order by similarity at that particular location. Better user query: select A.id, A.img, activation(A.img,loc) from fMRI A, thisimg T where high(activation(A.img,loc)) [weight=10.0] and simsignal(signal(A.img,loc), signal(T,loc)) thisimg loc thresh fMRI simsignal activation signal (, ) (,) signal prob fid id img activation is a function that returns the mean activation level at a given location. signal is a function that returns the signal at a given location. locations denote particular voxels.

  15. Q6 Return ids and images for images (A) that have the highest activation level of all database images (B) at a given location. For each of these, return id’s and images (C) of all database images whose activation is above a threshold at that location, ordered by their signal similarity to A at that location. select A.id, A.img, C.id, C.img from fMRI A, fMRI B, fMRI C where activation(A.img,loc) = max(activation(B.img,loc)) and activation(C.img,loc) > thresh orderby simsignal(signal(A.img,loc),signal(C.img,loc)) loc thresh fMRI simsignal activation signal (, ) (,) id img signal prob

  16. Q6 Return ids and images for images (A) that have the highest activation level of all database images at a given location. For each of these, return id’s and images (C) of all database images whose activation is above a threshold at that location, ordered by their signal similarity to A at that location. User query: select A.id, A.img, C.id, C.img from fMRI A, fMRI C where high(activation(A.img,loc)) [weight = 100.0] and high(activation(C.img,loc)) [weight = 0.0] and simsignal(signal(A.img,loc),signal(C.img,loc)) [weight=0.0] loc thresh fMRI simsignal activation signal (, ) (,) signal prob fid id img

  17. Q7 Extract and return a neighborhood of a given size about a given location from a given query image. select nbd(thisimg, loc, size) from thisimg thisimg loc size Simple example of an extraction function.

  18. Q8 Return id’s, images, and locations for images that have similar neighborhoods of a given size at that location to neighborhoods of the query image thisimg at location thisloc of the same given size. select A.id, A.img, I.loc, simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size)) as S from fMRI A, intloc I where A.img = I.img orderby S thisimg A thisimg thisloc fMRI intloc simnbd img nbd prob img loc id img The intloc relation contains the locations of interesting voxels in an image. This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.

  19. Q8 Return id’s, images, and locations for images that have similar neighborhoods of a given size at that location to neighborhoods of the query image thisimg at location thisloc of the same given size. User query: thisimg select A.id, A.img, I.loc from fMRI A, intloc I where A.fid = I.fid and simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size)) A thisimg thisloc fMRI intloc simnbd fid loc prob img nbd prob fid id img The intloc relation contains the locations of interesting voxels in an image. This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.

  20. Q9 Return id’s, images and pattern locations of database images that have an interesting pattern at a location that is similar to one of the locations of language sites in the same patient’s SUR data. select A.id,A.img, I.loc from fMRI A, SUR S, intloc I where A.img = I.img and simloc(I.loc, S.loc) fMRI SUR intloc simloc id img id loc img loc loc loc prob

  21. Q9 Return id’s, images and pattern locations of database images that have an interesting pattern at a location that is similar to one of the locations of language sites in the same patient’s SUR data. User query: select A.id,A.img, I.loc from fMRI A, SUR S, intloc I where l.fid = A.fid and simloc(I.loc, S.loc) fMRI SUR intloc simloc id img id loc loc loc prob fid loc prob

  22. Q10 We don’t understand this query yet Return id’s, images and pattern locations of database images that have an interesting pattern at a location that is similar to one of the locations of language sites in the same patient’s SUR data and whose fMRI signal at the fMRI location has a similar peak pattern to the SUR signal at its location. select A.id,A.img, I.loc, S.loc from fMRI A, SUR S, intloc I where A.img = I.img and A.id = S.id and simloc(I.loc, S.loc) and simpeaks(signal(A.img,I.loc),S.signal) fMRI SUR intloc simloc simpeaks id img id loc signal img loc loc loc prob signal signal prob

More Related