1 / 14

Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression

Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression. Gunhee Kim Eric P. Xing. School of Computer Science, Carnegie Mellon University. February 6, 2013. Image Ranking and Retrieval. Goal: Find the images for a given query. Text-based image retrieval.

jamal
Download Presentation

Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression Gunhee Kim Eric P. Xing School of Computer Science, Carnegie Mellon University February 6, 2013

  2. Image Ranking and Retrieval Goal: Find the images for a given query Text-based image retrieval ex. Cardinal

  3. Image Ranking and Retrieval Goal: Find the images for a given query Text-based image retrieval ex. Cardinal Url • Scalable and successful so far http://www.allaboutbirds.org/guide/Northern_Cardinal/id File name northern_cardinal_glamour.jpg • Ambiguity and noise due to mismatch.

  4. RecentImage Ranking and Retrieval Various efforts to improve text-based image search User relevance feedback [Wang et al. CVPR 11] Reranking on visual features Text-based search by apple chosen by a user Pseudo-relevance feedback Human labeled training data[Yang et al. MM10] [Liu et al. CVPR 11] Image click data [Jain et al. WWW11]

  5. Time-Sensitive Image Ranking and Retrieval Discovery of temporal patterns of Web image collections [Related work] Exploring temporal dynamics of Web queries • Popular search keywords and relevant documents change over time. • ex) Keyword search, Product search, News recommendation No previous work using temporal info on image retrieval From experiments of 7.5 millionsof Flickr images of 30 topics we found threegood reasons … • [D08] Dakka et al. CIKM 2008 • [M09] Metzler et al. SIGIR 2009 • [K10] Kulkani et al, WSDM 2011 • [V11] Amodeo et al, CIKM2011 • [R12] Radinsky et al, WWW 2012 • …..

  6. Why Time-Sensitive Image Retrieval? (1/3) 1. Knowing when search takes place is useful to infer users' implicit intents. Google Bing Cardinal: (1) the red bird in America. (3) St. Louis cardinals (baseball) (2) Arizona cardinals (football) Spring to Fall (Mar. ~ Oct.) Fall to Winter(Sep. ~ Feb.) • Severely redundant. Almost identical all year long.

  7. Why Time-Sensitive Image Retrieval? (1/3) 1. Knowing when search takes place is useful to infer users' implicit intents. Google Bing • Diversity can make search interesting. Football atFeb. 7, 2009 Ourresults baseball at May 4, 2009

  8. Why Time-Sensitive Image Retrieval? (2/3) 2. Timing suitability can be used as a complementary attribute to relevance. Google Bing Background: snow atFeb. 7, 2009 Ourresults Baby birds or eggs Background: Green at May 4, 2009 • There are so many almost equally good images.

  9. Why Time-Sensitive Image Retrieval? (3/3) 3. Temporal information is synergetic in personalized image retrieval. At Nov. 7, 2009 for user 30033302 Louisville Men's College Basketball Each user’ term usages are relatively stationary, and predictable once they are learned.

  10. Algorithm Regularized multi-task regression on multivariate point process • Goal: Scalablylearn temporal models for each topic keyword. • Multi-task framework: allows multiple image descriptors. • Several regularization schemes • Personalization by locally-weighted learning

  11. Thank you !Stop by our poster!

  12. Multivariate Point Process Models Given a stream of hornet pictures up to T t1t2t3t5t6t7t9 t10 Time Clustering by descriptor 1 Clustering by descriptor 2 (v1) (v2) 1st descriptor 2nd descriptor

  13. Regularized GLM on Point Processes Given a stream of hornet pictures up to T t1t2t3t5t6t7t9 t10 Time (v1,v2) (3, 2) (3, 2) (3, 2) (2, 1)(1, 3) (1, 4)(2, 3) (2, 1) Formulate a regression between occurrence rates and covariates. Covariates: any likely factors to be associated with image occurrence (ex. Time, season, and other external events) Compute sparse regularized MLE solutions For each visual cluster, we select only a small number of strong factors.

  14. A Toy Example of Image Reranking Covariates: only year and months Peaked in summer (Sea tour) (Ice hockey) Peaked in January (Aquarium) Stationary

More Related