170 likes | 307 Views
Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression. Gunhee Kim Eric P. Xing. School of Computer Science, Carnegie Mellon University. February 6, 2013. Image Ranking and Retrieval. Goal: Find the images for a given query. Text-based image retrieval.
E N D
Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression Gunhee Kim Eric P. Xing School of Computer Science, Carnegie Mellon University February 6, 2013
Image Ranking and Retrieval Goal: Find the images for a given query Text-based image retrieval ex. Cardinal
Image Ranking and Retrieval Goal: Find the images for a given query Text-based image retrieval ex. Cardinal Url • Scalable and successful so far http://www.allaboutbirds.org/guide/Northern_Cardinal/id File name northern_cardinal_glamour.jpg • Ambiguity and noise due to mismatch.
RecentImage Ranking and Retrieval Various efforts to improve text-based image search User relevance feedback [Wang et al. CVPR 11] Reranking on visual features Text-based search by apple chosen by a user Pseudo-relevance feedback Human labeled training data[Yang et al. MM10] [Liu et al. CVPR 11] Image click data [Jain et al. WWW11]
Time-Sensitive Image Ranking and Retrieval Discovery of temporal patterns of Web image collections [Related work] Exploring temporal dynamics of Web queries • Popular search keywords and relevant documents change over time. • ex) Keyword search, Product search, News recommendation No previous work using temporal info on image retrieval From experiments of 7.5 millionsof Flickr images of 30 topics we found threegood reasons … • [D08] Dakka et al. CIKM 2008 • [M09] Metzler et al. SIGIR 2009 • [K10] Kulkani et al, WSDM 2011 • [V11] Amodeo et al, CIKM2011 • [R12] Radinsky et al, WWW 2012 • …..
Why Time-Sensitive Image Retrieval? (1/3) 1. Knowing when search takes place is useful to infer users' implicit intents. Google Bing Cardinal: (1) the red bird in America. (3) St. Louis cardinals (baseball) (2) Arizona cardinals (football) Spring to Fall (Mar. ~ Oct.) Fall to Winter(Sep. ~ Feb.) • Severely redundant. Almost identical all year long.
Why Time-Sensitive Image Retrieval? (1/3) 1. Knowing when search takes place is useful to infer users' implicit intents. Google Bing • Diversity can make search interesting. Football atFeb. 7, 2009 Ourresults baseball at May 4, 2009
Why Time-Sensitive Image Retrieval? (2/3) 2. Timing suitability can be used as a complementary attribute to relevance. Google Bing Background: snow atFeb. 7, 2009 Ourresults Baby birds or eggs Background: Green at May 4, 2009 • There are so many almost equally good images.
Why Time-Sensitive Image Retrieval? (3/3) 3. Temporal information is synergetic in personalized image retrieval. At Nov. 7, 2009 for user 30033302 Louisville Men's College Basketball Each user’ term usages are relatively stationary, and predictable once they are learned.
Algorithm Regularized multi-task regression on multivariate point process • Goal: Scalablylearn temporal models for each topic keyword. • Multi-task framework: allows multiple image descriptors. • Several regularization schemes • Personalization by locally-weighted learning
Multivariate Point Process Models Given a stream of hornet pictures up to T t1t2t3t5t6t7t9 t10 Time Clustering by descriptor 1 Clustering by descriptor 2 (v1) (v2) 1st descriptor 2nd descriptor
Regularized GLM on Point Processes Given a stream of hornet pictures up to T t1t2t3t5t6t7t9 t10 Time (v1,v2) (3, 2) (3, 2) (3, 2) (2, 1)(1, 3) (1, 4)(2, 3) (2, 1) Formulate a regression between occurrence rates and covariates. Covariates: any likely factors to be associated with image occurrence (ex. Time, season, and other external events) Compute sparse regularized MLE solutions For each visual cluster, we select only a small number of strong factors.
A Toy Example of Image Reranking Covariates: only year and months Peaked in summer (Sea tour) (Ice hockey) Peaked in January (Aquarium) Stationary