450 likes | 609 Views
Contextual Image Search. Wenhao Lu , Jingdong Wang , Xian- Sheng Hua , Shengjin Wang , Shipeng Li Tsinghua University, Beijing, P. R. China, Microsoft Research Asia, Beijing, P. R. China,. MM 2011. Traditional image search. MM 2011. Contextual image search. company. iPhone.
E N D
Contextual Image Search Wenhao Lu , Jingdong Wang , Xian-ShengHua, Shengjin Wang , Shipeng Li Tsinghua University, Beijing, P. R. China, Microsoft Research Asia, Beijing, P. R. China, MM 2011
Traditional image search MM 2011
Contextual image search company iPhone MM 2011
Commercial image search engine query: Funny George Bush MM 2011
System overview Text input MM 2011
System overview Image input 2. Annotating images by mining search result(2008) MM 2011
Image input example Candidate queries: “Blue mosque”, “Istanbul”, “Turkey travel”, “Istanbul turkey” The mosque is one of several mosques known as the Blue Mosque for the blue tiles adorning the walls of its interior Input image Similar image Search Result MM 2011
Contextual Image Search With Text Input 1. Context Capturing • textual contexts: page title / document title • local context • visual contexts: vision-based page segmentation algorithm • (VIPS) MM 2011
Contextual Image Search With Text Input 2. Contextual Query Augmentation • Goal: remove possible ambiguities • Augmented query = query + textual context Candidate augmented query evaluate the relevance between the context and augmented query (Okapi BM25) MM 2011
Contextual Image Search With Text Input 3. Image Search by Text: Microsoft Bing image search 4.Contextual Reranking: Combine textually and visually context MM 2011
Quantitative Evaluation 0.95 0.65 nDCG curves MM 2011
YouPivot: Improving Recall with Contextual Search Joshua Hailpern 1, Nicholas Jitkoff 2, Andrew Warr 2 Karrie Karahalios 1,Robert Sesek 3, Nikita Shkrob 4 1 University of Illinois Urbana, IL USA 61801 2 Google Mountain View, CA USA 94043 3 Boston University Boston, MA USA 02215 4 University of Waterloo Waterloo, Ontario, Canada N2L 3G1 SIGCHI 2011
Why Contextual Search? “what was that website I was looking at when Yesterday by The Beatles was last playing? ” SIGCHI 2011
Improving Recall ? • Contextual cues: temporally related activities • Cognitive science: • leveraging context improves speed and • accuracy in recall tasks • Loses car keys: • “retrace your steps since the last time you know you had them.” SIGCHI 2011
Interface SIGCHI 2011
Features and Functionality Sarah Where is the website? ?title ?domain ?when Mr. Richfield graphic designer SIGCHI 2011
Features and Functionality Websites Layout SIGCHI 2011
Context Sensitive Paraphrasing with a Single Unsupervised Classifier Michael Connor and Dan Roth Department of Computer Science University of Illinois at Urbana-Champaign ECML2007
Context Sensitive Paraphrasing ‘X commanded Y’ ‘X spoke to Y’ • When can ‘speak to’ replace ‘command’ • in the original sentence and not change the • meaning of the sentence? ECML2007
Polysemous Nature of Verbs - command ECML2007
Definition of Context • derived from parsing information • subject and objectof the verb Marshall Formby of Plainview suggested a plan to fill by appointment future vacancies in the Legislature and Congress, eliminating the need for special elections. Local Context: obj:plan , subj:NE:PER ECML2007
Modeling Context Sensitive Paraphrasing obj:plan , subj:NE:PER 1. Context Sensitive Decisions v: original verb u: substitute verb creating, breaking or presenting type c contextual features of v/u c: sub / obj ECML2007
Unsupervised Training: Bootstrapping Local Classifiers ECML2007
Experimental Results • AQUAINT Corpus (News Articles) • test set has 721 S, v, u examples with 57 unique v verbs and 162 unique u. (random selection of polysemous verbs that occur in WordNet 2.1) ECML2007
Experimental Results ECML2007
Hierarchical summarization for delivering information to mobile devices Jahna Otterbacher a,*, Dragomir Radev b, Omer Kareem b a University of Cyprus, Nicosia, Cyprus b University of Michigan, Ann Arbor, MI 48109, United States SIGIR 2007
Recent News • 17歲天才寫APP 幫新聞摘要精華(2012/11/3) • 一名英國的17歲男生,設計出一款可以幫新聞摘要出「精華版」 • 的APP-Summly,這款應用程式在業界廣受好評。 SIGIR 2007
Limitation of Mobile Device • small screens • constrained wireless bandwidth SIGIR 2007
Architecture of summarization method SIGIR 2007
Sentence scoring • Centroid value: the importance of the sentence :TF*IDF values of word w in • Positional value: More weight is given to sentences that appear • earlier in the document than those that appear later.(news articles) :first sentence centroid value SIGIR 2007
Sentence scoring • First sentence overlap value: The first sentence in a text is likely • to convey information about its main theme or topic. SIGIR 2007
Hierarchical nesting SIGIR 2007
Experiment ─ 39 subjects in the experiment(student studying information and computer science) ─10 articles 10 questions SIGIR 2007
A Maximum Entropy Web Recommendation System: Combining Collaborative and Content Features Xin Jin, Yanzan Zhou, Bamshad Mobasher Center for Web Intelligence School of Computer Science, Telecommunication, and Information Systems DePaul University, Chicago, Illinois, USA SIGKDD 2005
Web Recommendation System SIGKDD 2005
About Web Recommendation ─ Goal: help users locate information on the Web ─ Input: • Web users’ navigation or rating data • contentfeatures of the items ─ Approach: Data mining or Machine Learning todiscover usage patternsthat represent aggregate user models. SIGKDD 2005
Maximum Entropy Recommendation Model ─ Maximum Entropy: P(飛行 | fly) + (P(搭機 | fly) + P(蒼蠅 | fly) = 1 s.t. P(飛行 | fly) + (P(搭機 | fly) = 4/5 P(蒼蠅 | fly)=1/5 SIGKDD 2005
Maximum Entropy Recommendation Model ─ Offline: 1.accept constraints to form the model 2.estimate the model parameters ─ Online: 1.reads an active session 2.runs the recommendation algorithm SIGKDD 2005
Maximum Entropy Recommendation Model ─ Distribution form: : a page being visited next : user’s recent navigational history : weight of = SIGKDD 2005
Maximum Entropy Recommendation Model ─ identification of features (navigation data): if ( ) SIGKDD 2005
Maximum Entropy Recommendation Model ─ identification of features (rating data): select highly correlated item pairs SIGKDD 2005
Maximum Entropy Recommendation Model ─ identification of features (content information): In a movie site, high ratings for “Indiana Jones” and “Air Force One” may suggest that a user is a Harrison Ford’s fan and enjoys Action-Adventure movies. Use Latent Dirichlet Allocation(LDA) to find the class of each item. SIGKDD 2005
Maximum Entropy Recommendation Model ─ identification of features (content information): SIGKDD 2005
Experiment ─Realty data,24,000 user sessions from 3,800 unique users SIGKDD 2005