1 / 20

Street-to-Shop: Cross-Scenario Clothing Retrieval via Parts Alignment and Auxiliary Set

Street-to-Shop: Cross-Scenario Clothing Retrieval via Parts Alignment and Auxiliary Set. Si Liu+, Zheng Song, Guangcan Liu , Changsheng Xu +, Hanqing Lu+, Shuicheng Yan* * ECE Department, National University of Singapore + NLPR, Institute of Automation, Chinese Academy of Science.

mariah
Download Presentation

Street-to-Shop: Cross-Scenario Clothing Retrieval via Parts Alignment and Auxiliary Set

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Street-to-Shop: Cross-Scenario Clothing Retrievalvia Parts Alignment and Auxiliary Set Si Liu+, ZhengSong, Guangcan Liu, ChangshengXu+, Hanqing Lu+, ShuichengYan* * ECE Department, National University of Singapore + NLPR, Institute of Automation, Chinese Academy of Science

  2. Outline Introduction Related Work Dataset Construction Framework Towards Cross-Scenario by Human Parts Alignment Towards Cross-Scenario by Auxiliary Set Experiments Conclusions and Future Work

  3. Introduction Given a human photo captured on street, finding similar clothing from online shops Collect a large online shopping dataset and daily photo dataset

  4. Introduction • Two-step calculation to handle the cross-scenario discrepancies • human parts alignment • auxiliary set

  5. Related Work Clothing Study Parts Appearance based Human Attribute Analysis Unsupervised Transfer Learning

  6. Dataset Construction • Clothing Image Collection • Online Shopping (OS) dataset • Daily Photo (DP) dataset • Clothing Attribute Labeling • global • upper-body • lower-body

  7. Dataset Construction

  8. Framework

  9. Towards Cross-Scenario by Human PartsAlignment • Train one human upper body and one human lower body detector • Extract 5 kinds of features from the upper-body parts and lower-body parts • HOG, LBP, Color moment, Color histogram and skin descriptor

  10. Towards Cross-Scenario by Auxiliary Set Aim is to find a new representation of daily photo, so that it can be directly compared with clothing in the online shopping dataset.

  11. Towards Cross-Scenario by Auxiliary Set as image features from the OS dataset as image features from the auxiliary DP dataset : sparse matrix : reconstruction error convert into the following equivalent problem then minimize the following augmented Lagrange function

  12. Towards Cross-Scenario by Auxiliary Set

  13. Experiments • Experimental Setting • Evaluation Criterion q :query image Rel(i) : the groundtruth relevance between q and the ith ranked image k : top k retrieved datum N : normalization constant

  14. Experiments • Within-Scenario Vs. Cross-Scenario • using the OS query set and the DP query set to retrieve images in the OS training set

  15. Experiments Performances of Different Features and Parts

  16. Experiments • Parts Alignment for Cross-Scenario Retrieval • To validate the effectiveness, we compare our method with a baseline using global features • Auxiliary Set for Cross-Scenario Retrieval • implement the Algorithm 1 with λ1= 0.1 and λ2 =0.01 to learn the transfer matrix

  17. Experiments Exemplar Retrieval Results

  18. Experiments

  19. Experiments Extension: Interactive Clothing Retrieval

  20. Conclusions and Future Work • We propose a solution including two key components • human/clothing parts alignment to handle human pose variation • an auxiliary daily photo dataset to bridge cross-scenario discrepancies • In the future, our system can be extended as an online learning system

More Related