1 / 39

Diverse M-Best Solutions in Markov Random Fields

Diverse M-Best Solutions in Markov Random Fields. ,. ,. ,. Dhruv Batra TTI-Chicago / Virginia Tech. Payman Yadollahpour TTI-Chicago. Abner Guzman-Rivera UIUC. Greg Shakhnarovich TTI-Chicago. Local Ambiguity. Graphical Models. Hat. x 1. x 2. MAP Inference. …. x n. C at.

moana
Download Presentation

Diverse M-Best Solutions in Markov Random Fields

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diverse M-Best Solutionsin Markov Random Fields , , , Dhruv Batra TTI-Chicago / Virginia Tech PaymanYadollahpour TTI-Chicago Abner Guzman-Rivera UIUC Greg Shakhnarovich TTI-Chicago

  2. Local Ambiguity • Graphical Models Hat x1 x2 MAP Inference … xn Cat Most Likely Assignment (C) Dhruv Batra

  3. Problems with MAP Model-Class is Wrong! -- Approximation Error • Human Body ≠ Tree (C) Dhruv Batra Figure Courtesy: [Yang & Ramanan ICCV ‘11]

  4. Problems with MAP Model-Class is Wrong! Not Enough Training Data! -- Approximation Error -- Estimation Error (C) Dhruv Batra

  5. Problems with MAP Model-Class is Wrong! Not Enough Training Data! MAP is NP-Hard -- Approximation Error -- Estimation Error -- Optimization Error (C) Dhruv Batra

  6. Problems with MAP Model-Class is Wrong! Not Enough Training Data! MAP is NP-Hard Inherent Ambiguity -- Approximation Error -- Estimation Error -- Optimization Error -- Bayes Error ? ? Rotating clockwise / anti-clockwise? Old Lady looking left / Young woman looking away? One instance / Two instances? (C) Dhruv Batra

  7. Problems with MAP Single Prediction = Uncertainty Mismanagement Model-Class is Wrong! Not Enough Training Data! MAP is NP-Hard Inherent Ambiguity -- Approximation Error -- Estimation Error -- Optimization Error -- Bayes Error Make Multiple Predictions! (C) Dhruv Batra

  8. Multiple Predictions x x x x x x x x x x x x x Sampling Porway & Zhu, 2011 TU & Zhu, 2002 Rich History (C) Dhruv Batra

  9. Ideally: M-Best Modes Multiple Predictions M-Best MAP Sampling ✓ Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Flerova et al., 2011 Fromeret al., 2009 Yanover et al., 2003 (C) Dhruv Batra

  10. Ideally: M-Best Modes Multiple Predictions M-Best MAP Sampling ✓ This Paper: Diverse M-Best in MRFs Porway & Zhu, 2011 TU & Zhu, 2002 Rich History • Don’t hope for diversity. Explicitly encode it. • Not guaranteed to be modes. Flerova et al., 2011 Fromeret al., 2009 Yanover et al., 2003 (C) Dhruv Batra

  11. MAP Integer Program kx1 (C) Dhruv Batra

  12. MAP Integer Program 1 0 0 0 kx1 (C) Dhruv Batra

  13. MAP Integer Program 0 1 0 0 kx1 (C) Dhruv Batra

  14. MAP Integer Program 0 0 1 0 kx1 (C) Dhruv Batra

  15. MAP Integer Program 0 0 0 1 kx1 (C) Dhruv Batra

  16. MAP Integer Program 0 0 0 1 kx1 k2x1 (C) Dhruv Batra

  17. MAP Integer Program 0 0 0 1 kx1 k2x1 (C) Dhruv Batra

  18. MAP Integer Program Graphcuts, BP, Expansion, etc (C) Dhruv Batra

  19. Diverse 2nd-Best Diversity MAP (C) Dhruv Batra

  20. Diverse M-Best (C) Dhruv Batra

  21. Diverse 2nd-Best Q1: How do we solve DivMBest? Q2: What kind of diversity functions are allowed? Q3: How much diversity? See Paper for Details (C) Dhruv Batra

  22. Diverse 2nd-Best • Lagrangian Relaxation Diversity-Augmented Energy Many ways to solve: upergradient Ascent. Optimal. Slow. Primal See Paper for Details 2. Binary Search. Optimal for M=2. Faster. Dualize 3. Grid-search on lambda. Sub-optimal. Fastest. Dual Div2Best energy Concave (Non-smooth) Lower-Bound on Div2Best En. (C) Dhruv Batra

  23. Diverse 2nd-Best Q1: How do we solve Div2Best? Q2: What kind of diversity functions are allowed? Q3: How much diversity? See Paper for Details (C) Dhruv Batra

  24. Diversity • [Special Case] 0-1 Diversity M-Best MAP • [Yanover NIPS03; Fromer NIPS09; Flerova Soft11] • [Special Case] Max Diversity [Park & RamananICCV11] • Hamming Diversity • Cardinality Diversity • Any Diversity See Paper for Details (C) Dhruv Batra

  25. Hamming Diversity 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 (C) Dhruv Batra

  26. Hamming Diversity • Diversity Augmented Inference: (C) Dhruv Batra

  27. Hamming Diversity • Diversity Augmented Inference: Unchanged. Can still use graph-cuts! Simply edit node-terms. Reuse MAP machinery! (C) Dhruv Batra

  28. Experiments • 3 Applications • Interactive Segmentation: Hamming, Cardinality (in paper) • Pose Estimation: Hamming • Semantic Segmentation: Hamming • Baselines: • M-Best MAP (No Diversity) • Confidence-Based Perturbation (No Optimization) • Metrics • Oracle Accuracies • User-in-the-loop; Upper-Bound • Re-ranked Accuracies (C) Dhruv Batra

  29. Experiment #1 • Interactive Segmentation • Model: Color/Texture + Potts Grid CRF • Inference: Graph-cuts • Dataset: 50 train/val/test images Image + Scribbles MAP 2nd Best MAP Diverse 2nd Best 1-2 Nodes Flipped 100-500 Nodes Flipped (C) Dhruv Batra

  30. Experiment #1 +3.62% +1.61% +0.05% (Oracle) (Oracle) (Oracle) M=6 (C) Dhruv Batra

  31. Experiment #2 • Pose Tracking • Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11] • Inference: Dynamic Programming • Dataset: 4 videos, 585 frames (C) Dhruv Batra Image Credit: [Yang & Ramanan, ICCV ‘11]

  32. Experiment #2 • Pose Tracking w/ Chain CRF M BestSolutions (C) Dhruv Batra Image Credit: [Yang & Ramanan, ICCV ‘11]

  33. Experiment #2 MAP DivMBest + Viterbi (C) Dhruv Batra

  34. Experiment #2 Better DivMBest (Re-ranked) 13% Gain Same FeaturesSame Model [Park & Ramanan, ICCV ‘11] (Re-ranked) PCP Accuracy Confidence-based Perturbation (Re-ranked) #Solutions / Frame (C) Dhruv Batra

  35. Experiment #3 • Semantic Segmentation • Model: Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09] • Inference: Alpha-expansion • Dataset: Pascal Segmentation Challenge (VOC 2010) • 20 categories + background; 964 train/val/test images (C) Dhruv Batra • Image Credit: [Ladicky et al. ECCV ’10, ICCV ’09]

  36. Experiment #3 Input MAP Best of 10-Div (C) Dhruv Batra

  37. Experiment #3 DivMBest (Oracle) Better MAP 22%-gain possible Same FeaturesSame Model PACAL Accuracy DivMBest (Re-ranked) [Yadollahpouret al.] Confidence-based Perturbation (Oracle) #Solutions / Image (C) Dhruv Batra

  38. Summary • All models are wrong • Some beliefs are useful • DivMBest • First principled formulation for Diverse M-Best in MRFs • Efficient algorithm. Re-uses MAP machinery. • Big impact possible on many applications! (C) Dhruv Batra

  39. Thank you! • Think about YOUR problem. • Are you or a loved one, tired of a single solution? • If yes, then DivMBest might be right for you!* * DivMBest is not suited for everyone. People with perfect models, and love of continuous variables should not use DivMBest. Consult your local optimization expert before startingDivMBest. Please do not drive or operate heavy machinery while on DivMBest. (C) Dhruv Batra

More Related