1 / 15

MAP estimation in MRFs via rank aggregation

MAP estimation in MRFs via rank aggregation. Rahul Gupta Sunita Sarawagi (IBM India Research Lab) (IIT Bombay). Background. Approximate MAP estimation a must for complex models used in collective inference tasks Min-cuts, belief propagation, mean field, sampling

jewell
Download Presentation

MAP estimation in MRFs via rank aggregation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MAP estimation in MRFs via rank aggregation Rahul Gupta Sunita Sarawagi (IBM India Research Lab) (IIT Bombay)

  2. Background • Approximate MAP estimation a must for complex models used in collective inference tasks • Min-cuts, belief propagation, mean field, sampling • Family of Tree Re-weighted BP1 algorithms • Decompose graph into trees • Perform inference on trees and combine results • Can we generalize and do better? • Can we provide better upper bounds on the MAP score? 1[Wainwright et.al.’05, Kolmogorov ’04]

  3. Goal • Efficient computation of the MAP solution (xMAP), using inference on simpler subgraphs OR • Return an approximation (x*,gap) s.t. Score(xMAP) – Score(x*) < gap

  4. MAP via rank-aggregation • [Step 1] Decompose graph potentials into a convex combination of simpler potentials • E.g. Set of spanning trees that cover all edges = + + T1 + T2 + T3 G = Score(x) Score1(x) Score2(x) Score3(x) = + + =>

  5. Rank aggregation (contd.) • [Step 2] Perform top-k MAP estimation on each constituent and compute upper bound (ub) S1 : x1 x2 x3 x4 x5 x6 x7 x8 ub = score1(x8) Si : x7 x20 x4 x1 x9 x11 x8 x2 + scorei(x2) SL : x15 x6 x2 x8 x22 x3 x5 x4 + scoreL(x4) ties

  6. Rank-Merge • [Step 3] Merge the ranked lists using the aggregate function Score(x) = i Scorei(x) Computed directly from the model x1 x2 x3 x4 x5 x6 x7 x8 x2 x1 x8 x7 x20 x4 x1 x9 x11 x8 x2 x* (MAP estimate) x15 x6 x2 x8 x22 x3 x5 x4

  7. Rank-Merge (contd.) • If Score(x*) ≥ ub, then xMAP = x* • From the property of RankMerge algorithm • If Score(x*) < ub, then Score(xMAP) < ub • From convexity of ‘max’ and decomposition of G • Tighter bounds can be obtained by increasing k • Can do even better • Generate top-K and upper bounds ubi incrementally.

  8. TRW-BP Generates bounds only from K=1 Outputs xMAP only when all trees agree on a common MAP May require enumerating all MAPs if MAP is not unique Comparison with Tree Re-weighted BP Rank-Aggregation • Tighter bounds obtainable by increasing K • xMAP does not have to be in ALL the lists • No agreement criteria • Comparison with ub sufficient • xMAP need not be the best in any list.

  9. Reparameterization • No guarantee that xMAP will be in the top-K list of ANY tree • Need to align tree potentials with the max-marginals of the graph • Can use existing reparam algorithms: TRW-T, TRW-E, TRW-S • TRW-S most expensive but gives monotonically decreasing bounds and converges fastest. • Rank-aggregation gives significant improvements with all the reparameterization algorithms. G ’G reparam s.t. ScoreG(x) ´ Score’G(x) = iTi = i’Ti

  10. Final Algorithm Construct potentials for simpler constituents Get Top-k MAP estimates for each constituent Rank Aggregate the sorted lists Reparameterize graph potentials next iteration found (X*, 0) bored neither (X*, gap)

  11. Experiments (synthetic data) • Improves even upon TRW-S • Success in fewer iterations • Smaller gap values on failure • Effect much more significant for TRW-Tree and TRW-Edge Successes Failures

  12. Experiments (real data) • Bibliographic data • 24 labels (AUTHOR, TITLE, ….), avg. record size 11. • Uniqueness constraints for some labels → Clique models • Substantially better MAP estimates and Gap values, less number of iterations.

  13. Sensitivity to selection Behavior also depends on reparameterization! Experiments : Tree Selection Grid(M) Failures Clique(M) Failures Grid(M) Successes

  14. Experiments : Gap Evolution Grids Cliques • Gap converges for RankAgg in both scenarios • Cliques: Erratic gaps shown by TRW-S • Bounds are monotonic but MAP-estimates are not!!

  15. Summary and future work • Rank aggregation significantly better in MAP-estimation • Fewer iterations, Much tighter bounds, Low k. • No dependence on the tree-agreement criteria • Can generalize to non-tree constituents as long as top-k is supported. • Future work • Collective inference on constrained models • Intelligent constraint generation • Decrease sensitivity to tree selection

More Related