Jigsaw Puzzles with Pieces of Unknown Orientation

1. Jigsaw Puzzles with Pieces of Unknown Orientation Andrew C. Gallagher Eastman Kodak Research Laboratories Rochester, New York

2. Outline • Introduction • Solving Puzzles • Measuring Pairwise Compatibility • Tree-Based Reassembly for Types 1 and 2 • An MRF for Solving Type 3 Puzzles • Experiments • Conclusion

3. Introduction • Solving puzzle assembling the pieces of a jigsaw puzzle into a complete picture

4. Introduction

5. Introduction • Puzzle type • Type 1: known Rotation, unknown Location • Type 2: Unknown Rotation and Location • Type 3: Unknown Rotation, known Location

6. Solving Puzzles • a measure of jigsaw piece compatibility • puzzle assembly

7. Solving Puzzles • Measuring Pairwise Compatibility • describes the local gradients near the boundary of a puzzle piece • Mahalanobis distance

8. Solving Puzzles • Measuring Pairwise Compatibility compute the compatibility DLR(xi, xj) of a jigsaw piece xj on the right side of piece xi mean distribution: also compute the covariance matrix SiL

9. Solving Puzzles • Measuring Pairwise Compatibility compute the gradient from the right side of piece xi to the left side of piece xj then, Mahalanobis distance:

10. Solving Puzzles • Measuring Pairwise Compatibility modified the above equations to compute DRL(xj, xi), then get the symmetric compatibility measure CLR(xi, xj) store the confidence ratio in the 3D array S(xi, xj, r)

11. Solving Puzzles • Evaluation in Puzzle Assembly Similarity performance for types 1 and 2

12. Solving Puzzles • Tree-Based Reassembly for Types 1 and 2 • a greedy assembly algorithm inspired by Kruskal’s algorithm for finding a minimal spanning tree • three stages: • constrained tree stage • Trimming • Filling

13. Solving Puzzles • Tree-Based Reassembly for Types 1 and 2 • The constrained tree stage • nothing prevents the MST from being a graph that results in an assembled puzzle that overlaps onto itself • If a collision has occurred then the edge is discarded without merging the forests

14. Solving Puzzles • Tree-Based Reassembly for Types 1 and 2 • The constrained tree stage

15. Solving Puzzles • Tree-Based Reassembly for Types 1 and 2 • Trimming and Filling

16. Solving Puzzles • An MRF for Solving Type 3 Puzzles An natural function to minimize is the total sum of the cost across the boundaries of any two pieces

17. Experiments • Four measures • Direct comparison • Neighbor comparison • Largest Component • Perfect Reconstruction

18. Experiments • Type 1 Puzzles • Type 2 Puzzles

19. Experiments • Type 3 Puzzles • orientation accuracy is 97.2% when considering puzzles with 432 pieces each with 28 × 28 pixels • Result

20. Experiments • Mixed-Bag Puzzles

21. Conclusion • a new class of square piece jigsaw puzzles that having pieces with unknown orientations • a new measure (MGC) for the compatibility of a potential jigsaw piece matches • a tree-based reassembly that greedily merges components • a pair-wise MRF where each node represents a jigsaw piece’s orientation