Solving Jigsaw Puzzles

1. Solving Jigsaw Puzzles Presentation by Perouz Taslakian McGill University COMP644 – Winter 2005

2. Presented Paper “A Global Approach to Automatic Solution of Jigsaw Puzzles” by David Goldberg Christopher Malon Marshall Bern Proc of the 18th Annual ACM Symposium on Computational Geometry; 2002 June 5-7; Barcelona; Spain. NY: ACM; 2002; 82-87.

3. Introduction • Solutions for jigsaw puzzles by shape alone date back to 1964 • Two main difficulties : • Combinatorial: Too many ways in which pieces can be assembled • Geometric: difficult to detect if a pair of pieces really match.

4. Standard Rules • Rectangular outside border • Each interior piece has 4 primary neighbors • Pieces interlock by “tabs” (indent & outdent) • Each piece has no neighbors except its primary neighbors

5. F B E C P Indent A D Outdent Standard Rules • The primary neighbors of piece P are the pieces A, B, C, D

6. Algorithm Overview • Find the ordering of the border pieces • Embed the border pieces in the plane • Find all the pockets • Fill in the pockets • Optimize • Repeat steps 3 – 5 until no more pieces are left

7. Data Acquisition • Use color copier to copy pieces and then scan the copies (at 300 dpi) • Extract pieces by computing the connected components of the complement of the background • Use morphological operations to smooth the pieces • Take every other pixel on the boundary and do Gaussian smoothing

8. 2 1 3 5 4 Finding Tabs • Find indents • Mark the boundary near the indents • Find outdents along the unmarked parts of the boundary

9. s(B, A) A B s(A,B) Ordering the Border • s(A,B) = how well the right side of Afits the left side of piece B • Use an asymmetric TSP heuristic to find an ordering of the border pieces B A

10. Computing s(A, B) • s(A, B) = • Best case: x1 = x2 … = x5 s(A,B) = 0 A B x1 x5

11. For each piece A we know its right & left neighbor To align adjacent pieces, we define 1 fiducial pointper tab (indents and outdents) fiducial point: the centerof a fitted ellipse passing through the two inflection points Embedding the Border in the Plane

12. Using the fiducial points, pick points along the ellipse which are equiangular w.r.t. its center For two pieces A and B to fit, the corresponding tab points of each piece has to fit. Embedding the Border in the Plane

13. To align A and B, use a least-squares fit to find the best rigid motion that minimizes the sum of the squared distances between corresponding points Embedding the Border in the Plane A B a’ a c’ c b b’

14. Problem!

15. A B ziB ziA Global Relaxation • Distribute error equally among pieces • Pick k = 30 pairs of corresponding points along each common border of two pieces, and minimize the sum of the squares of all intra-pair distances. • Min

16. At each step we find all the eligiblepockets Placing Interior Pieces

17. For each eligible pocket P, and for every piece A we compute: The position of A if it really belongs to P The score of A using this position Placing Interior Pieces A P

18. The position of A is the one that minimizes the sum of squares of distances between corresponding fiducial points of A and P Placing Interior Pieces Fiducial pt. A

19. The score of A is the average distance between a boundary vertex of A & its closest pocket point in P Placing Interior Pieces tangent pt. A

20. For each pocket Pi Ai = 1st best fit (min score) Bi= 2nd best fit Fill the Pi with the Minimum ratio: Ai / Bi P1 P4 P2 P3 Highest-Confidence Placement

21. For each pocket Pi Ai = 1st best fit (min score) Bi= 2nd best fit Fill the Pi with the Minimum ratio: Ai / Bi Highest-Confidence Placement P1 P4 P21 P22 P3

22. Using a Sun Ultra-60 workstation, it took: 3 minutes to solve a 100-piece puzzle 20 minutes to solve a 204-piece puzzle Techniques applicable to other problems, for example shattered glass reconstruction. Experimental Results

23. Thank You