Sketch Beautification: Progress and Plans

Sketch Beautification: Progress and Plans 2005-09-05 Weekly Seminar Kaisuke Nakajima

Motivation It’s hard to draw diagrams on computers • for documents, slides, ...

Goal Facilitate diagram drawing • As easy as (or easier than) sketching

Previous work • Many systems with sketching interface • But: not mature enough for practical use • Limited types of shapes • Problems with ambiguity

Limited types of shapes (1/2) Lines, circles, arcs [Igarashi et al. 1997] [Arvo and Novins 2000] [Unpublished; Rejected for Eurographics 2005]

ellipse Limited types of shapes (2/2) But: supporting more shapes is not so easy • If we add support for ellipses... • How can we draw a perfect circle by sketch? circle An ellipse never fits worse than a circle...

Problems with ambiguity • No support for error correction • Too many recognition candidates Be identical!  [Pavlidis and Van Wyk 1985] Too many candidates shown in a small area!  [Igarashi et al. 1997]

Focus • Support richer set of shapes • Circles, arcs, ellipses, and Bézier curves • Better handling of ambiguity • By a better recognition algorithm & UI

Rest of today’s talk • Current implementation • Plans

Current Implementation

Status • We can beautify a (very) simple diagram

Algorithm overview 1. Fit primitive shapes to strokes 2. Infer geometric constraints 3. Solve obtained constraints

1. Fit primitive shapes • For each stroke • Fit each shape  Choose the best shape • Supported shapes • Only lines & circles for now...

2. Infer constraints • Example: • Others: • length[i] = length[j] • angle[i] = angle[j] • distance(point[i], point[j]) = length[k] for all (point1, point2) if (distance(point1, point2) < 32) add constraint(point1 == point2);

3. Solve constraints External constraint solver: Chorus [Hosobe 2001] ChSolver s = new ChSolver(); ChVariable2D p1, p2; ChVariable d; ... s.add(new ChEuclideanConstraint(p1, p2, d)); s.solve(); //  distance(p1, p2) == d

Flaws • Too rough & ad-hoc • Too few types of shapes & constraints • Irresponsible thresholds • Unstable & slow • Redundant (or unsolvable) set of constraints if (distance(point1, point2) < 32) add constraint(point1 == point2); add constraint(point1.x == 10); ... add constraint(point1.x == 20);

Plans

Plans • Handling constraints • Handling ambiguity • Adding more shapes • Extensions

apply constraint #1: ignore constraint #2: Handling constraints (1/3) add constraint(point1.x == 10); ... add constraint(point1.x == 20); • Select a proper subset of constraints • When we have N possible constraints: • There are candidates

Handling constraints (2/3) • Less candidates actually • To bind V variables, V constraints are enough • e.g. If P points are to be beautified, V = 2P • candidates • V = 4 in Pegasus [Igarashi et al. 1997] • V ≫ 4 occurs in our approach  Impossible to try them all

apply constraint #1: ignore Bad! Good! constraint #2: Handling constraints (3/3) • Pick up only the better few candidates • cf. game tree search • Evaluate candidates & prune bad branches • Evaluation factor: error from original stroke, beauty

Handling ambiguity (1/3) • 100% accurate recognition is impossible • Worst way: ignore errors! • Better way: choose from candidates • Still bad if too many candidates

Handling ambiguity (2/3) • cf. Japanese IME (input method) 78 kanji’s in no specific order!  Much user friendly 

Handling ambiguity (3/3) • Analogy: Let the use redraw • The system can reduce candidates Ambiguity (Noise/Signal ratio) decreases by redrawing.

circle ellipse Adding more shapes • Redrawing can handle the ellipse-or-circle problem • Which candidate should come first?  Determine based on user profile (logging)

Extensions • Coloring • Embedding in text • Designing fonts

Extensions: Coloring • Coloring of vector graphics can be difficult • e.g. Adobe Illustrator requires complicated path editing commands

v u angle Extensions: Embedding (Takeo’s idea.) • Embed diagrams in a text file • Not a separate file Pop up!

Extensions: Designing fonts • Fonts: good target for testing • Difficult to design & modify • Partial similarity (curves, serifs, ...)

Schedule • Job priority (from high to low) • Improve candidate search • Implement user redrawing • Add more shapes / constraints • Extensions • Quite urgent... • WISS 2005 Demo (Due 9/23)

Summary • I want to facilitate diagram-drawing • Previous systems not mature for practical use • Implemented a rudimental prototype • Much improvements to be done Thanks!

Sketch Beautification: Progress and Plans