Viz : A Visual Analysis Suite for Explaining Local Search Behavior

1. Viz: A Visual Analysis Suitefor Explaining Local Search Behavior Steven Halim (School of Computing, NUS) Roland Yap Hock Chuan (School of Computing, NUS) Lau Hoong Chuin (School of Information Systems, SMU) http://www.comp.nus.edu.sg/~stevenha/viz User Interface Software and Technology 2006 Montreux, Switzerland

2. Outline • Short Background: • Local Search (LS) for attacking NP-hardCombinatorial Optimization Problems (COPs), • The problem: Understanding and Tuning LS… • Our Approach: • Explaining Local Search using Visualization (Viz) • Viz DEMO • Q & A

3. Background – Local Search • Local Search • e.g. Tabu Search, Iterated Local Search, Simulated Annealing • …is commonly used for attacking NP-hard (intractable)Combinatorial Optimization Problem (COP): • e.g. Traveling Salesman Problem (TSP) • Simplified explanation of Local Search:

4. Background – Tuning Problem • Local Search algorithms have tunable configurations: • Parameter values, • Components, and/or • Search strategies. • These configurations affects its behavior  performance… • To make a LS perform well in context is not straightforward. • Its heuristic/stochastic behavior is not well understood. • Empirical analysis is needed.

5. Common Practice: Ad Hoc Tuning Bottleneck !!! Implement the local search Evaluate its performanceTextually/Statistically… Modify the local search using something??(“ad hoc”) Good or Give up Not good Stop

6. We propose: Visual Diagnosis Implement the local search Evaluate its performanceUsing Visualization Tool… Modify the local search after understanding what is happening. The tool helps us to think more systematically. Good or Give up Not good Stop

7. Why Use Visualization? • Visualization is a cognitive tool. • Human brain is a powerful pattern finding machine. • As long as the data is visualized in a way that conformwith the rules of human visual perception. • Information Visualization: “The use of interactive visual representations of abstract data to amplify cognition.” • Facilitates hypothesis formation in empirical analysis. • Allows users to answer questions they didn't know they had.

8. Explaining Local Search Behaviorvia Visualization http://www.comp.nus.edu.sg/~stevenha/viz

9. Explaining Local Search Behavior It is more useful to understand how and why something works, Rather than just simply know that something works… By understanding the underlying principles, We can adjust properly if presented with different scenarios! To understand LS, we need to answer these questions: • Does it behave like as what we intended? • How good is the local search in intensification? • How good is the local search in diversification? • Is there any sign of cycling behavior? • How does the local search algorithm make progress?

10. Explaining Local Search Behavior • Where in the search space does the search spend most of its time? • How far is the starting (initial) solution to the global optima/best known solution? • Does the search quickly find the global optima/best known solution region or does it wander around in other regions? • How wide is the local search coverage? • What is the effect of modifying a certain search parameter/component/strategy w.r.t the search behavior? • How do two different algorithms compare? The behavior of heuristic and stochastic local search is hard to explain without appropriate tools!

11. How to Understand LS Behavior? • Various local search analysis tool: • Data Analysis (text-based): • Solution Quality • Robustness • Speed/Running Time • Run Time/Length Distribution [Hoos 98; Hoos & Stuetzle, 05] • Fitness Landscape (Statistical) Analysis [Jones, 95; Watson et al., 05] • Information Visualization: • Visualization of Fitness Landscape/properties/search behavior

12. Visualizations for Local Search • Existing Visualizations: • Objective Value over Time • Fitness Distance Correlation [Jones, 95; Watson et al. 05] • Problem Specific, e.g. TSP [Klau et al., 02] • 2-D Animation of 2 Dimensional Problem [Syrjakow & Szczerbicka, 99] • N-to-2-Space Mapping [Kadluczka, 04] • Search Trajectory Analysis [Lau et al., 05; Halim et al., 06a,b]

13. Analogy of Search Trajectory Visualization:Finding the Highest Mountain

14. 1. Without anchor points, the behavior of the pink trajectory is hard to be explained. 2. Do several local search runs with different configurations, record diverse local optima/anchor points (circled). 3. With anchor points, the behavior of the pink trajectory is as follows: trapped in region that contains red/blue anchor points, thus failed to visit good solutions, the green/orange anchor points. 4. The behavior of thepale blue trajectory is as follows: after reaching a local optima, it diversifies to another place. It manages to reach green and orange anchor points, and thus its performance is better than the pink trajectory in Figure 3.

15. Anchor Points are quite close to each other. Trapped in cycling near Anchor Point C Only covers regions near Anchor Point D and E Trapped in cycling near Anchor Point C because it is attractive (green) Only covers regions near Anchor Point D and E, which are good regions (blue and green) Anchor Points are quite close to each other, but their quality are different. • Search Trajectory VisualizationIn Practice • Layout the points in Abstract 2-D Space • Points that are close in N-dimensional space in terms of distance metric (hamming, permutation distance, etc)are laid out close to each other in the abstract 2-D space and vice versa. • This utilizes human strength in discerning 2-D spatial information. • Layout First Phase: • The anchor points are measured with each other using distance metric. • The anchor points are installed greedily in abstract 2-D space • Re-optimize using the Spring Model layout algorithm. • Layout Second Phase: • Again, using Spring Model algorithm, the points along search trajectory are aid out in abstract 2-D space using these anchor points as reference. • Presentation Aspects: • Color coding is used to enhance our understanding: blue: good, green: medium, brown: pooranchor points. • The search trajectory is animated over time.

16. Explaining Local Search Behaviorusing Viz: Local Search Visual Analysis Suite http://www.comp.nus.edu.sg/~stevenha/viz Note: we use the updated version of Viz for the demo.

17. Design Goals • Enable algorithm designer (human) to understand LS behavior. • Must obey Information Visualization Rules: • Overview first, Zoom and Filter, then Details on Demand. • Utilize human visual perception strengths. • Interactive. • Responsive. • Aim for widespread use: • Generic: • Support various (/all) Local Searches. • Support various (/all) Combinatorial Optimization Problems. • Viz as Analysis Suite. • Facilitates information sharing with other algorithm designers.

19. Using Viz

20. Viz Features (Overview) • Multi-Source Visualizations • Visual Comparison • Animated Search Playback • Information Visualization + Data Analysis • Color and Highlighting • Multiple Level of Details • Viz GUI is: • User friendly, use familiar widgets • Interactive • Responsive • Customize-able

21. Multi-Source Visualizations • The local search playback are viewedfrom several angles: • Generic • Search Trajectory • Objective Value • Fitness Distance Correlation • Event Bar • Algorithm Specific • Problem Specific • Coordinated Visualizations

22. Visual Comparison • Human is creature of comparison. • Actually, human is not a perfect judge. • But, good at relative comparison when given a baseline. • Two comparison modes: • Juxtapose • Superimpose • Useful for understanding: • Behavior: Same LS & configuration, different parts of search • Robustness: Same LS & configuration, two runs • Effect of Modifications: Same LS, different configurations • Overall behavior: Two totally different LS

23. Animated Search Playback • Animations • Local Search info is too big: cannot be presented at one time. • But rather: animated over time.(the search trail is drawn using alpha blending). • Local Search typically runs in thousand iterations… • Analyzing from start to end may be overwhelming • We use Event Bar for quick reference to interesting regions. • Viz computes highlights/index points for generic events(e.g. new best found solution)

24. Info Visualization + Data Analysis • We combine strengths from both worlds. • Information Visualization • Data Analysis (Textual/Statistics)

25. Viz DEMO http://www.comp.nus.edu.sg/~stevenha/viz

26. Viz is Useful for: • Local Search Algorithm Analysis tool: • Individual • Addressing Tuning Problem • Debugging • Improving algorithm • Collaborative • Researchers can annotate and publish *.viz file for information exchange. • Benchmarking • Pedagogical tool: • For Studying/Teaching Local Search Algorithms. • Presentation tool: • To illustrate behavior of a new Local Search Algorithm.

27. Summary • We have seen that proper visualization can be used to help explaining/understanding local search behavior. • Some of these are hard to derive without proper visualizations. • A necessary step for tuning/improving local search. • To be useful, a visualization tool should utilizehuman visual perception strengths. • Viz’s capability in explaining local search is so far promising. • Lessons for UIST: • Regardless whether your research is on Local Search or not. Hopefully you can get something from what I share here with my visualization tool Viz  • Careful application and extrapolation of existing techniques in a new domain can result in fascinating extensions. (multidisciplinary). • Many other fields can be enhanced using proper Information Visualization tools, developing specific visualization for these domains may be fruitful.

28. End of TalkTime for Q & A Thank you for your attention  For further details and to download Viz, please visit:http://www.comp.nus.edu.sg/~stevenha/viz