NVIS: An Interactive Visualization Tool for Neural Networks

1. NVIS: An Interactive Visualization Tool for Neural Networks by Matt Streeter advised by Prof. Matthew O. Ward and Prof. Sergio A. Alvarez

2. What is a Neural Network? • Weighted, directed graph, organized into layers • Set of neurons (nodes) and synapses (edges), with signals transmitted between neurons via synapses • Valuable tool for pattern recognition and function approximation

3. Why create a visualization tool for neural networks? • Understand how neural networks work, gain insight into problem being solved • Understand how genetic algorithm evolves networks • Other tools exist, but do not show neuron activations or genealogical relationships

4. Feedforward network visualization • Synapse strength represented by length and brightness of colored bars (linear scale). Blue lines indicate positive weights; red lines indicate negative • Diameter of white circles represents neuron’s output or activation • Each weight acts as a slider

5. Compact matrix representation • Purpose is to allow many networks to be displayed on the screen at once • One matrix for each level of weights • Row x, column y of matrix n represents weight from node y of layer n to node x of layer n+1 (same colors)

6. Generations & family trees • Row of compact matrix for each generation, ordered by fitness • User can select any network in the population history • Separate window shows family tree of selected network

7. Interactive environment • Set evolution strategy, network architecture, and training set • Graph representation and family tree available for any network in population history • Load/save networks • Real-time fitness graph

8. Designing networks • By dragging weights, user can design a network to solve a problem, or refine a network that has already been trained • Real-time display of fitness score; easy to see importance of particular weight • Not a practical way to find a network to solve a problem

9. Understanding genetic drift • Genetic drift is tendency for members of artificial populations to all be alike • Initial diversity in generations 0-2, rapidly lost in generations 3-5 • Best (leftmost) network in generation 3 is parent of best network in generation 4, grandparent of best 8 in generation 5, and ancestor of all later networks (not shown)

10. Changing weights & local optima • Error backpropagation algorithm performs gradient-based search (local optimum) • Weight dragged while backprop is running will either “snap back” to original optimum, or all weights will shift to new optimum • Can estimate the length of a local optimum with respect to each axis in weight-space

11. Extracting domain knowledge • Positive weights in all but first layer; effect of input nodes therefore directly related to incident weights • Higher crime rates (C) tend to reduce value of house; higher number of rooms (R) tends to increase value • Analysis could be applied to problem domains where no a priori knowledge exists

12. Future work • Graph representation does not scale well • Implement a variety of evolutionary algorithms (breeding & selection schemes) • Depict network architectures other than feedforward • User evaluation

13. For more information . . . • Visit http://www.wpi.edu/~mjs/mqp • See technical report WPI-CS-TR-00-11, available at: http://www.cs.wpi.edu/Resources/techreports/index.html Questions?