Interactive Display Models for Information Visualization in Virtual Reality

1. Interactive Display Models for Information Visualization in Virtual Reality Jonathan I. Maletic, Ph.D. Software DevelopMent Laboratory <SDML> Department of Computer Science Kent State University (formerly with The University of Memphis)

2. General Research Topic The research investigates the use of Virtual Reality Environments (VE) to visualize abstract (complex) data and information. Much of the work in VR is concerned with the representation of real (physical) objects. Little work, in VR, has address abstract concept visualization.

3. Specific Problem Domain We are particularly interested in the visualization of large scale software systems. • Visualize design and architectural information • Reduces, in part, to (large) connected graphs • Nodes represent complex entities i.e., software module, class, function, component, subsystem, etc. • Edges represent abstract relationships between the nodes i.e., aggregation, association, inheritance, invocation, etc.

4. What is Software Visualization? Building visualization environments to support: • Software Development, • Maintenance, Re-engineering, • Project Management of large scale software systems. Note: We are not addressing algorithm animation or visual programming languages.

5. A Framework for Software Visualization Software visualization environments can be described by five characteristics: • Target – what to represent? • Medium – where to represent? • Representation – how to represent? • Audience – who will use the visualization? • Tasks – why is the visualization needed?

6. Target • Static features (relationships) • Design-level information • Source code-level information • Documentation • Dynamic features (behavior) • Control and data flow at execution • Trace information

7. Existing Mediums for Software Visualization • The Desktop Display • High resolution but limited display area • 2D+ (for the most part) • Paper Documents • 2D only, poor navigation, static • White board • 2D only, static

8. Additional Features Needed for Software Visualization Mediums • More real estate. • Support for collaborative problem solving • Utilize: • 3D, 3D+ • Virtual Reality (combine stereopsis with motion) • Immersive Environments • Stereo displays, Multi-resolution displays • Multi-type medium (e.g., laptop + VE) • Provide navigational controls

9. Representation • Cognitive based - the building blocks of the visual language could be geon-based and map to natural concepts/abstractions • User centric (versus compiler centric) – the visualization should present features of the software in concepts from the user’s universe. • Does not overload the user – each element should have multiple attributes, but there should be a limit on the diversity. This limit should be driven by cognitive factors and the medium.

10. Additional Features of Representation • Visualize multiple levels of abstraction: • Source code • Design • Design Patterns • Architecture • Mapping between abstraction levels (e.g., drill down).

11. Audience • Experienced developers can handle multiple abstraction levels • they need access to both design- and code-level information, as well as to the dynamic features. • Project managers • they need access to design- and process-level information. • they might not be skilled programmers.

12. Tasks • Support for industrial-size systems and processes. • Software development: • Design, product evolution, etc. • Reengineering • Comprehension, visual debugging, etc.

13. Visualizing Software in Virtual Reality Environments • Developing a research prototype to experiment with different display models and languages • IMSOvision – IMmersive SOftware VISualizatION

14. IMSOvision – an overview Source Code (C++) and Design Documents Translator: -Parsing -Metrics -Analysis LOOC in VRML Interaction

15. IMSOvison: Target • Currently object-oriented software systems. • Static features • Design-level information (e.g., class diagram) • Source code-level information (metrics) • Dynamic features (future work) • Control and data flow at execution (much like a visual debugger).

16. IMSOvison: Medium • Stereoscopic projection (DLP) displays • Passive/active monitor • Virtual Environments • CAVE (immersive), ImmersaDesk • Utilize VRML and/or OpenGL

17. The CAVE Commercialized by Pyramid Systems & VRCO Images from EVL

18. The ImmersaDesk I, II, III Images from EVL

19. Advantages of VR • Offers “limitless” real estate. • Combines stereopsis and motion. • Makes full use of 3D space. • Natural and easy navigation. • User-centric representation. • Collaborative problem solving.

20. IMSOvision: Representation • Visual language (built in VRML, OpenGL) • The building blocks of the language are: • User centric (rather than compiler centric) • Natural • Support multiple abstraction levels • Does not overload the user • Maps naturally between abstraction levels

21. Elements of the Visual Language (1)

22. Visual Language (2)

23. Visual Language (3)

24. Example • Simple voice mail system • UML Class Diagram • Written in C++

25. Mail System in Imsovision

26. Elements of the Visualization

27. Information Hiding

28. Demo Run demonstration

29. Future Work • More complete automation of the translation into the VL. • Visualization of more features of the software, both dynamic and structural. • User studies. • Experiments on collaborative problem solving and project management.

30. Students • Graduate Students Funded: • Greg Dunlap – Memphis - M.S. Graduated 12/2001 • Andi Marcus – Kent (formerly at Memphis) - Ph.D. Student • Mike Collard – Kent – Ph.D. Student • Louis Feng – Kent - M.S. Student • Undergraduate Students Involved: • Corey Donahoe – Memphis • Leslie Saputra – Memphis - B.S. 12/2001

31. Publications • Maletic, J.I., Collard, M.L., Marcus, A.,  “Source Code Files as Structured Documents”, in Proceeding of  the 10th IEEE International Workshop on Program Comprehension (IWPC'02), Paris France, June 27-29, 2002, pp. (to appear). • Maletic, J.I.,  Marcus, A., Collard, M.L., “A Task Oriented View of Software Visualization”, in Proceeding of  the IEEE Workshop on Visualizing Software for Understanding and Analysis (VISSOFT 2002), Paris France, June 26, 2002, pp. (to appear). • Dunlap, G., Visualizing Object Oriented Software in a Virtual Reality Environment, M.S. Thesis, The University of Memphis, December, 2001. • Maletic, J.I., Leigh, J., Marcus, A., Dunlap, G.,  “Visualizing Object Oriented Software in Virtual Reality”, in Proceedings of the 9th IEEE International Workshop on Program Comprehension (IWPC'01), Toronto, Canada, May 12-13, 2001, pp. 26-35. • Maletic, J.I., Leigh, J., Marcus, A.,  “Visualizing Software in an Immersive Virtual Reality Environment”, in the proceedings of the ICSE'01 Workshop on Software Visualization, Toronto, Canada, May 13-14, 2001, pp. 49-54.