VERY LARGE TEXT FILE VIEWER Project presentation

1. By: David Nasi & Amitay Svetlit Supervisor: Oved Itzhak VERY LARGE TEXT FILE VIEWERProject presentation Software Systems Lab Department of Electrical Engineering Technion - Israel Institute of Technology

2. Motivation • Conventional text file viewer applications poorly suitable for very large text files (user response time wise). • Conventional text file applications do not provide tools for creating custom search and index.

3. Project Objectives • A user friendly graphic interface specialized for viewing VLTFs. • Application responsiveness is independent of input file size. • Extensible indexing architecture for employing indexing by arbitrary criteria.

4. User Interface Design Main Features: Text View Pane Search Results Pane Search Bar Go-to-Line Bar Progress Bar Scroll Knob

5. User Interface Design – Scroll Knob Scrolling over a VLTF using a conventional scroll bar may be very tedious Scroll Knob functionality allows to scroll over the VLTF with adaptive speed The speed is adjusted by a polynomial factor dependant on the distance of the knob from the center

6. Software Design – Segmentation Basic Concept • Goal: • Create a highly responsive viewer when displaying VLTFs (in the order of GBs). • Challenge: • Simple file access patterns (i.e. reading the entire file upon opening) are unsuitable. • Approach: • Segmentation: Upon opening the VLTF is partitioned to fixed-size segments. The segmentation process maps line numbers to segments in which they are held. Each segment holds only complete lines.

7. Segmentation - Example

8. Software Design – Estimation Basic Concept • Goal: • Create a highly responsive viewer when displaying VLTFs (in the order of GBs). • Challenge: • Since segmentation requires reading the entire file, a question arises about the user experience before the file is fully indexed. • Approach: • Using some metrics (average number of lines in a segment) and provide the user with the estimated line with an indication that this is only estimation.

9. Software Architecture – Three Tier

10. Data Structure Layer Responsibilities: • Segmentation Process • Estimation metrics gathering • Serving Business Logic data request

11. Data Structure Layer - Asynchronous Request Handling • Queue of Read Requests from the Business Logic • No duplicate segment requests are handled • The requests are handled in a FIFO order • Each request is processed on a separate thread • A callback to the Business Logic transferring the data and meta data for analyzing it - sessionless principle

12. Business Logic Layer Responsibilities: • Receives read request from the GUI • Serves as cache for the GUI layer requests • Sends data requests to the Data Structure • Custom Search and Index capabilities

13. Custom Search & Index Capabilities • We provide the developers with an abstract class of a searcher • Through implementing this class it’s possible to: 1. Construct a custom index 2. Interconnect with the GUI performing arbitrary search requests

14. Choosing a Custom Searcher

15. Search Progress

16. GUI Read Requests Handling • As a result of the paging work being done on separate threads library functions are used in order to update the GUI controls • Race Condition between different requests is handled by assigning a time stamp ID to each request

17. Design Components

18. Class Overview

19. End • Many thanks to our dedicated supervisor, Oved Itzhak for his great help and support • Thanks to the software lab staff for the support (Ilana David and Victor Kulik)