1 / 26

VisualizeME

VisualizeME. Kia Manoochehri. Motivation. Data Analysis Learn about AWS Struggle to do so Data Visualization An exercise in scalability. Motivation. Do something different. Introduction. Original intent: Gather Windows Event Log files Determine when an error would occur

lora
Download Presentation

VisualizeME

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VisualizeME Kia Manoochehri

  2. Motivation • Data Analysis • Learn about AWS • Struggle to do so • Data Visualization • An exercise in scalability

  3. Motivation Do something different.

  4. Introduction • Original intent: • Gather Windows Event Log files • Determine when an error would occur • Visualize this somehow

  5. Introduction • Problems: • Gather Windows Event Log files • No access to them outside of my own • Determine when an error would occur • Easy if I had access to a monumental amount of data • Visualize this somehow • Complicated because *what* would I visualize?

  6. Introduction • Actual design: • A Log File (data) Analyzer of a different system • Ran through AWS to exploit parallelization of the log files • Exercise in scalability • Visualize this data in a “meaningful way”

  7. Side note about the Data Came from another project I previously worked on .txt files of varying size Total Volume of data = 6gb

  8. Potential Results • Show that using AWS can relieve problems • More data? • Errors? • Runtime? • Have a cool visualization tool! • Data in an easy to read way.

  9. Tasty Preview of the Result

  10. Tasty Preview of the Result

  11. Organization

  12. Design Decisions • MATLAB • Many pros • Many many cons • AWS • We weren’t given any other option • Project itself

  13. Problems Encountered • MATLAB • Able to use the “Parallel Computing Toolbox” • Costs $$$ - Need licenses for the cloud • Solution? • Be dirty…

  14. Problems Encountered • AWS • Came into this class with 0 cloud experience and knowledge • Solution? • Spent more time outside of class learning and reading about the cloud and running an application on AWS than inside of class.

  15. Problems Encountered • Project itself • Multiple aspects to this project • Analyzer (coding) • Running it on AWS (design, choices, and 0 exp) • Visualizing the data (coding and design) • On the surface: • Not an impressive use of AWS • Good lesson on Scalability aspect of Cloud Services • Great lesson on Trust/Security of data…

  16. Implementation

  17. Implementation MATLAB on EC2

  18. Implementation

  19. Implementation • Development and Testing: • My home desktop and laptop • GIT makes this easy • After Toby’s presentation on AWS early in the semester I chose to isolate my development and testing environment (from the cloud).

  20. Performance • Scalability: • Time and effort saving • Fundamentally, space saving • >6gb (6144 MB) of .txt files compressed/converted to 229MB (wow) • Visually pleasing?

  21. Lessons Learned • AWS • Application of some of the topics we discussed • First hand account of security issues and reluctance to use the Cloud • Don’t over commit… • I was already addicted to Caffeine.

  22. DEMO…?

  23. Demo Fail???

  24. Demo Fail???

  25. Demo Fail???

  26. Demo Fail???

More Related