1 / 35

Putting Big D ata to Work

AURIMS/ANZUIAG Conference 2014. Putting Big D ata to Work. Who Am I. Mathew Benwell Information Security Specialist at the University of Adelaide Worked in Information Security for 8 years. First, a Disclaimer.

roary-holloway
Download Presentation

Putting Big D ata to Work

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AURIMS/ANZUIAG Conference 2014 Putting Big Data to Work

  2. Who Am I • Mathew Benwell • Information Security Specialistat the University of Adelaide • Worked in Information Security for 8 years University of Adelaide

  3. First, a Disclaimer • I work in a highly technical field, there will be a technology slant to this talk! • However, the concepts in this talk translate to non technical fields • My experiences are with a specific product called Splunk University of Adelaide

  4. About This Presentation • What is Big Data? • Big Data at the University of Adelaide • Technology Use Cases University of Adelaide

  5. What is Big Data Volume • Big Data 3 V’s Velocity Variety University of Adelaide

  6. How is Big Data Useful? • Analyse very large data sets quickly • Add context using variety • Can help spot unusual events University of Adelaide

  7. How is Big Data Useful? • Analysis • Arithmetic operations • Trending • Anomalous data University of Adelaide

  8. How is Big Data Useful? • Visualisations University of Adelaide

  9. A Simple Big Data Analytics Process University of Adelaide

  10. Big Data and Audit • Why wait for the good old 90 day review?? • Why not have our Big Data system tell use when an interesting event occurs? • Why not take it a step further and add context • Advise system owner at the time it occurred University of Adelaide

  11. Big Data and Audit • During an Audit we ask lots of questions The Question: • Who maintains access to privileged information? • More specifically, we aim to identify those with unauthorised access to privileged information Data that could support an answer: • System logs of changes to user groups • List of groups which maintain privileged access • Change system records University of Adelaide

  12. Big Data and Audit Question:Is Domain Admins group restricted to authorised IT personnel? Required Data: Current Members + Active Directory event log that fires when someone is added to the Domain Admins group John Doe added to Domain Admins Active Directory BIG DATA SYSTEM Alert • Could be any question: • Monitoring changes to bank transaction file • Monitoring anomolous pay runs • Overrides in requisition request • Mismatched invoices University of Adelaide

  13. Big Data and Compliance • Assist with Compliance to standards • Payment Card Industry – Digital Security Standard (PCI-DSS) • ISO 27001 University of Adelaide

  14. Big Data and Compliance • PCI-DSS • Many technical controls • Identify credit card data • Known pattern • On the network • Emails University of Adelaide

  15. Big Data and Risk • We could use Big Data to identify financial risks • Help prioritise risk treatment • Identify unusual events • Transaction without a purchase order • Higher than normal transaction • High volume or scheduled, low value transactions University of Adelaide

  16. Big Data and Risk • Profiling financial transactions • Say we see a regular payment that occurs routinely • Imagine the transaction one day starts occurring more frequently, or the transaction value changes significantly? • This would be worth investigation University of Adelaide

  17. About This Presentation • What is Big Data? • Big Data at the University of Adelaide • Technology Use Cases University of Adelaide

  18. What is Splunk • First the most asked question! Where did the name come from? • Derived from the word ‘Spelunk’ ‘to explore caves, especially as a hobby’ Our customers told us that finding their IT problems was like "digging through caves with headlamps and helmets, crawling through the muck" University of Adelaide

  19. What is Splunk • Software that can be used to store, analyse and report on Big Data! • Simple licence model, based on the total volume of data consumed daily • Highly scalable. Performance is only limited by hardware resources University of Adelaide

  20. What Data Can Splunk Consume • Machine data, any data generated by a computer • System logs • Text files • Databases • Output from systems University of Adelaide

  21. Getting Data into Splunk • Getting data into Splunk • Syslog • Splunk Forwarder • Tail/dump any local file • Windows registry • WMI • Script • Active Directory • DB Connect – Oracle, MSSQL, MySql, PostGres • API – Push data using Splunk API University of Adelaide

  22. Splunk at the University of Adelaide • Community driven collaboration University of Adelaide

  23. Splunk at the University of Adelaide • Initially purchased for the Security team to help deal with the ‘Phishing’ problem • Uses are expanding significantly • Quick Statistics • 3 Primary Servers • Total 19TB storage capacity • 89 billion events, 30 event sources University of Adelaide

  24. Splunk at the University of Adelaide • Google for your data University of Adelaide

  25. Splunk at the University of Adelaide • More than Google for your data University of Adelaide

  26. Splunk at the University of Adelaide • Analysis University of Adelaide

  27. About This Presentation • What is Big Data? • Big Data at the University of Adelaide • Technology Use Cases University of Adelaide

  28. Use Case – Vulnerability Data • System vulnerability data (Nessus, Nexpose, Qualys, etc) University of Adelaide

  29. Use Case – Vulnerability Data • Add context, this data becomes far more Useful! • Is the system accessible from the Internet (Firewall policies) • Is the system actively being attacked (Intrusion Detection System data) • Is the system actually vulnerable • Additional information leads to a more educated assessment of impactand likelihood of occurrence University of Adelaide

  30. Use Case – Internet Charges • AARNet users pay subscription costs • Most Australian Universities control using quota systems • Beginning 2014, the University of Adelaide removed the quota system University of Adelaide

  31. Use Case – Internet Charges • Potential Financial Risk • High volume of Internet usage • Internet usage is not cheap when you account for ~25k students! • We have a budget to stick to • What are we doing to control the cost? • Big Data!! University of Adelaide

  32. Use Case – Internet Charges University of Adelaide

  33. Use Case – Internet Charges • Constantly analysing Internet traffic • Comparing our traffic with a list of unmetered content • Applying technical controls to limit impact of known high cost, non University related activities University of Adelaide

  34. Use Case – Internet Charges University of Adelaide

  35. Putting Big Data to Work • In Summary: • Big Data systems are very powerful • Big Data principles can be applied to many needs, just ask the question • Big Data can help find needles in many haystacks • I hope you enjoyed my presentation! • Thank You University of Adelaide

More Related