1 / 49

Do lafzon ki hai DATA ki kahani...............

If ramayana can be reduced to one shlok….then can’t I Complete covering “SPATIAL Big DATA & SECURITY “ IN 15 MIN ?. Do lafzon ki hai DATA ki kahani. Ek hai ZERO ....duja hai ONE. Big Data. Security. WELCOME.

anjolie-bradford
Download Presentation

Do lafzon ki hai DATA ki kahani...............

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. If ramayana can be reduced to one shlok….then can’t I Complete covering “SPATIAL Big DATA & SECURITY “ IN 15 MIN ?

  2. Do lafzon ki hai DATA ki kahani............... Ek hai ZERO....duja hai ONE.....

  3. Big Data Security WELCOME

  4. SPATIAL BIG DATA has been with us for ages in various forms…but pretty invisible!!

  5. Ancient Egypt • River nile • Engineers used to try data analysis to predict crop yields • LOGS & LEVEL • 6695 Km long

  6. Challenges Perceptions Concepts Basic Intro …the 15 min route to THANK YOU slide

  7. An English professor wrote the words : “A Woman without her man is nothing” On the chalk board and asked his students to punctuate it correctly…. “A Woman,without her man,is nothing”. “A Woman: Without her, man is nothing”

  8. How we understand it ? DEFINING BIG SPATIAL DATA

  9. BIG SPATIAL DATA Spatial data sets exceeding capacity of current computing systems…… ….to manage, process or analyze the data with reasonable effort due to Volume, Velocity, Variety and Veracity DEFINING BIG SPATIAL DATA

  10. DATA is Exploding in Volume Velocity VARIETY While decreasing in Veracity

  11. BIG SPATIAL DATA Data at rest Finding actionable info in Massive volumes of both structured and unstructured geo data that is so large and complexthat it’s difficult to process with traditional database and software techniques…… Volume Data in Motion Velocity Data in Many forms VARIETY VERACITY Data in Doubt DEFINING BIG SPATIAL DATA

  12. Gigabyte (GB) - 1,024MB Terabyte (TB) - 1,024GB Petabyte (PB) - 1,024TB Exabyte (EB) - 1,024PB U.S. drone aircraft sent back 35 years worth of video footage in 2012 90% of data in the world was created in the last 2 years 3 EB of data is created every day

  13. growth of geospatial data is outpacing both software and services and is set to become a major contributor to the overall growth of the industry * Estimated revenue FY 2013

  14. The bad things in life open your eyes to the good things you weren’t paying attention to before SECURITY 100% security is a myth No one has said this!!! But it remains a fact Increasing attack surface

  15. The technology is ready…. But are we ready ?

  16. DISASTER RELIEF RETAIL UTILITIES FINANCIAL FRAUD DETECTION DISEASE SURVEILLANCE ECO-ROUTING TELECOMMUNICATIONS INSURANCE CALL CENTER REQUESTS 17

  17. The other of the side story

  18. Security challenges before we adopt spatial Big data

  19. Ek Distributed programming frameworks

  20. Utilise parallelism in computation & storage to process massive amounts of data Local Reduce Reduce Map Intermediate Combining Input file Shuffle Output File Mapper performs computation & outputs a key/value pairs Reducer combines the values belonging to each distinct key and outputs the result Distributed programming frameworks

  21. MAP REDUCE • Aggregate results from map phase • performs a summary operation • Splits the input data-set into independent chunks which are processed • in a completely parallel manner FRAMEWORK • Schedules and re-runs tasks • Splits the input • Moves map outputs to reduce inputs • Receive the results Distributed programming frameworks

  22. Read 1 TB 10 Machine’s One Machine 4 i/o Channels Each channel : 100 MB/s 4 i/o Channels Each channel : 100 MB/s 4.5 Min 45 Min So challenge is not storage but it is I/O speed

  23. Untrusted Mappers Securing the data in the presence of an untrusted mapper Distributed programming frameworks

  24. TWO NO SQL ISSUES

  25. First off : the name NoSQL is not “NEVER SQL” NoSQL is not “No To SQL “

  26. NoSQL Is simply Not Only SQL!!!!!

  27. MongoDB NoSQL DB are still evolving with respect to security infrastructure Redis

  28. Data storage & transaction logs

  29. STORAGE TIERS - Multi-tiered storage media • Necessitated by scalability, availability & the growth ie exponential • Different categories of data • Different types of storage Data storage & transaction logs

  30. Keeping track of data location Lower tier means reduced security, loose access controls Data storage & transaction logs

  31. INPUT VALIDATION/FILTERING

  32. How can we trust data ? Validating data when source of input data is not reliable? Filtering malicious data @ BYOD Input validation/filtering

  33. REAL TIME MONITORING

  34. Humongous number of alerts!!!! False positives Filtering malicious data @ BYOD REAL TIME MONITORING

  35. Secure communication

  36. End to end security ? Data encryption : attribute based encryption!!!to be made richer Secure communication

  37. Granular audits

  38. New attacks will keep happening…and to find out we need detailed audit logs Missed true positives Granular audits

  39. PRIVACY ISSUES

  40. EG : How a retailer was able to identify that a teenager was pregnant before her father knew In the world of big data,privacy invasion is a business model PRIVACY ISSUES

  41. And... We Also Have cloud with us?

  42. At 1.4% in 2011-12 Cloud was a very small percentage of the total IT spend

  43. Pace of Big Spatial Data adoption has been Sluggish

  44. There is unlikely to be a day soon in near future when we have a “FIND TERRORIST” BUTTON

  45. We have mostly been reactive till date…..

  46. USE KERBEROS FOR NODE AUTHENTICATION – (BUT WE KNOW IT’S A PAIN TO SET UP) STRINGENT POLICIES STANDARD TO INTRA COUNTRY LAWS SECURE COMMUNICATION EXHAUSTIVE LOGS STRINGENT POLICIES

  47. DISCLAIMER This presentation reflected the personal views and opinions in my individual capacity only. It does not represent the views and opinions of my organization or anyone else, and is not sponsored or endorsed by them in any way. This is an individual presentation.

More Related