1 / 10

Convergence of HPC, Databases, and Analytics

Convergence of HPC, Databases, and Analytics. Tirthankar Lahiri Senior Director, Oracle TimesTen In-Memory Database. Enabling The Real-Time World. Financial Services. Social Media. Real-Time Analytics . Telecommunications. eCommerce. Next Generation DBMSs. High-Speed Networks.

yan
Download Presentation

Convergence of HPC, Databases, and Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Convergence of HPC, Databases, and Analytics TirthankarLahiri Senior Director, Oracle TimesTen In-Memory Database

  2. Enabling The Real-Time World Financial Services Social Media Real-Time Analytics Telecommunications eCommerce Next Generation DBMSs High-Speed Networks Multicore 64-bit Processors SSD/Persistent Memory Massive DRAM capacity

  3. Analytics Requirements • Intuitive interfaces • Instantaneous response time • Real-time reporting • Structured / Unstructured data • Extreme concurrency

  4. Hardware Trends • Processor throughput • Cores per socket: 5x in last 5 years • Clock frequency: Flat in last 2 years • Memory capacity 2x every 2 years • High-speed networking: Infiniband, RapidIO, 10GigE, etc. • Persistent memory technologies • Flash replacing disks for online storage • PC-RAM, MRAM : extension of RAM, or superfast disks for hot data

  5. Database Design Trends • Emerging Industry Trend • Exadata, Exalytics, Big-Data Appliance • Balanced Compute/Capacity/Power • Co-developed components • Built-in scale up, scale out • Built-in interoperability • Pushdown functionality into hardware • Exadata Smart Storage • Integrated Appliances

  6. Example: Oracle ExalyticsIn-Memory Machine TimesTen for Exalytics 1 TB RAM 40 Processing Cores High Speed Networking Memory Optimized Essbase Adaptive In-Memory Tools Optimized Oracle Business Intelligence Foundation Suite In-Memory Analytics Software In-Memory Analytics Hardware

  7. Database Design Trends • CPU core performance is flat • Parallelize (don’t paralyze) • Coarse-grained parallelism • Exploit workload parallelism • Parallelize query execution • Parallelize maintenance operations (backup/restore) • Exploit high speed communication primitives (e.g. Infiniband RDS) • Fine-grained parallelism • Vector execution • Multi-threading of low-level primitives • Parallelism Everywhere

  8. Database Design Trends • Use all the available tools: • In-memory storage when applicable • Column storage for sequential accesses • Row storage for random accesses • Advanced compression techniques • More bang for your storage buck • !!Beware of NUMA !! • NUMA locality awareness • Lock free or well partitioned data structures • Avoid global updates to shared memory • Storage Management

  9. Database Design Trends Where’s that %^@#$ plan?? • Enhance for analytics • Analytic functions, data mining models, graph models, etc • Cache-friendly access methods • Sequential scans better than random access • Cost modeling for modern hardware • Disk IOs are no longer the dominant cost • Cache Misses / Memory References • CPU cycles, execution time • Query Optimization

More Related