1 / 14

Optimization with Big Data

=. Extreme* Mountain Climbing. Optimization with Big Data. * in a billion dimensional space on a foggy day. Peter Richtarik School of Mathematics. BIG DATA. BIG Volume BIG Velocity BIG Variety. BIG Volume BIG Velocity BIG Variety.

afra
Download Presentation

Optimization with Big Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. = Extreme* Mountain Climbing Optimization with Big Data * in a billion dimensional space on a foggy day Peter Richtarik School of Mathematics

  2. BIG DATA BIG Volume BIG Velocity BIG Variety BIG Volume BIG Velocity BIG Variety • digital images & videos • transaction records • government records • health records • defence • internet activity (social media, wikipedia, ...) • scientific measurements (physics, climate models, ...) Sources

  3. Arup (Truss Topology Design) Western General Hospital (Creutzfeldt-Jakob Disease) Ministry of Defence dstl lab (Algorithms for Data Simplicity) Royal Observatory (Optimal Planet Growth)

  4. GOD’S Algorithm = Teleportation

  5. If you are not a God... x2 x3 x0 x1

  6. Optimization as Lock Breaking A number representing the “quality” of a combination x =(x1, x2, x3, x4) F(x) = F(x1, x2, x3, x4) Setup: Combination maximizing F opens the lock Optimization Problem: Find combination maximizing F

  7. Optimization Algorithm

  8. How to Open a Lock with Billion Interconnected Dials? # variables/dials = n = 109 x1 x2 Assumption: F = F1 + F2 + ... + Fn ----------------------- Fjdepends on the neighbours of xjonly x4 xn x3 Example: F1 depends on x1,x2,x3 and x4 F2 depends on x1 andx2, ... F : RnR

  9. Optimization Methods Computing Architectures Effectivity Efficiency Scalability Parallelism Distribution Asynchronicity Randomization • Multicore CPUs • GP GPU accelerators • Clusters / Clouds

  10. Optimization Methods for Big Data • Randomized Coordinate Descent • P. R. and M. Takac: Parallel coordinate descent methods for big data optimization, ArXiv:1212.0873 [can solve a problem with 1 billion variables in 2 hours using 24 processors] • Stochastic (Sub) Gradient Descent • P. R. and M. Takac: Randomized lock-free methods for minimizing partially separable convex functions [can be applied to optimize an unknown function] • Both of the above M. Takac, A. Bijral, P. R. and N. Srebro: Mini-batch primal and dual methods for SVMs, ArXiv:1302.xxxx

  11. Theory vs Reality

  12. Parallel Coordinate Descent holy grail settle for this start

  13. TOOLS Probability HPC Matrix Theory Machine Learning

More Related