1 / 30

CS345: Advanced Databases

CS345: Advanced Databases. Chris Ré. What this course is. Database fundamentals: Theory Old Crusty, Good SQL stuff No/New/Not-Yet SQL New stuff: Knowledge bases & Inference Databases is a strange and beautiful area: Theory, Algorithms, Systems, & Applications

keith
Download Presentation

CS345: Advanced Databases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS345: Advanced Databases Chris Ré

  2. What this course is Database fundamentals: • Theory • Old Crusty, Good SQL stuff • No/New/Not-Yet SQL New stuff: Knowledge bases & Inference Databases is a strange and beautiful area: Theory, Algorithms, Systems, & Applications It’s a bit scattered, and I love it.

  3. A Brief, BiasedDatabase History

  4. Three Turing Award Winners Charles Bachmann Edgar Codd Jim Gray Seminal contributions made in Industry

  5. The Birth of the Relational Model(1971) database: a handful of relations (tables) with fixed schema. WorksIn(Employee,Dept) Query with small # of operations: Selection (filter), Projection, Join, Union. Basically, an operational finite model theory.

  6. Data and Query Model R(A,B) = { (a1,b2),…,(an,bn) } Data S(B,C,D) = { (b’1,c1,d1),…,(b’m,cm,dm) } PA(R) ={ a : exists b. (a,b) in R } Projection sF(R) ={ (a,b) : F( (a,b) ) for t in R } Selection F : D(R) -> {True, False} Join(R,S) = { (a,b,c,d) : (a,b) in R & (b,c,d) in S} Join

  7. Key idea of the Relational Model Declarative User says what they want--- not how to get it.

  8. Key question: Can one implement the Relational Model efficiently?

  9. System R Pat Selinger In,1974 System R shows possible to get good performance. 1st Implementation of SQL. IBM didn’t Push it, worried about IMS cannibalization, but…

  10. Others Come on to the Scene… Larry Ellison hears about IBM’s Research prototype and founds a company….

  11. Fast Forward to Today Relational model is dominate model of data.

  12. Takeaways about Database Research Started with mathematical elegance and with close ties to industry. Improve runtime performance as a proxy to increase programmer productivity.

  13. The Big Ideas

  14. Independence Declarative languages can improve productivity • Different team members work independently • Backend, Storage, UI, BI, Etc. • Transactional model. • Challenge: Support efficient concurrent access?

  15. Performance Parallel programming is hard; SQL is most popular parallel programming language. • How do you deal with asymmetry of memory hierarchy (Disk/MM/Cache)? • How do you structure parallel optimization? • Concurrency?

  16. Manageability Systems live over time, and the system should automate many routine tasks. • Maintain derived data products (views) • Self-monitoring systems (autonomic)

  17. Course Topics

  18. A user says what they want—not how to get it.

  19. Topic 1: QP Fundamentals Query Processing Fundamentals • Empirical Join evaluation from 70s! • System R: The Archetype (Cardinalityw) • Formal Query Languages • Acyclic Query Evaluation (Structure) • Worst-case Optimal Join Algorithms (S + C) This will be the most formal part of the course.

  20. Analyzing your data before it was big (when it was just very large…)

  21. Topic 2: OLAP-Style Analytics Building new and old data systems: • Theory of Materialized View • Gamma (Parallel DBs) • MapReduce & the Rise of NoSQL (2000s) • NewSQL & Optimizing Joins on MR (theory) • Fagin’s Algorithm (theory) • Statistical Analytic Systems

  22. My biased view of the future…

  23. Topic 3: Next-Generation Systems • Information Extraction • Probabilistic Query Evaluation (Theory) • Scalable Inference • Knowledge Bases

  24. Transactions.

  25. Topic 4: OLTP Style Transactional Systems • The rise of Key-Value Stores • The case for determinism • CALM & CAPs • The Return of Main Memory DBs. • Spanner, F1, and Data Centers

  26. Course Logistics

  27. Grading • Course Project (More next) • Do something interesting with data. • Teams OK • Form teams soon and email me by Jan 12. • Midterm Exam

  28. Projects in each topic • Knowledgebase Construction • Pick a domain and build a KBC system for it with DeepDive • Join Algorithms • Certificate versions (see me) • MapReduce? GraphLab? Spark? 3. Analytics Systems 4. Transactional Systems. You are free to choose other projects

  29. Datasets • Snapshot of the web marked up with NLP tools and structured data (KBP and KBA challenges) • 500k+ docs used by PaleoBiologists and structured data. • We can mark up even more stuff. • Benchmark ML, graphs if you want to work on analytics or join evaluation.

  30. Wednesday • Wednesday we begin the ancient art of join evaluation. All who pass this way must pass through this ancient topic! • Read: Shapiro. • not too carefully, we’ll go through details

More Related