1 / 23

First Indico Workshop

First Indico Workshop. Database Technology. 27-29 May 2013 CERN. Pedro Ferreira. The Future. Let’s start from the end. A more «social» Indico User-oriented approach Personal home page, user-tailored information More data in less time Need for an adequate DB.

jerome
Download Presentation

First Indico Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. First Indico Workshop Database Technology 27-29 May 2013 CERN Pedro Ferreira

  2. The Future Let’sstartfrom the end • A more «social» Indico • User-orientedapproach • Personal home page, user-tailored information • More data in less time • Need for an adequate DB

  3. Avoidingdisaster

  4. ZODB Why not? • It’sobject-oriented • It’s been around for a long time • We know how to use it • We are alreadyusingit for some of thesethings

  5. ZODB An object-oriented database Index Transactions O1 … O1 O2 O2 time O3 O3 O4 O1 O4

  6. ZODB The good parts • It’s simple • No need for ORM or mappinglayers • Tightlyintegratedwith Python • It is ACID – thingswork as expected

  7. ZODB The bad parts • No server-sidequeries • No built-inindexing – application level • Data recoveryis slow (latency, unpickling, setting object state) • No way to fetch more than an objectat once (pre-load)

  8. ZODB Yes, there’s more… • Has to bepackedregularly • No caching on the server side (besides OS cache) • FileStorageReplicationis not Open Source - money • RelStoragecouldwork, but requires migration • It’s a niche product

  9. Don’t Panic! We’reworking on it!

  10. NG DB Project The NextGeneration DB for Indico http://indico-software.org/wiki/Dev/FutureDB • Aims to find DB infrastructure thatcan support growth • 6 month initial phase (techpreview/boilerplate – end 2013) • Tech Survey, Evaluation, Prototyping…

  11. NG DB Project The criteria • Availability (OSS) • Scalability/Replication • Ease of use/development • Transactions/Consistency • Community/Momentum • Costs

  12. The Contestants The NoSQLcrowd • Key-value stores • Riak, Redis, Voldemort • Document-oriented • MongoDB, CouchDB • Column-oriented • Cassandra, Hbase • Neo4J

  13. Key-Value Just a mapping structure user1 Pedro Ferreira user2 Alberto Resco user3 {name: {first:“Jose Benito”, last: “Gonzalez”}

  14. Column-oriented

  15. Document-oriented Closer to the OO philosophy user1 name Pedro Ferreira city Geneva user2 name Alberto Resco hobbies Running Cycling

  16. The Contestants The usual (relational) suspects • MySQL • MariaDB • Drizzle • Percona • PostgreSQL

  17. Relational vs. NoSQL A verycoarsecomparison

  18. Relational vs. NoSQL The problems

  19. Typicalquery «All events in a user’sfavouritecategories» • Relational • Simple joinbetweentwo tables • MongoDB • Eitherreplicate data or use DB refs (slow!)

  20. HybridApproach The best of twoworlds? • ZODB - excellent storage for business objects • SELECT style queries… • ZODB as primarystorage? • Alreadykind of doingthat(Redis) • Need for transition period

  21. HybridApproach No suchthing as a free lunch… • Keeping data consistent • Multiple DB calls per request • Yetanotherthing to install

  22. Conclusion • Researchproject • Still a lot of ground to cover • Hard to evaluate the hype • Hybridcouldbe a good option

  23. Pedro Ferreira http://github.com/pferreir @pferreir

More Related