1 / 31

SciDB: Lessons Learned

SciDB: Lessons Learned. XLDB2 Workshop September 29–30, 2008 Kian-Tat Lim. Outline. 10 lessons learned over the past year. Lesson 1. Fulfill your commitments. Mini-Workshop. Asilomar 2 days. Attendees. Lesson 2. Talk to lots of people. Science. Industry. Implementers. DB Research.

kana
Download Presentation

SciDB: Lessons Learned

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SciDB: Lessons Learned • XLDB2 Workshop • September 29–30, 2008 • Kian-Tat Lim

  2. Outline • 10 lessons learned over the past year

  3. Lesson 1 • Fulfill your commitments

  4. Mini-Workshop • Asilomar • 2 days

  5. Attendees

  6. Lesson 2 • Talk to lots of people

  7. Science

  8. Industry

  9. Implementers

  10. DB Research

  11. Lesson 3 • Learn the new buzzwords

  12. Terms • Data-Intensive Scalable Computing • Cloud Computing

  13. Lesson 4 • Similar requirements:the usual suspects

  14. Usual Needs • Scalable • Fast • Fault-tolerant • Cost-effective • Open Source

  15. Lesson 5 • Similar requirements:the unusual ones

  16. Unusual Needs • Order and position: • vectors and arrays

  17. Unusual Needs • Flexible schema • Uncertain data • Provenance • Versioning • Resource management

  18. Unusual Needs • Interfaces to imperative programming

  19. Lesson 6 • People are hungry

  20. Why A New DBMS? • Novel requirements • Large scale and complexity • Overlap of science and industry

  21. Lesson 7 • Work with smart people

  22. Brain Trust

  23. Lesson 8 • Re-think from the bottom up

  24. SciDB Storage • Array data model • Attribute-based storage • Compression • Versioning

  25. Lesson 9 • New operators are needed

  26. SciDB Operations • Spatial and temporal • Parallelized user-defined • Provenance

  27. Lesson 10 • Create an organization

  28. Nonprofit Foundation • SciDB.org • (Mozilla model)

  29. Initial Science Partners • LSST / SLAC • PNNL • LLNL • Fermilab • UCSB • NSF

  30. Industrial Partners • Several interested • Some with real $

  31. Summary • SciDB is becoming real

More Related