Starcounter

1. Starcounter NoSQL (NewSQL) database

2. Who am I? • Bojan Brankov • PMF, Novi Sad 2000 - [redacted] • Levi9 Global Sourcing Balcan 2005 - 2012 • Java, .NET • Drey doo 2012 - 2014 • .NET • UZROK (entrepreneurship) 2014 - • .NET, Data Visualisations, multimedia, freelance

3. What is Starcounter? • In-memory Application Platform (NoSQL*) • Built in REST style Web server • Supports SQL queries • Fully ACID* compliant • Native .NET object API (C#, VB.NET, Managed C++, etc.) • .NET Framework 4.5 *Atomicity, Consistency, Isolation, Durability - Guarantee of database transactions reliability

4. Why are we talking about it? • Let’s discuss modern databases!

5. Modern database Part 1 • Data processing is ubiquitous • Over 150 special-purpose DBMS solutions “a software system that uses a standard method of cataloging, retrieving, and running queries on data.” 1970 • Often expensive huge server farms • Compromises between extremes: 6 trade-offs

6. Modern database Part 2 • Trade-off #1: Human-Generated vs. Others • Building blocks upon buildingblocks of data • Exclude scientific data(Machine learning) • Focus on human-generated(Everyday use application,DBMS-friendly)

7. Modern database Part 3 • Trade-off #2: Structured vs. Unstructured • Unstructured - human-written part of the internet (sequence of letters, word of natural language) • Structuredness - defined meaning in a certain context (name value in CRM, business transactions) • Structured << Unstructured • eg. Amazon annual transactions 56Gb

8. Modern database Part 4 • Trade-off #3: Transactions vs. Analytics • Big Data phenomenon caused DBMS partitioning • OLTP - classic transactions, possible write conflicts • OLAP - static, read only, peak performance • Hybrid cutting-edge solutions (search engines, high-frequency trading) - “materialized aggregation”, requires high performance transac. DBMS

9. Modern database Part 5 • Trade-off #4: In-memory vs. Disk-based • OLTP DBMS crucial for many evolving businesses • In 2005 RAM price dropped -> In-memory is a must! • Today modern servers come with 128GB of RAM • Disk-based databases still important for OLAP • In-memory DBMS still use disks (persistency and reliability features)

10. Modern database Part 6 • Trade-off #5: Scale-in vs. Scale-out • Virtually - Consistent data vs. Inconsistent data • Scaling-out increases speed linearly with servers • System is in non-conflicted state all the time • Scaled-in systems keep transactional consistency

11. Modern database Part 7 • Trade-off #6: SQL vs. NoSQL • or “Classical” vs. “Modern” • From 2007 to 2012, the “classical” DBMS systems with rich SQL-based syntax shifted to NoSQL DBMS • NoSQL - lightweight, cheap, fast but non-consistent • Now DBMS users demand rich syntax, operations and basic OLAP • NoSQL + SQL = NewSQL

12. Modern database - Recap • Human-generated data • Structured (leave science to sciencers) • Transactional (OLTP with OLAP support) • In-memory with disc support • Scale-In and consistent • Modern, NewSQL

13. NoSQL Distributed data, many more transactions per second No consistency, no simplicity NewSQL Millions of transactions per second on a single machine ACID guaranteed consistency Alternatives • OldSQL • Cca 10.000 transactions / seconds on a single machine • ACID guaranteed consistency

14. Application of Starcounter? • For modern applications: web, mobile services and massive real time software (games, advertising…) • Optimised for stellar concurrency and low latency • Tight integration with HTTP, REST, JSON • Simplicity to create view-models with zero glue code (VMMV, VMC) • Uses integration with WPF and XAML for flexible desktop applications

15. How does it work? • VMDBMS • ?? => PROFIT • Integration between application runtime VM and the data management system (just like CLR or Java VM) • Data in one single place in RAM all the time • Direct access: No temporary copies, no serialization and deserialization • Uses disks to secure the transaction log and a checkpointed database image

16. Why is that good? • Fundamental criteria: Moving both the model and controller logic inside the db => less strain on db • Moving memory around is a bottleneck! • Clock cycles for business logic << Clock cycles for communicating and moving data

17. Starcounter database Part 1 • Do not think relational! • Classes are tables and instances are rows • Database objects live in the database from the beginning (new operator) using Starcounter; [Database] public class Persistent { public string Name; }

18. Starcounter database Part 2 • Public fields, public auto-created properties and public properties getting and setting private fields are columns using Starcounter; [Database] public class Person { public string FirstName; public string LastName { get; set; } public string FullName { get { return FirstName + " " + LastName; } } }

19. Starcounter database Part 3 • Offers relational, graph, object oriented and document access, all rolled into one. • Relations are established through object references rather than explicit keys [Database] public class Quote { public Person Who; private string _Text; public string Text { get { return _Text; } set { _Text = value; } } }

20. Starcounter database Part 4 • The Database attribute is inherited from base to subclasses • Any class that directly or indirectly inherits a class with the Database attribute implicitly becomes a database class [Database] public class Vehicle { public int YearOfManufacturing; } public class Car : Vehicle { public int Wheels; }

21. Starcounter database Part 5 • Every Database class is available to standard SQL expressions in C#, no ORM • You can use queries with path expressions to increase performance and replace complex joins [Database] public class Person{ public String Name; public IEnumerable Quotes(){ return SQL(”SELECT q FROM Quotes q WHERE q.Who=?”, this); } ..person.City.Name =?”, this);

22. Starcounter database Part 6 • You can exclude fields and auto-created properties from becoming columns by using the Transient custom attribute. • It will remain a regular .NET field/property -> value stored on the CLR heap and garbage collected • Row is removed using Delete() method.

23. Starcounter database Part 7 • Starcounter supports: • Transaction scopes • Long running transactions • Nested transactions • Parallel transactions

24. SQL CREATE TABLE INSERT UPDATE SELECT DELETE Starcounter database - Recap • STARCOUNTER class definition new() operator = operator SQL(String) Delete() method

25. Is that all? • Integrated Communication Server • Extremely fast gateway • Handles ≈1M HTTP requests per second • Serves REST, MVVM, MVC code, business logic and database transactions • Can communicate using REST and JSON, through mirrored MVVM, perfect for Web Component frameworks (AngularJS)

26. What about the performances? 500k ACID read-only transactions per second per core Scales with number of cores almost linearly 100k ACID read-write transactions per second per core, to maximum of 300k on 4 cores 1.5M clients 0.5M accounts 5% transfers, 95% reads XEON E5 (10 cores), 128GB RAM and SSD

27. How much does it cost? • Cost projection for mid size ERP system • MySQL solutions still require powerful hardware or even server farm • SC runned complete system on a single lap top

28. Any other questions? • I thought so! • bojan.brankov@gmail.com • bb@uzrok.com