1 / 46

The Evolving Architecture

B2B Distributed Database. The Evolving Architecture. brave.taoy@alibaba-inc.com. About. Large Scale Data Domain. 陶勇 Platform Technology. Distributed Database Domain. @bravetao. Contents. 数据 存储与产品 架构 演变. Cobar 1.0.x / 1.1.x / 1.2.x. Oracle / MySQL. 软硬结合架构. SSD / FlashCache .

walda
Download Presentation

The Evolving Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. B2B Distributed Database The Evolving Architecture brave.taoy@alibaba-inc.com

  2. About • Large Scale Data Domain 陶勇 Platform Technology • Distributed Database Domain • @bravetao

  3. Contents 数据存储与产品架构演变 • Cobar 1.0.x / 1.1.x / 1.2.x • Oracle / MySQL 软硬结合架构 • SSD / FlashCache

  4. Evolving Database • 2012- • 2008-2011 OLTP&OLAP CobarCluster Cobar Oracle • 1999-2008 • 2011-2012

  5. Long-term Architecture

  6. Cobar Distributed MySQL Proxy 1.0.x Schema Based MySQL Server 1.1.x Distributed MySQL Server 1.2.x

  7. Issues • 业务：CCBU • 07年，中文站offer总数3kw • 08年，上亿 • 数据库：Oracle • 系统瓶颈 • 依赖高端硬件设备 • Standby切换故障 • Scale-Up

  8. Issues • Connection limits

  9. 基础架构

  10. 数据拆分 • select * from table where partition_key=?

  11. Hash • Virtual Partition Hash • Consistent hash

  12. 执行流程

  13. Stats online • CCBU offer • 3亿+/ 8 cobar/ 16*2 mysql • 高峰：3w tps • ICBU hermes • 8亿 / 3T+ disk / 3 cobar / 6*2 mysql • 20+ cobar cluster

  14. Limits • F5 Load-Balancer • mysql jdbc driver 5.0.4 • 不支持事务 • 不支持跨库的join、子查询、分页排序 • 分表需包含拆分字段 • 拆拆分函数值域元素数最大1024

  15. Cobar Distributed MySQL Proxy 1.0.x Schema Based MySQL Server 1.1.x Distributed MySQL Server 1.2.x

  16. Issues

  17. Issues • 单库事务 Local/Partition Transaction • 大结果集 Large ResultSet cause OutOfMemory • MySQL JDBC Driver 版本限制

  18. 基础架构

  19. 架构优化

  20. Solutions • 单库事务 • MySQL Protocol • Transaction Connection Session Binding • 大结果集 Large ResultSet cause OutOfMemory • Streaming Buffered ResultSet • MySQL JDBC Driver 版本限制 • Full MySQL Protocol Support

  21. 线程复用模型

  22. 性能对比 - Select TPS Latency

  23. 性能对比 - Update TPS Latency

  24. 性能对比 - Insert TPS Latency

  25. TODO • MySQL • 全局序列号 • 数据异步复制的延时的问题 • DDL操作锁表问题 • 大量分表分库带来的数据和表结构维护问题 • 读写分离 • Master-Slave复制延迟造成的不一致问题

  26. Cobar Distributed MySQL Proxy 1.0.x Schema Based MySQL Server 1.1.x Distributed MySQL Server 1.2.x

  27. 基础架构

  28. Data Flow

  29. Modules HA & Failover Druid Client SQL Rebuild Routing SQL to Partition SQL Parser SQL Router Cobar Merge Join Transaction SQL Merge & Join Local & Distributed Transaction Backbone Engine Core Framework

  30. SQL Parser • Goals: • More sql support • Re-build SQL • Split SQL • JOIN Hand Writing 1.1.x JavaCC 1.0.x ANTLR 1.1.x MySQL 5.5 syntax Full support

  31. SQL Parser SQL语句 Set/Show Delete Update Insert Replace Select 语法元素 Call OrderBy Limit Table References 表达式 ... Parser 基本元素 Identifier Keyword Variable Literal Punctuation

  32. Performance

  33. Merge

  34. Transaction • Realtime Commit • 2PC • 3PC • SAGA • Async-Callback Commit • Napoli

  35. Join • 单库Join • 支持 • 跨库Join • SQL Rebuild • Execution Plan • Merge Engine • 多表Join • <3张表 MySQL

  36. 架构多元化 • MySQL • 会产生海量数据的新应用 • Oracle 数据库访问压力已经或预期占比所在数据库达到 10% 以上，且满足分布式分拆的条件 • 与现有 Oracle 数据库中的数据表不需要进行关联查询（Join）的新独立应用集群 • Oracle • 不满足分布式拆分条件的应用 • 对事务和数据强一致性有苛刻要求的应用 • 必须使用 Hash Join 访问类型的应用 • 目前已经在 Oracle 中的应用，功能需求的增/改

  37. Oracle A-A • Semi-Realtime Sync

  38. 软硬结合架构 • Offer Cluster

  39. SSD Stats

  40. 软硬结合架构 • Offer SSD • Media WearoutIndicator 磨损程度 • 结果：offer集群基本上都在99-100，磨损的程度非常低 • Re-allocated Sector Count 坏块数量 • 结果：offer集群基本上都是0，坏块的数量很少 • Host Writes Count 累计写入量 • 结果：offer集群的SSD单块盘累计写入量大约是40T，有些盘甚至达到100T

  41. MySQL优化 单机多实例 • 存储技术飞速发展，IO不再是瓶颈 • MySQL对多核CPU利用率低 单机高性能 • RAID：开启BBWC(Battery Backed Write Cache) • RAID：开启为SSD优化的Fastpath • Fusionio • Flashcache：Linux Module • Semi-Sync

  42. On the way…

  43. And • What’s the difference • What I want is a robust, high performance virtual relational database that runs transparently over a cluster, nodes dropping in an out of service at will, read-write replication and data migration all done automatically. • Greg Linden

  44. Resource Center • Cobar • http://b2b-doc.alibaba-inc.com/display/RC/Cobar • Erosa • http://b2b-doc.alibaba-inc.com/display/RC/Erosa • Eromanga • http://b2b-doc.alibaba-inc.com/display/RC/Eromanga

  45. Q&A

  46. Thank You !

More Related