1 / 17

The New cmw-rda motivation, design and technical choices BE/co TC 28th june 2012, geneva

The New cmw-rda motivation, design and technical choices BE/co TC 28th june 2012, geneva. A. Dworak BE-CO-IN, CERN. Agenda. Sum up of the previous report Middleware prototyping Transport Serialization Design concepts Keep proven solutions Modularity Data Model Transport Client

teague
Download Presentation

The New cmw-rda motivation, design and technical choices BE/co TC 28th june 2012, geneva

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The New cmw-rdamotivation, design and technical choicesBE/co TC28th june 2012, geneva • A. Dworak • BE-CO-IN, CERN

  2. Agenda • Sum up of the previous report • Middleware prototyping • Transport • Serialization • Design concepts • Keep proven solutions • Modularity • Data Model • Transport • Client • Server • Work progress Andrzej Dworak, The new CMW-RDA

  3. Previous reports • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC Andrzej Dworak, The new CMW-RDA

  4. Previous reports • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC • Requirements gathering, user interviews, internal reviews • Shortcomings of the system, missing functionality Andrzej Dworak, The new CMW-RDA

  5. Previous reports Major problems with CORBA and current code base using it: CORBA: • C++ and Java implementations differ • Shrinking community • Lack of new releases and bug fixes • Expensive on resources • Complex error prone API • Problems with asynchronous and non-blocking communication • No support for pub/sub • Missing support for versioning or flexible interfaces (hard to upgrade) Current design: • Monolithic (hard to maintain, extend and test) • Not scalable (poor parallelization) • Not optimal design makes it unnecessarily heavy and slow • Lack of QoS, message priorities • Missing server-side management of client connections • After all the years and fixes it is BIG • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC =========================================================== From users’ perspective • Service instability • Client may influence Server • Fails when overloaded • Does not scale • Impossible to optimize the usage • Unnecessarily limited (e.g. connection #), slow and heavy. • Needs workarounds (e.g. Proxy) • Unfriendly API • Lack of full diagnostics Andrzej Dworak, The new CMW-RDA

  6. Previous reports • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC Andrzej Dworak, The new CMW-RDA

  7. Previous reports • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC Andrzej Dworak, The new CMW-RDA

  8. Previous reports • “Middleware Trends and Market Leaders 2011” • 14/10/2011, ICALEPCS 2011 • 20/10/2011, BE-CO-IN TC Ice YAMI4 Presented so far Andrzej Dworak, The new CMW-RDA

  9. Middleware prototyping • More tests • Scalability • Correctness • Error handling • Exceptional situations • Prototyping of the system parts Ice YAMI4 USE Andrzej Dworak, The new CMW-RDA

  10. Serialization Prototyping • Necessary to transport data between different architectures • Currently self-describingserialization  • Based on CORBA (on top of IDL)  • Stay with self-describing serialization, chose better library • Used so far in RDA  • No need to manage IDLs  • Easier to upgrade or change the protocol  • Need to encode type description  • Slightly slower • Slightly bigger output • Review of +10 serialization libraries • Testing and prototyping with: • Google Protocol Buffers • MessagePack • Apache Avro • Compared with CORBA MessagePack USE Andrzej Dworak, The new CMW-RDA

  11. More details at: • Middleware • General review and testshttp://wikis/display/MW/CMW+Review • Presentation of the review and test resultshttp://wikis/download/attachments/49218519/FRBHMULT05_talk.pdf?version=1&modificationDate=1328608532000 • More details on the 1st step also in A. Dworak “Middleware trends and market leaders 2011” • More details on the 2nd step also in A. Dworak “The new CERN Controls Middleware” • Serialization • Presentation with review details and test resultshttp://wikis/download/attachments/49218519/Self-describing+Serialization.pdf?version=1&modificationDate=1328608647000 Andrzej Dworak, The new CMW-RDA

  12. Design concepts – proven concepts Keep the proven concepts: • Device/property model • Communication through get, set and subscribe/publish calls • Properties represented as dynamic Data objects • Self-describing serialization • Authorize and authenticate user access Andrzej Dworak, The new CMW-RDA

  13. Design concepts – modularity • Less coupling between system parts • Easier to test and maintain • dependency injection • mocking • Will allow to exchange parts of the system (e.g. transport = ZeroMQ or serialization) • Will help during the migration process (old RDA to new RDA) Andrzej Dworak, The new CMW-RDA

  14. Design concepts – transport • Transport executed asynchronously and independently of the user code execution • Separate channels for different types of communication • Administrative • Get and Set • Subscriptions • Optimization • Internal and through user API • Intelligent buffering on the transport level • Immutable data allows better and safer parallelization • 0-copy (when sending, for receiving not 0 but also optimized) • Extended diagnostics Andrzej Dworak, The new CMW-RDA

  15. Design concepts – data model • Property as a set of named fields with dynamic types • Current type set extended to support: • Fixed length integer types (C++ int64_t, int32_t, int16_t, int8_t) • Multidimensional arrays • Nested Data objects • Management of data ownership • Allows 0-copy for arrays and nested data objects • Context class with direct support for • Filters • RBAC tokens Andrzej Dworak, The new CMW-RDA

  16. Design concepts – client • Can get, set and monitor values of properties • Uses predefined policies which will handle communication events and errors • Connect/reconnect policy • Subscribe policy • Underlying communication is always non-blocking and asynchronous • For convenience API allows to perform the calls as • Asynchronous • Synchronous • Buffered asynchronous • Buffered synchronous • RBAC role may guarantee a client a higher priority • Client may be managed on-line (similarly to servers and cmw-admin) Andrzej Dworak, The new CMW-RDA

  17. Design concepts – server • Asynchronous execution of implemented get, set and (un)subscribe handlers • Non blocking data publishing • May set delivery policy (normal/background) • Message priority and QoS • Extended management (also on-line via CMW Admin) • Turn on/off get, set, publishing • (Un)block client / low-priority clients • Disconnect client / low-priority clients • Send commands to clients • Ping/hello • Disconnect request • “Failover to” request Andrzej Dworak, The new CMW-RDA

More Related