140 likes | 261 Views
OOI CyberInfrastructure: Data Distribution Network architectural overview. January 2009 Claudiu Farcas OOI CI Architecture & Design Team UCSD/Calit2. Outline. Prototype Goals Logical Architecture Deployment Architecture Cloud Deployment Strategy Documentation. Functional Goals.
E N D
OOI CyberInfrastructure:Data Distribution Networkarchitectural overview January 2009 Claudiu Farcas OOI CI Architecture & Design Team UCSD/Calit2
Outline Prototype Goals Logical Architecture Deployment Architecture Cloud Deployment Strategy Documentation
Functional Goals Demonstrate deployment of an existing data distribution application in the cloud high availability and elastic scalability to demand technical feasibility to the different OOI stakeholder communities, from ocean scientists to cyber-infrastructure experts Investigate distributed cloud application provisioning mechanisms cloud instance contextualization mechanisms load balancing mechanisms mechanisms for load reduction on data providers cloud deployment environments, in particular Amazon EC2
DDN Messaging • Messages by component (application level)
DDN Messaging • Provisioner (contextulization) Message Routing Keys rabbitmq.dnsrabbitmq.runningrabbitmq.runscriptrabbitmq.statusregapp.config_templregapp.dnsregapp.runningregapp.runscriptregapp.statusregapp_db_agent.config_templregapp_db_agent.dnsregapp_db_agent.runningregapp_db_agent.runscriptregapp_db_agent.status dispatch.runningdispatch.runscriptdispatch.statuserddap_crawl.dnserddap_crawl.runningerddap_crawl.runscripterddap_crawl.statuserddap_util.dnserddap_util.runningerddap_util.runscripterddap_util.statusloadbalancer.dnsloadbalancer.runningloadbalancer.runscriptloadbalancer.statusmemcached.dnsmemcached.runningmemcached.runscriptmemcached.status mysqlcluster_api.dnsmysqlcluster_api.runningmysqlcluster_api.runscriptmysqlcluster_api.statusmysqlcluster_data.dnsmysqlcluster_data.runningmysqlcluster_data.runscriptmysqlcluster_data.statusmysqlcluster_mgmd.dnsmysqlcluster_mgmd.runningmysqlcluster_mgmd.runscriptmysqlcluster_mgmd.status
Summary of Technologies ERDDAP data transformation and distribution middleware THREDDS Dataset catalog server Memcache share memory state across distributed application instances HAproxy load-balancer Django (using python) As web user interface framework providing integrated access to MySQL database (cluster) MySQL Cluster highly reliable, scalable application database Amazon Elastic Cloud (EC2), Simple Storage Service (S3) Cloud execution environment
Documentation • DDN (wiki structure) • Introduction (what, why/benefits) • User Guide • Data Provider’s Guide (add/manage datasets, catalogs, servers) • Data Consumer’s Guide (subscribe/manage subscriptions, ERDDAP) • Developer Guide • Installation how-to • Logical Architecture • Deployment Architecture • Administration/maintenance how-to • Context sensitive help • “?” after each option/button provides explanatory text bubble