Large Scale Computing • Introduction • Large scale distributed system • Caters to dynamic resource allocation and configuration of software stack • Challenges • Application/System configurations are “hardwired” • No standard method of capturing whole system configurations • multiple definitions of each value • multiple mechanisms for access • multiple notations (XML, INI files, SQL, RDF, CIM,...) • No separation of concerns between functionality and configuration

SmartFrog - Focus • How to describe the applications we want to run • How to realize these application descriptions, deployingthe applications securely and reliably • How to manage the applications through their lifecycles

deployment system Host A Host B SF Daemon sf language …… SF Daemon component model MyComponent.class SF Daemon SmartFrog System • You can use it to construct configuration-drivensystems • Realized as: • a language for describing system configurations • a component model defining the interfaces that components provide to implement the component lifecycle • a deployment system for reifying the descriptions and managing the running systems through their lifecycles

SmartFrog Language • A simple and expressive data description language • Supports the description of: • system configuration data • software components • code • locations • relationships

SmartFrog Language Design • Configuration data is organised as hierarchical attribute:value pairs • Supports inheritance (through prototyping) • Supports templates and parameterisation • Supports flexible variable linking and component binding

Language Example webServerTemplate extends { sfProcessHost “localhost”; port 80; useDB; } system extends { ws1 extends webServerTemplate { sfProcessHost “15.144.59.34”; } ws2 extends webServerTemplate { sfProcessHost “15.144.59.64”; port 8088; } db extends dbTemplate { userTable:rows 6; } } dbTemplate extends { userTable extends { columns 4; rows 3; } dataTable extends { columns 2; rows 5; } }

new values new attributes … system extends { … } system.sf Example : Prototyping dbTemplate extends { userTable extends { columns 4; rows 3; } dataTable extends { columns 2; rows 5; } } to build your own system you can use existing templates by: - importing them, and using them in your description - extending them by overwriting prior values - extending them by adding attributes and values #include “db.sf”; #include “webserver.sf”; your own description is now also a prototype system extends { ws1 extends webServerTemplate { } ws2 extends webServerTemplate { } db extends dbTemplate { } } db.sf sfProcessHost “15.144.59.34”; webServerTemplate extends { sfProcessHost “localhost”; port 80; useDB; } sfProcessHost “15.144.59.67”; port 8088; type “backup”; webserver.sf userTable:rows 6; version “Oracle8.2”;

Example : Linking & binding • Simple, containment naming tree based linking • “LAZY” means “resolve after deployment” system extends { commonPort 8088; ws1 extends webServerTemplate { sfProcessHost “15.144.59.34”; port ATTRIB commonPort; useDB LAZY ATTRIB db; } ws2 extends webServerTemplate { sfProcessHost “15.144.59.64”; port ATTRIB commonPort; } db extends dbTemplate { fooTables:rows 6; } } • Additional mechanisms are provided to link attributes (host-based, SLP) • New mechanisms may be added (JNDI, JINI, …)

SmartFrog Component Model • A SmartFrog component • has specified management interfaces • implements a pre-defined lifecycle • is written in Java (wrapping code in other languages) • (and has full access to a rich set of SmartFrog APIs) • You can use sf language to describe “SmartFrog components”

a component definition component class component location other configuration data Describing components myComponent extends { sfClass “com.hp….”; sfProcessHost “…”; sfProcessName “…”; val1 42; val2 “a string”; }

determines semantics of grouping; maps to the sfClass that implements this new types of collections may be added mySystem extends Compound { webServer extends … appServer extends … database extends … } Component Collections • systems are composed of applications that are composed of components • applications: collections of components that are deployed and managed as a group • it’s useful to support a variety of deployment semantics for collections • shared lifecycle (“compound”): start and end components together • sequential: when one component stops, the next starts, … • parallel: start components together, but end separately • … • SmartFrog defines (system-supplied) components that manage other components

Component Lifecycle parts of an application must be activated in phases, e.g.: • servers ready before clients • simple ordering • peers must both be ready to handle requests, before either does • interleaved initialisation this is only possible if components have an explicit, phased lifecycle: parse component lifecycle edit described deploy() instantiated terminated failed ready start() running terminate()

sf system configuration description Host A Host B Host C Host D Host E SF Daemon SF Daemon A distributed deployment environment SF Daemon SF Daemon SF Daemon SF Daemon discovery system (e.g. SLP) SF Daemon

SmartFrog Workflows • SmartFrog provides simple, lightweight workflow engine • You can use different notions of component composition, e.g. • compound (shared lifecycle) • sequence (one after the other) • parallel (start together, finish independently) • retry a component until it terminates successfully • try a component and if it fails try another • delay a component until some time, for some time • …

deployment node X SF Daemon components components Security: threat model 1. valid user impersonation web server not trusted! extra deployment info “virus” info “bad” user deploying on node X 2. modify configurations, spoof server… 4. added platform vulnerabilities network not trusted! “good” user deploying on node X 3. modify, spoof or snoop packets “good” deployment info

Security System implementation • RMI over SSL (JSSE) with mutual authentication • identity based on public/private RSA keys • signed jar files for dynamic loading of classes and SmartFrog descriptions • basic certificate authority (CA) support integrated with the build process (from openSSL) • full integration with Java 2 security model web server CA signed jar http RMI/SSL B SFTC A RMI/SSL RMI/SSL C signed jar

SmartFrog components : Building blocks • pre-defined components to aid in the building of complete systems • Emailer • SLA [Implements Service Location Protocol] • SSH & Net • Jetty • CDDLM • Scripting • Distributed Logger • Distributed Installer • ...

Developers Guide • Think of the configurable parts in the systems • Write SF description • Write/Reuse a Java class • Deploy the component (use scripts or ant) • Monitor the lifecycle using SF GUI tool • Terminate (if required) using tools

References • https://sourceforge.net/projects/smartfrog/ • www.smartfrog.org

Thank You

Backup Slides

extend core classes, provides default implementation of methods, etc implementation of lifecycle methods to be called by system replaces “main” Java Implementation of a component import com.hp.SmartFrog.Prim.*; import java.rmi.*; public class MyPrim extends PrimImpl implements Prim, … { /* any component specific declarations */ public MyPrim() throws RemoteException { } public void sfDeploy() throws Exception { super.sfDeploy(); /* any component specific initialization code */ } public void sfStart() throws Exception { super.sfStart(); /* any component specific start-up code */ } public void sfTerminateWith(TerminationRecored tr) { /* any component specific termination code */ super.sfTerminateWith(tr); } /* any component specific methods */ } any other code required

Host A sfDaemon compC compA compB sfConfig D S D D D T S S S T T T lifecycle: compound sfConfig extends Compound { compA extends Prim {..} compB extends Prim {..} compC extends Prim {..} } the compound is responsible for the lifecycle of its children: • whenit is deployed: - it creates the children - it calls their sfDeploy() methods

Host A sfDaemon compC compA compB sfConfig D S D D D T S S S T T T lifecycle: compound sfConfig extends Compound { compA extends Prim {..} compB extends Prim {..} compC extends Prim {..} } the compound is responsible for the lifecycle of its children: • whenit is deployed: - it creates the children - it calls their sfDeploy() methods • when it is started it calls its childrens sfStart() methods at this point every child stepping into its own start phase has its parent compound’s guarantee that its siblings have all been correctly deployed

Host A sfDaemon compB compA compC sfConfig D S D D D T S S S T T T lifecycle: compound termination sfConfig extends Compound { compA extends Prim {..} compB extends Prim {..} compC extends Prim {..} } • the compound may terminate for two reasons: one of its children may terminate, or it may be asked to terminate • in both cases the compound terminates all its remaining children, then terminates itself

Contents

Contents

Presentation Transcript

Contents

Contents

Contents

Contents

Contents

Contents

Contents

CONTENTS

Contents

Contents