1 / 0

Management and contextualization of scientific virtual appliances

Germán Moltó Associate Professor at the Universidad Politécnica de Valencia ( Spain ) gmolto@dsic.upv.es. Management and contextualization of scientific virtual appliances. For the Cloud!. OUTLINE of the talk. Outline Introduction and Overview of the GRyCAP

damisi
Download Presentation

Management and contextualization of scientific virtual appliances

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Germán Moltó AssociateProfessor at the Universidad Politécnica de Valencia (Spain) gmolto@dsic.upv.es

    Management and contextualization of scientific virtual appliances

    Forthe Cloud!
  2. OUTLINE of thetalk Outline Introduction and Overview of theGRyCAP Scientific Cloud Computing Contextualization: Scientific Virtual Appliances Virtual Appliances Repositories and Catalogs ScientificApplications Conclusions and Future Challenges
  3. Thegrycap in a slide Grid and High Performance Computing Group Group of the Area of Information Technologies and Computational Science Created on 1986 by Vicente Hernández and Composed by 28 Researchers (http://www.grycap.upv.es). Adoption of Parallel and Distributed Computing Technologies for Improving the Performance of Scientific Applications. Evolution to Grid and Cloud Technologies E-Science: Support for Science Research through the Collaborative Use of Distributed Resources. e-Government EngineeringSimulation Proteomics BiomedicalComputation MedicalImaging Photonics e-Science e-Infrastructure Middleware Grid Technologies Cloud Technologies Parallel Computing Distributed Computing NumericalComputation
  4. Scientificapplications ScientificApplicationstypicallyrequire: Largecomputationalpower. Itsrequirementsmightexceedtheresources of a single machine Processinglargeamount of data. Combination of SeveralTechniques High Performance Computing Using multiple processors to solve a problem. Grid Computing Enablethecollaborativeusage of resourcesfrommultipleorganizationstofacetheefficientexecution of large-dimensionproblems.
  5. Gridcomputing Pros and Cons Grid Computing has beensuccessfullyemployed in manyscientificareas, althoughsamecaveatsexist.
  6. Cloud computing ForScientific Computing Cloud Computing advantagesoverGrid Computing: Itallowstheresourceconsumersto configure theirspecificExecutionEnvironments. A controlledenviromentiscriticaltoguaranteethesuccessfulexecution of scientificapplications. Dynamicscaling of infrastructuresforresourceproviders. Virtual Machines can bedeployedusingworkload-awarestrategies. Fast and easyaccessto a largeamount of resources. No needforscientificcomission’sapproval, just use yourCreditCard. Reduced energy consumption (Green Computing) Machines are onlyprovisionedwhenthey are requested. Virtualizationleverages server consolidation.
  7. Thepoint of view of thescientist/Engineer Scientists and Engineersshouldnotbeconcernedwithimplementationdetails of technology. I don’tcareabouttechnology, I justwant my appstorunthefastestpossible X.509 Proxies CAs Grid VOs SE gLite Globus LFN SURL … Hypervisor Configuration Cloud Focusonabstractingthedetails of applicationportingtothe Cloud. Deployment Monitoring APIs …
  8. SCIENTIFIC Cloud computing Scientific Cloud Computing focusesontheexecution of scientificapplicationson a (typically) IaaScloud. It requires the management and provision of Scientific Virtual Appliances from a Virtual Machine Manager. Google Docs Office Live … MS Azure … Google AppEngine Eucalyptus OpenNebula … Amazon EC2 Source: www.saasblogs.com
  9. Virtual machine managers VMMsprovidethebasictoolstobuildanIaaS Cloud Differenttools in thecloud arena for VM management.
  10. Currentlimitation of cloudcomputingtools Virtual Machine Managers focus on supporting the life cycle of VMs. Scientific Cloud Computing also requires: (semi-)Automatedcontextualization of Virtual Machines forscientificapplications Scientific Virtual Appliances (SVA). ReusingSVAsfromoneexperimenttoanother, alsotoenhanceSVAssharingamongdifferentresearchers. Wefocuson: Applicationcontextualization (From a VM to a SVA). Repositories and catalogs of SVAs.
  11. Virtual appliances A Virtual Appliance (VA) consists of a Virtual Machine speciallyconfiguredforanApplication. Application App Data Application ComputationalLibraries ApplicationRequirements Middlewares OperatingSystem PersistenceLayer Services Virtual Appliance OperatingSystem Scientific Virtual Appliance
  12. Contextualizingscientific virtual appliances FromVMstoproductionSVAs … Contextualizationmeanscreatingtheappropriate SW/HW environmentforthesuccessfulexecution of anapplication. Virtual Machines needtobecontextualized (IP, DNS, etc.). SupporttypicallyprovidedbytheVMMs. Applications need to be contextualized. Deployed, configured, built, executed. Virtual Machine Scientific Virtual Appliance Contextualization Plain OS ScientificApplicationrunning
  13. Software configurationtools Many machine configurationtools. Focusonautomatingthe: Machine configuration DNS, Config files, etc. Installation of commonlyusedpackages: Web Servers, Application Servers, etc. Client-Servicetools.
  14. Deployingscientificapplications Manyscientificapplicationsfollowthesamepatterns …
  15. Automatingapplicationcontextualization (I) ForScientificApplications We are workingon software for (scientific) applicationcontextualization. Goal: Software inoculation and configurationintothe VM withminimumuserintervention. Automation vs SSH-based Manual Installation InstallPackages App AppDescription (XML) CNTXTLZR Configure Contextualization Plan Build Software Dependences Deploy / Run
  16. Automatingapplicationcontextualization (II) Developed a proof-of-concept toolforscientificapplicationcontextualization. Python-basedtoensuregoodportability. Plugin-basedto describe thedeployment of software packages. XML language Thetool, application and requirements are stagedintothe VM at boot time viathe VMM capabilities (OpenNebula). VM isturnedinto a SVA byapplicationcontextualization at boot time.
  17. Toward virtual machine cataloguing Thereexist VM catalogsoutthere: VMWareMarketplace ScienceCloudsMarketplace BUT… For human consumption, no APIs, unstructured metadata, etc. The VM Catalogincludes: VM Metadata (OS, Software Environment, etc.) OVF (Open VirtualizationFormat), XML-based. Links to VM repositories (either local orremote). MatchmakingalgorithmstoretrievethemostappropriateVMsaccordingtouserrequirements (hard vs soft).
  18. Management of scientific virtual appliances Theuser/adminprovides a description of the VM in OVF format. FTP server instances are createdondemandwithdynamic and temporarycredentialsfor VM upload. Client-SideLibrariestoeasetheinteractionwiththecatalog.
  19. Virtual Machine Repository The VM Repositoryincludes: Storage of VMs Data Access Mechanisms HTTP and FTP. GridFTPwouldprovideenhanced X.509-based security. Virtual Machines considered: GoldenVMs Example: JeOS-based, Lowfootprint (Ubuntu JeOS , 380 Mbytes HD) Pre-ContextualizedVMs Reusethework done. No needto re-deploy software forever. Example: A Globus Tookit 4-based VM that can be reused for the deployment of different Grid Services.
  20. Thebigpicture Catalogs, Repositories and Contextualization
  21. Remotecontrollinganapplication Howto control theApp and accessthe output files insidethe VA? Werelyonthe Opal 2 Toolkit Opal 2 Toolkitprovides a WS WrapperforApplications Operationsforstarting, monitoring and terminatingtheapplication. Supportfor local, MPI and Globus-basedexecutions. Output files accessible through Tomcat (computational steering). Generic Opal 2 WSDL App App App Opal 2 Toolkit Application Server (Apache Tomcat) Virtual Appliance Opal 2 Toolkitdeveloped @ NBCR
  22. Web serviceswrappertocomputationalapplications WS-WrappedApplications can nowbeorchestratedbythe Cloud Enactor (acting as a Task Manager). Applications can nowbecontrolled (started and monitored) insidetheScientific Virtual Appliance. Manyinstances of theapplication can beconcurrentlymanaged. Virtual Appliance Cloud Enactor (Task Manager) WS Wrapper (OPAL) Control, Monitor, Access files App API Client-Side OPAL API Hypervisor
  23. Scientificapplications Simulation of CardiacElectricalActivity ActionPotentialPropagationonCardiacTissues. Simulation of Guided Light in Photonic Crystal Fibers Optimization of SupercontinuumSpectrumusingGeneticAlgorithms. Optimization of ProteinDesignwith Target Properties ComputationallyIntensive, SimulatedAnnealing, Monte Carlo.
  24. conclusions Scientific Cloud Computing requirestoolstoabstracttheinteractionwith Cloud infrastructures. From Applications to Scientific Virtual Appliances At theGRyCAPwe are workingon: ApplicationContextualization Virtual Appliances Management The Cloud looks likeanalternativeapproachfortheexecution of scientificapplications. Definition of SpecificExecutionEnvironments
  25. Challenges in thenearfuture InteroperabilityamongClouds Avoidvendorlock-in Software GatewaysamongInfrastructureProviders LargeEcosystem of Virtual Machine Managers They share somefunctionalities and goals Developers like to code for the winning horse CommonAPIsfor Cloud Computing Apache LibCloud, Deltacloud, jclouds, Dasein Cloud API, Fog, etc. Clouds and Grids must provide Computational Support to Scientific Applications
More Related