int.eu.grid: A grid infrastructure for interactive applications

int.eu.grid:A grid infrastructure for interactive applications Gonçalo Borges LIP on behalf of Int.EU.Grid Collaboration INGRID’08, Italy, April 2008

Do you really need to be motivated? • What is interactivity? • Feedback channel between you and your application • Interactivity is strongly connected to visualization • Why interactivity? • Users want answers in seconds and not hours • Grid debug, check application evolution, ... • Is it possible to have it on the grid? • Yes it is!!! • Although the several middleware layers and the crossover of multiple administrative domains. • Benifit for you as a grid user? • Power of the grid available at your fingertips! INGRID’08, Italy, April 2008

Interactivity on the Grid • VOs and users are anxious to get it • Needed by a wide set of applications in different scientific domains • Interactivity has been neglected by larger Grid projects… • Standard grids are aimed to sequential jobs… • How to properly set “Matchmaking” and “Brokering” for interactive tasks? • How to start the application immediately? • Even in scenarios in which all computing resources might be running batch jobs? • And what about online application input/output? Int.EU.Grid Provide an advanced grid empowered infrastructure for scientific computing targeted to support demanding interactive(andparallel) applications. INGRID’08, Italy, April 2008

Int.EU.Grid grid infrastructure • 12 sites, 7 counties • 9 in production • 4 in development • ~ 900 COREs • Xeon • Opteron • Pentium • ~ 45 TB of storage space • Interconnection by Geant INGRID’08, Italy, April 2008

Int.EU.Grid services • Distributed services • Taking advantage of the partners expertise • Redundancy • Better use of resources • Helpdesk • Security • coordination • Production Core • Services • CrossBroker • RAS • BDII • VOMS • LFC • MyProxy • R-GMA for development • APEL accounting • GridICE • R-GMA • SAM • Network • monitoring • Development Core • Services • CrossBroker • RAS • BDII • VOMS • LFC • MyProxy • Pure gLite WMS • Autobuild • Repository • Production Core • Services • CrossBroker • RAS • BDII • VOMS • LFC • MyProxy INGRID’08, Italy, April 2008

Int.EU.Grid Virtual Organizations • Applications • ifusion • ienvmod • iusct • ibrain • ihep • iplanck • iwien2k • icompchem • imrt • euforia • ihidra • icesga • imain, imon, itut, itest INGRID’08, Italy, April 2008

Public ips Legend: EGEE Node R-GMA Server LFC VOMS Server MyProxy I2G CB I2G RAS Top-BDII Core Services EGEE Nodes with I2G software I2G Node I2G WN software gLite WN I2G WN software I2G UI Site-BDII MonBox SE LCG-CE Batch Server I2G CE software gLite WN I2G WN software gLite WN Public ips Public/Private ips int.eu.grid Arch • int.eu.grid is a gLite based infrastructure but... • enhancing several services (lcg-CE, WN) and deploying new components (CB, RAS, UI) towards interactivity and MPI Local Services INGRID’08, Italy, April 2008

Submission of Interactive Jobs Submission of Parallel Jobs Interactivity and Visualization Data Management UI RAS MD GUI to access Grid Services Migrating Desktop and RAS (I) • Migrating Desktop (MD): User Friendly Grid Access • Java based GUI; Hides the details of the grid • Allows to log-in in the GRID independently from • where you are (laptop, desktop, everywhere ...) • what kind of Computer/OS you are using… • Roaming Access Server (RAS): • Gateway for Grid Access • Performs actions on the grid on behalf of the MD INGRID’08, Italy, April 2008

Migrating Desktop and RAS (II) • Work together in a client/server fashion • The RAS is a core service… • … installed at a central location with direct network connectivity • MD can run from private workstations… • … as long as it connects to the appropriate RAS ports • The RAS offers a well defined set of web services • Represents the integration level of interfaces for different middleware • Java 1.6 support • Job submission, job monitoring, data management operations, job channel forwarding, … • MD/RAS supports interactivity and real time visualisation • A plugin has to be developed for each application • Acts as an interface enabling bidirectional communication between a job and the local GUI. INGRID’08, Italy, April 2008

(i2)glogin ::== grid interactive tool • Interactivity is wrapped around (i2)glogin • CrossGrid tool extended in int.eu.grid to execute with GT4 • Supports GSS-based encryption • Enables online communication providing shell functionality for access to the Grid nodes • Sort of (grid) ssh using certificates • MD/RAS integration with i2glogin • A local instance of i2glogin is started in the RAS • Acting as the server • The client part of i2glogin is submitted to the grid through a JDL • The connection point (server:port) is specified as an argument • When the grid job connects, i2glogin connects back to the local instance • Creates a secure, low latency, bidirectional connection. INGRID’08, Italy, April 2008

I/O streaming $ i2g-job-submit interactive.jdl VirtualOrganisation = "imain"; JobType = “Normal"; Interactive = TRUE; InteractiveAgent = “i2glogin“; InteractiveAgentArguments = “-r –p 24599:158.109.65.149 -c“; Executable = “/bin/sh"; InputSandbox = {“i2glogin"}; $ i2glogin -p 24599:158.109.65.149 INGRID’08, Italy, April 2008

I/O streaming CrossBroker Job $ i2glogin -p 24599:158.109.65.149 WN User application i2glogin INGRID’08, Italy, April 2008

I/O streaming CrossBroker $ i2glogin -p 24599:158.109.65.149 WN User application i2glogin INGRID’08, Italy, April 2008

I/O streaming CrossBroker $ i2glogin -p 24599:158.109.65.149 sh-2.05b$ hostname aow5grid.uab.es sh-2.05b$ exit exit Connection closed by foreign host $ WN User application i2glogin INGRID’08, Italy, April 2008

GVID ::== Grid Video streaming • Visualisation capabilities are based on the compression of the OpenGL graphics generated by the application • GVID performs the encoding/decoding on both sides • Video encoding saves bandwidth • Communication over glogin • A GVID display client java implementation was develloped to follow requirements of MD/RAS application plugins • A generic java package is ready to be used by all grid devellopers trying to built a MD plugin • User inputs (such as mouse clicks) generated through the application specific plugin are sent via RAS to the application • The application has to be adapted to react to these input events INGRID’08, Italy, April 2008

JDL , TCP TCP TCP Transport Transport Transport MP4 MP4 MP4 Decoder Decoder Decoder Socket Socket Socket Java Video Player Java Video Player Java Video Player Transport Transport Transport File File File Transport Transport Transport Event Event Event Encoder Encoder Encoder Pipe Pipe Pipe Transport Transport Transport TCP Transport MP4 Encoder Socket Transport remoteGlut File Transport Event Decoder Pipe Transport Integration of the Interactivity & Visualization tools Simulation +visualisation Roaming Access Server JDL JDL CrossBroker CrossBroker Job Submission Job Submission Services Services Migrating Migrating Desktop Desktop gLogin gLogin Gatekeeper Gatekeeper , , LRMS LRMS WorkerNode WorkerNode Logging Logging & & Bookkeeping Bookkeeping WorkerNode TCP TCP Transport Transport MP4 MP4 Application Decoder Decoder Socket Socket GVid Java Video Player Java Video Player Transport Transport TCP TCP File File Transport Transport Transport Transport MP4 Encoder MP4 Encoder Event Event Socket Socket Encoder Encoder Transport Transport Pipe Pipe vtk App vtk App remoteGlut remoteGlut Transport Transport File File Transport Transport Event Decoder Event Decoder Pipe Pipe Transport Transport INGRID’08, Italy, April 2008

Support for interactivity via GVid/Glogin PACX-MPI and OpenMPI support RB CrossBroker MPI job startup based on MPI-START CrossBroker (I) • CrossBroker: Int.EU.Grid meta-scheduler • Offers the same functionalities as the EGEE Resource Broker, plus: • Support for Interactive Applications • Interactive agent injection • Scheduling priorities; • Time Sharing • Full support for Parallel Applications • PACX-MPI and OpenMPI • Flexible MPI job startup based on MPI-START INGRID’08, Italy, April 2008

CrossBroker (II) • The user can decide which interactive agent to use through special JDL requirements • The CrossBroker can inject it transparently to the user • If the job is recognized as interactive... • The CrossBroker treats it with higher priority • There is in place a mechanism to use bandwidth measurements in the matchmaking process • Great for application needing visualisation • But not really implemented... • If there are no available resources • Use a time sharing mechanism INGRID’08, Italy, April 2008

Time Sharing: Glide-in mechanism • Main idea: • Wrap every batch job with an agent (glide-in) • Agent will get control of the remote machine independently of its local resource manager. • Glide-in benefits on the interactivity framework • Agents enable simple multiprogramming between interactive and batch jobs. • Interactive jobs may run even when no free resources are available. • Agents can also be used as a fast start-up mechanism. • Agent can control the amount of CPU that an interactive job gets according to QoS requirements expressed by the user in the JDL. INGRID’08, Italy, April 2008

Response Time CrossBroker CE + WN INGRID’08, Italy, April 2008

Local configurations • Address the possibility of starting up the application immediately • Local Resource Management System level • Some sites have configured their queues so that • (Interactive) jobs do not remain in queue • Or there is immediate resources to execute them or they are resubmitted elsewhere • LCG-CE JobManager (JM) level • Some site have decided to move from the “lcgpbs” JM to the “pbs” one • In EGEE, a direct submission to a site completes in ~25s if the sites uses “pbs” JM against > 2m for the “lcgpbs” JM • “pbs” JM requires sharing homes between CE and WN (via NFS) • However, lcgpbs have been coded for scalability instead speading • Less CPU consumption when the CE is handling hundred of jobs in parallel INGRID’08, Italy, April 2008

EGEE short deadtime jobs (sdj) WG • sdj definition: • short execution time; unexpected and urgent • cannot be dealt on a best effort basis in a full production regime. • Use case & Requirements • Visualisation, Inspection, Interaction • Immediate access and small latencies • Dinamic attach a sheel to a running process • Debugging process • Bypassing middleware delays • “An alternative solution is “agent scheduling” or “overlay” systems which provide user task management” • int.eu.grid relies on gLite with additional enhancements Opportunity window to test interoperability with EGEE INGRID’08, Italy, April 2008

LFC REGISTRY LFC REGISTRY lcg-RB Top-BDII I2G CrossBroker Top-BDII I2G WN software I2G WN software I2G WN software gLite WN gLite WN gLite WN Migrating Desktop Batch Server SE Scheme: Enhancing an EGEE site with Int.EU.Grid features Int.EU.Grid Infrastructure EGEE Infrastructure Core Services gLite WN I2G WN software MPI, Visualization MPI, Visualization gLite WN Local Services UI LCG-CE gLite WN I2G WN software MonBox Site-BDII LCG-CE UI Site-BDII I2G UI MonBox LCG-CE I2G UI software I2G CE software gLite WN I2G CE software gLite WN I2G WN software gLite WN Batch Server SE INGRID’08, Italy, April 2008

Open Issues and Drawbacks • Information System is still a major problem • Delay reporting the site change of availability to Core Services • Still missing a generalized priority schema for grid users and jobs • Things are more easy if infrastructure is simple, but… • institutions will share the same local infrastructure between several Grids • Some solutions (under investigation): • Information system: CrossView • Fair share with different penalty factors • Batch jobs worsen the priority according to the resources used • Interactive jobs worsen the priority faster than batch ones • Interactive CE • Automatic injection of glidein wrappers for all jobs submitted from other sources different of CrossBroker INGRID’08, Italy, April 2008

Int.EU.Grid added value • int.eu.grid offers • Interactivity and Visualization • “On the Fly” interaction, “On the Fly” response; Graphical VO applications • Inter and intra cluster parallel tasks • PACXMPI & OpenMPI • User friendly access to resources • The Migrating Desktop • int.eu.grid features go in favour of fulfilling most of the requirements of users from other major grids… • VOs can take advantage of the interoperability solutions developed within int.eu.grid • Add/configure VO resources without losing their normal (gLite) capabilities. INGRID’08, Italy, April 2008

int.eu.grid: A grid infrastructure for interactive applications