VLAM-G Run Time system

1. VLAM-GRun Time system VLAM-G developers team Computer Architecture and Parallel Systems GroupDepartment of Computer ScienceUniversiteit van Amsterdam National Institute for Nuclear and High Energy Physics

2. Outline • Principles • VLAM-G Architecture • Writing VLAM-G modules • VLAM-G C and C++ API • VLAM-G module skeleton generator tool • Conclusions

3. VLAM-G:Run Time System • Features: • Data-flow-like experiments with modules • Control parameters & read state • Interact with the Grid layer • Interaction with VL RTS? • Module developers: API • End-users: None (transparent)

4. VLAM-G:Run Time System • Principals: • Modules are continuously active • Instantiated once • awaken when data arrives • Modules interacts by passing data over Gftp streams • The VLAM-G RTS inteacts with the module using the Command Channel of the Gftp streams

5. VLAM-G:Run Time System • VLAM-G RTS • Instantiate the modules • create the data channels between the modules (Third party arbitration). Contacts the producer and the consumer and specify the specify the communication port number. • VLAM-G RTS implements a CORBA interface

6. VL RTS Architecture AM VL RTS-Manager FrontEnd Node A Node B VL VL AM AM RTS-Local-Manager RTS-Local-Manager Globus (Duroc/GRAM) Globus (Duroc/GRAM)

7. XML description of the experiment topology XML Translator IIOP VLAM-G RTS manager Corba Interface Module Factory Module 1 Module 2 Module n Connection Factory Conn. n Conn. 1 Conn. 2 Control Connection Module 1 Module n Module 1 Globus Network

8. Connection setting VLAM-G RTS manager Connection Factory Module connector (1) (2) Module Skeleton Module Skeleton Module Body Module Body Gftp server Gftp server (3)

9. Module parameters interface VLAM-G RTS manager Module Factory Module controler 1 Module controler 2 (1) (2) Module Skeleton Module Skeleton Module Body Module Body Gftp server Gftp server

10. Launching VLAM-G jobs VLAM-G RTS manager Module Instance Module Instance Module Instance Module Launcher Module Launcher Module Launcher Globus-run gatekeeper gatekeeper gatekeeper jobmanager jobmanager jobmanager

11. Writing VLAM-G modules • VLAM-G Module is composed of: • Ports definition: the module writers need to define a port for each data type he wants to exchange with other modules within the VLAM-G environment • user code: the module writers C or C++ code • a few calls of read/write functions composing the RTS API: Whenever the module writer want his code to read or send data to other module, he must insert a call to the appropriate read/write function.

12. Defining the module’s ports • Ports are unidirectional and should have a specific data type • XDR functions have to be defined for each data type which allow to exchange data in a standard network format • simple data type: the standard XDR library provides primitives which converts simple types (integer, float, string, etc.) to XDR representation • complex data type: need to be written

13. VLAM-G RTS: C API • C API composed of a set of primitives that allow the module writer to manipulate basic data types • The C API hides the generation of XDR function from the module writer • Complex data types such as structures and union must be decomposed into basic types.

14. VLAM-G RTS: C API • Read methods • int readInteger(int *) • double readDouble(int *) • float readFloat(int *) • Write methods • void writeInteger(int) • void writeDouble(double) • void writeFloat(float)

15. Example of porting C code #include <vlstream.h> int vlmain (int argc, char ** argv) { int a; double b; vlistream readPort(“IN”); vlostream writePort(“OUT”); /* read data from input port */ readPort.readInteger(&a); /* process the data user code */ b= ProcessData(a); /* write data to the output port */ writePort.writeDouble(b); } Module Skeleton Module Body Gftp server

16. VLAM-G RTS: C++ API • The VLAM-G C++ API provides a number of classes to refer to different constructs in VLAM-G model such as ports and parameters. They should be extended: • VLAB_input_port • VLAB_output_port • VLAB_param

17. Example of C++ code #include <rpc/xdr.h> #include “vloutport.h” class wr_port: public VLAB_output_port {public : wr_port(const std:: string &nm):VLAB_input_port{}; bool_t xdrf(XDR *xdr, void *pdata) { return xdr_double (double *) pdata) }; }; int vlmain (int argc, char** argv) { // declaring the port wr_port wport(“Port_0”); // some processing of the data double pa; ... // write the data to the output port wrport.write(&pa); } Module Skeleton Module Body Gftp server

18. Module Skeleton generator Utility Module Description Editor Module Body Xml description of the module Source Code Module Skeleton Module skeleton Generator Module Body Note: The module writer still needs to do some minor editing on the generated code to make it real VLAM-G module: add some read and write to ports inside the source code. Gftp server

19. Conclusions

20. End of the new version of the slides

21. Communications VL-AM RTS User code User code GridFTP enabled HPSS