How to run GILDA tutorials

1. How to run GILDA tutorials Riccardo Bruno INFN.CT 30.06.2008 – 04.06.2008

2. GILDA t-Infrastructure https://gilda.ct.infn.it • Reasons why people should need a t-Infrastructure: • Introduce the Grid paradigm to new scientific communities • Provide expertise on application porting • Test of new releases of Grid services • Development on Grid (new services) • Test of middleware co-existence and interoperability • Provide support • Deliver training • Provide training materials

3. GILDA Tutorials • Tutorial for Users • Tutorial for Site Administrators • Grid Schools/Tut. for App. developers • Customized events • Tutorial for Trainers

4. Some concepts … • The virtualization • GILDA uses VMWare to virtualize PCs/Grid Services • VMWare – (http://info.vmware.com/) • An alternative software is: Xen – (http://www.xen.org/) • GILDA Virtual Services • Several virtual services are ready to use at: • https://gilda.ct.infn.it/VirtualServices.html • The most important Virtual Service is the virtual User Interface • https://gilda.ct.infn.it/GILDAVM/gLite-UI-VM3.0.tar.gz • The UI PnP (https://gilda.ct.infn.it/UIPnP.html) • Sometimes it is useful to install a UI on an existitng Linux distro • No root privileges are necessary • Easy to install and configure • Grid2Win (http://grid2win.gilda-forge.ct.infn.it/new/index.php) • Recently GILDA supports/tests a Windows porting of gLite

5. Tutorial for users … • Provides mostly Training on Grid middleware • But GILDA also supports the multi-middleware co-existence: gLite, GTK4, UNICORE, Condor, OMII-UK (OURGrid, GTRS) • GILDA offers tools developed by supported projects like: Grid2Win, SecureStorage, GSAF, gLibrary/DRI • The aim of the tutorial is to let student APPRECIATE the service offered by the GRID and let him able to operate on it We will focus on training for gLite

6. Tutorial for Users Requisites • Audience Requisites: • Basic Unix/Linux commands knowledge (command line) • Basic concepts of networking (ssh, scp, etc.) • Technical Requisites: • A computing room equipped with computers that can be used by students or a simple class room where students can bring their own laptops • To access to the Grid, 3 possibilities: • Use of a dedicated UI server • Use of UIs running on virtual machines • Use of the UI PnP on Linux Accounts • Use of the GILDA glite-tutor and glite-tutor2 UI servers • A network connection with the outside Internet bandwidth of at least 8 Mbit/s • a FastEthernet (10/100 Mbit/s) and/or a Wi-Fi (IEEE802.11b/g802.11b/g at 11/54 Mbit/s) local area network • a big screen and a video-projector; • a sound system (microphone + amplifier + loud speakers) if the room is big. Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.

7. Requesting tutorial for users 1/3 • From GILDA web site: https://gilda.ct.infn.it

8. Requesting tutorial for users 2/3

9. Requesting tutorial for users 3/3 • From GILDA site admin. a email will come containing: • A tarball file containing usercertificates • [location]1/ [location]1_userkey.pem [location]1_usercert.pem [location]1_userkey.p12 [location]2/ [cityname]2_userkey.pem [location]2_usercert.pem [location]2_userkey.p12 … [location]n/ [location]n_userkey.pem [location]n_usercert.pem [location]n_userkey.p12 • The account names and passwords associated to students • Normally follow simple rules related to the account numbers ATTENTION! The number of users is limited to max 80 persons Accounts will be created on GILDA UI servers: glite-tutor.ct.infn.it glite-tutor2.ct.infn.it Specified IP address in the request form will be enabled for incoming connections to both machines The PASSPHRASE is common to ALL generated user certificates It is: [location] (all capital letters) GILDA tutorial certificates normally expire a week after the end of the tutorial. Users might want to continue to use GILDA have to follow the instructions: https://gilda.ct.infn.it/users.html

10. Tutorial for Users: The UI • A single UI server • Usage of UIs running on Virtual Machines The Grid The UI server The student room The Grid The student room

11. Tutorial for Users: glite-tutor • GILDA relies on two UI servers: • glite-tutor.ct.infn.it • glite-tutor2.ct.infn.it The student room glite-tutor.ct.infn.it The Grid Users’ /home/… GENIUS Web portal glite-tutor2.ct.infn.it

12. Tutorial for Users: Exercises • Students need to make practice on • GILDA maintains a tarball file containing lot of exercises ready to be used by students • It’s a good practice to copy exercises on ALL students home directories. • The tarball of exercises is available on:https://gilda.ct.infn.it/RPMS/GILDA-TAR-APPLICATIONs.tar.gz • For each student account just execute: • wget https://gilda.ct.infn.it/RPMS/GILDA-TAR-APPLICATIONs.tar.gz --no-check certificate • tar xvfz GILDA-TAR-APPLICATIONs.tar.gz ATTENTION! Install the tarball file into the $HOME directory The classic and simplest JDL example among exercises is under: gLite/Other/hostname.jdl Other examples come from supported application portings made by the GILDA Team Ohter examples show different gLite middleware capabilities, like: MPI Jobs Use of APIs Use of well known software packages like: OpenFoam, Matlab, etc.

13. Tutorial for Users: Testing the UI • In case the students´ UI is centralized, it is a good practice to make stress tests on the UI in order to discover potential problems. • GILDA developed several stress tests for WMS and Storage and Cataloguing services. • Stress tests may discover problems on resource overloading, network latencies, certificates, etc. • It’s a good practice to perform at least once a stress test over the UI hosting students´ accounts. ATTENTION! To request the stress test scripts we kindly invite you to send a email to grid-prod mailing list: grid-prod.ct.infn.it

14. Tutorial for Users: Other scenarios • A local RB serving the site The Grid The local UI server The student room The local RB server The local Grid Portal

15. Tutorial for users: Student PCs • Two different solutions: • PCs running Windows • Office suite + PDF viewer (Adobe) • SCP Client • SSH Client • PCs running Linux • X Interface • Office suite + PDF viewer (xpdf) • Suggestions (available free software): • Windows: • MS Office (if licenced) or OpenOffice (http://www.openoffice.org/) • SCP: WinSCP (http://winscp.net/eng/docs/introduction) • SSH: Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/) • LinuxLike solution with CygWin • Linux: • Office package: OpenOffice (http://www.openoffice.org/) In this tutorial followin SSH client will be used:

16. User tutorial: Running the tutorial • Every day ask students to sign a list containing at least: • Name, Surname, Institute, Country, email • Tutors will describe theory first • For practice sesssions Tutors may choice among: • Use of dedicated slides to explain exercises • Use of the GILDA Wiki Pages (recommended): • https://grid.ct.infn.it/twiki/bin/view/GILDA/WebHome • During practice sessions ALL participating tutors will support students requesting help • At the end of the tutorial don’t forget to ask students to fill the Feedback Form! (see later) • At the end of the tutorial provide the Training Task Leader with the list of students and the feedback values.

17. Tutorial for site administrators ..… • Provides mostly Training on Grid Middleware • GILDA actually supports the multimiddleware co-existence: gLite, GTK4, UNICORE, Condor, OMII-UK (OURGrid, GTRS) • The aim of this tutorials is to explain how to INSTALL and MAINTAIN grid sites. We will focus on training for gLite

18. Tutorial for S.Adm: Requisites • Audience Requisites: • Advanced Unix/Linux commands knowledge (command line) • Unix/Linux administration knowledge • Good knowledge of networking and relative services (DNS, Firewall, etc.) • Technical Requisites: • A network connection having outside Internet bandwidth of at least 8 Mbit/s • a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN • A computing room equipped with desktops that can be used by students or a simple class room where students can bring their own laptops • At least one machine (either real or virtual, with Xen or Vmware) each 2 users. Machines must have direct and inverse resolution, so either  :     - Machines with public IP, direct/reverse solved by public DNS (or)     - Machines with private IP solved by a local DNS ("pointed"  by students) • Real machines should have at least 1 GB of memory per processor (dual processor machines are preferred) and at least 40 GByte of internal hard-disk • The machines must be at the disposal of tutors at least 1 week before the tutorial • A big screen and a video-projector (a 2nd video-projector for the machine to install would be better); • A sound system (microphone + amplifier + loud speakers) if the room is big. Sites with local network slower than 100 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.

19. A further step is necessary after the form submission. It is necessary to send a list of hostnames to the: GILDA-CA mailing list (gilda-ca@ct.infn.it) This list of hostnames will be used to generate GILDA HOST certificates. Tutorial for S.Adm: Request • The requesting procedure is the same as for users … ATTENTION! The number of users is limited to max 30 persons Specified IP addresses in the request form will be enabled for incoming connections to both GILDA UI servers GILDA host certificates for tutorials normally expire after the end of the tutorial. Users might want to continue to use GILDA and have to follow this procedure: https://gilda.ct.infn.it/sites.html Requested GILDA host certificates have one year of validity

20. Tutorial for S.Adm: Preparation • Dedicate a PC hosting a web server and make it accessible by students • Create a mirror of the INFN Grid software repository (instructions will follow on this) • http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:tips:mrepo • Create a directory seen by the web server and copy on it: • GILDA Host certificates • GILDA User certificates • The java JDK 1.5.0 RPM (Java Development Kit 5.0 Update XX) • The tarball of GILDA exercises • A directory containing all slides (sometime indico is slow or unreachable or just contain last minute slides not yet uploaded into the Agenda). • VM Image with BASIC installation (SL4 + pre-requisites)

21. Tutorial for S.Adm: Scenario • A Dedicated PC for each service • Usage of Virtual Machines Any Grid Service The student room The student room

22. Tutorial for S.Adm: Final Exercise • A key point in tutorial for site administration: • Each student installs a service storing the VM image file • Let installed machine cooperate as a Local grid site ( ) The student room

23. Tutorial for S.Adm : Run the Tut. • Every day ask students to sign a list containing at least: • Name, Surname, Institute, Country, email • Tutors will describe theory first, then the installation of the grid service will follow • Students will install the service in a dedicated and separated slot in the Agenda. • Students may also have a look on GILDA wiki pages: • https://grid.ct.infn.it/twiki/bin/view/GILDA/GILDASiteInstall • During practice sessions ALL participating tutors will support students requesting help • At the end of the tutorial don’t forget to ask students to fill the Feedback Form! (see later) • At the end of the tutorial provide the Training Task Leader with the list of students and the feedback values.

24. Grid schools … • Aims to support students in porting their own applications • Requires a big effort in terms of human and physical resources • A survey of potential application must be done in advance • A selection of posted application must be done in accordance of well defined selection and evaluation criteria

25. Grid Schools: Requisites • Audience Requisites: • Researchers having User tutorial requisites • The Application responsible must partecipate to the GridSchool • Technical Requisites: • A network connection having outside Internet bandwidth of at least 8 Mbit/s • a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN • A computing room equipped with desktops that can be used by students or a simple class room where students can bring their own laptops • At least 10 real or virtual (with Xen or Vmware) machines. These machines must have publicIPaddresses and must be registered on a local DNS with direct and inverse name resolution. • Real machines should have at least 1 GB main memory per processor (dual processor machines are preferred) and at least 40 GByte of internal hard-disk • The machines must be installed by tutors at least 1 week before the tutorial • A big screen and a video-projector (a 2nd video-projector for the machine to install would be better); • A sound system (microphone + amplifier + loud speakers) if the room is big. • Appropriate Surveillance, Fire estinguishing systems, Cooling fans and Electricity power must be granted during the school Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid schools.Sites with local network slower than 100 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.

26. Grid Schools: Request • There is a standardized way to organize a GridSchool, like tutorials for users and site admin tutorials • Normally Grid Schools are planned at the beginning of a project due to their huge request of resources • Planning are normally done on TB Meetings or during Workshops. • In order to collect applications a survey must be submitted to the community • The list of potential applications must be provided to NA2 and the TB in order to be evaluated ATTENTION! The number of application is limited to max 15 The number of days shouldn’t be less than 10 (working days) Applications will be installed on GILDA or other Organizations (EELA-2)

27. Grid Schools: Preparation • A local grid site must be installed by tutors one week before the school • Schools requires 2 rooms • One room dedicated to Grid Services • One room dedicated to Students • Computer room should be deployed far from the student room in order to reduce heating and noise • Student room can be arranged in different fashions (U/O-shape, classroom shape, etc.) • Student PCs must be installed exaclty like in a user tutorial

28. The Local Grid The student room Computers room Grid Schools: Scenario

29. Grid Schools: ROC on Duty • Normally Grid Schools are a very good opportunity to train site managers to the ROC on Duty tutorial • The Regional Operation Center (ROC) has the responsability for providing the Grid Services. This responsibility includes: • Middleware certification • Middleware deployment • Day to day operations • User support • This tutorial AIMS to explains the ROC activities and Policies adopted by the GILDA’ supported project.

30. Customized events … • Sometime a generic tutorial does not fit training requests coming from the hosting institute • In such case GILDA has to provide a focused training: • Examples: • Training focused on HPC and MPI Jobs • Application developers days • Training on GILDA services: SecureStorage, GSAF, gLibrary/DRI…

31. Customized events: Requisites • Due to the unhespected nature of the target topic to cover; requisites may be very different • A good starting point maight be to refer to: • Same requisites of: Tutorial for Users • Same requisites of: Tutorial for SiteAdministrators

32. Customized events: Request • There is a standardized way to organize a Customized event, like tutorials for users and site admin tutorials • Normally Customized events are planned during the lifetime of the supported project • Planning are normally done on TB Meetings or during Workshops. • Customized events normally need more time to be organized due to the unhespected nature of the target topic

33. Training for Trainers … • GILDA currently supports many projects: EELA-2, EGEEIII, COMETA, etc. • Projects actually cover many continents: Europe, Asia, Latin America, etc. • Training means costs for the project, then the need of: Local Tutors. • GILDA ever encourages the creation of local team of tutors • Training for Trainers is the opportunity to become a GILDA Local Tutor. • The aim is to form Local Tutors

34. Training for Trainers: Requisites • Audience Requisites: • Same requisites of Site Administration Tutorial • Knowledge of gLite or a previous participation to any GILDA Tutorial • Technical Requisites: • A network connection having outside Internet bandwidth of at least 8 Mbit/s • a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN • A computing room equipped with desktops that can be used by students or a simple class room where students can bring their own laptops • At least 10 real or virtual (with Xen or Vmware) machines. These machines must have publicIPaddresses and must be registered on a local DNS with direct and inverse name resolution • Real machines should have at least 1 GB of memory per processor (dual processor machines are preferred) and at least 40 GByte of internal hard-disk • The machines must be at the disposal of tutors at least 1 week before the tutorial • A big screen and a video-projector (a 2nd video-projector for the machine to install would be better); • A sound system (microphone + amplifier + loud speakers) if the room is big Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid schools.

35. Training for Trainers : Request • There is a standardized way to organize a Training for Trainers event, like tutorials for users and site admin tutorials • Normally Training for Trainers events are planned at the beginning of the supported project • Planning are normally refined on TB Meetings or during Workshops.

36. Common operations Following actions are common for any kind of Training Event …

37. Common: The Agenda • The training event need to be published; • GILDA uses CERN’ Indico Agenda System • EELA-2 Project: (http://indico.eu-eela.org/) • EGEEIII Project: (http://indico.cern.ch/categoryDisplay.py?categId=41) ATTENTION For EELA-2 project the preferred timetable default style is: Indico Style Please do not forget to ask Tutors to provide slides, Then upload them into the Agenda

38. Common: The feedback form • The training event need a Feedback Form; • Why the feedback form: • Improve the quality of training material • Improve the quality of training • Obtain suggesions from students • Report the quality of training to the funding institute • The FF just collects student feedbacks • Normally one feedback per Talk • 6 degree of choice • Poor, Insufficient, Fair, Good, Very Good, Excellent • 1 comment field ATTENTION! There is no actually a official request form for the Feedback Form To request a FeedbackForm just send a email: TO: rita.ricceri@ct.infn.it CC: grid-prod@ct.infn.it

39. Common: Images • Few images of GILDA tutorials ATTENTION! Please do not forget to bring your camera! Photos are very useful for many cases like: Creating project booklets Designing Web sites Creating document reports …. Grid SchoolComputer Room User Tutorial Grid SchoolStudent Room

40. Questions …