1 / 20

Introduction to the CrossGrid Testbed

George Tsouloupas UCY HPCL. Introduction to the CrossGrid Testbed. What is the “CrossGrid testbed” ? A collection of distributed computing resources Supporting a “Grid environment” Objectives Development, Testing and validation Emphasis on interoperability with EU-DataGrid (EDG)

adair
Download Presentation

Introduction to the CrossGrid Testbed

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. George Tsouloupas UCY HPCL Introduction to the CrossGrid Testbed

  2. What is the “CrossGrid testbed” ? • A collection of distributed computing resources • Supporting a “Grid environment” • Objectives • Development, Testing and validation • Emphasis on interoperability with EU-DataGrid (EDG) • Extension of GRID across Europe

  3. Testbed Overview • A collection of distributed computing resources • 16 sites (small & large) in 9 countries, connected through Géant + NRN • 115 CPUs (Worker Nodes) • 4 TB (Storage Elements) • + Grid Services: Middleware (based on Globus) RB, VO, RC… • A group of people and resources make up the CrossGrid Virtual Organization (VO) and trust each other through national Certification Authorities.

  4. Testbed resources Network

  5. The Local Testbed We currently have: • 1 CE (apelatis.grid.ucy.ac.cy) 2xPIII 1.2GHz 1GB • 1 SE (se001.grid.ucy.ac.cy) 2xPIII 1GB 40GB • 1 UI (thales.grid.ucy.ac.cy) P4 2GHz 1GB 40GB • 4 WN (wn001-004.grid.ucy.ac.cy) P4 2GHz 1GB 40GB • CA • LCFG Installation server

  6. CE UI Submit apelatis thales 2xPIII 1.2GHz 1GB RAM 18GB scsi 100Mbit P4 2GHz 1GB RAM 40GB scsi 1000Mbit SE nfs 2xPIII 800MHz 1GB RAM 128GB scsi /home NFS WN WN nfs WN WN LCFG CA P4 2GHz 1GB RAM 40GB 1000Mbit (installation server) (certificate generation) CURRENT local testbed status local testbed

  7. Grid Security • Security is a primary concern when it comes to Grids. • The Grid Security Infrastructure is based on Public Key Infrastructure. • A user is authenticated based on possession of a private key (known only to the user). • Authorization depends on whether the user's certificate is signed by an accepted Certification Authority (In our case the CyGrid CA) • The gridmap-file at the target resource will determine the if a resource will accept a specific user, and specifies a mapping to a local user

  8. Crossgrid CA's

  9. Create a certificate request file: Under Globus use command: grid-cert-requestOr using openssl command:    openssl genrsa -des3 -out user.key;  openssl req -new -x509 -days 365 -key user.key -out user-req.crt DN Format :    person:   "/C=CY/O=CyGrid/O=HPCL/CN=user name"   machine:   "/C=CY/O=CyGrid/O=HPCL/CN=host/machinename.ucy.ac.cy" • Send by email your request file to: cygrid-ca@ucy.ac.cy and your RA: hpcl@ucy.ac.cy DO NOT delete either the key file or the request file. It is the sole copy of your private key, Most importantly: keep it secure! • You will receive your certificate which is good for one year. • Install your certificate on the User Interface machine. Obtaining a certificate

  10. ex: globus-job-submit CE A machine with globus client CE ex: dg-job-submit CE A machine installed as a EDG UI Resource Broker Job Submission CrossGrid/Datagrid Services

  11. Running a Job • User logs in on the UI • User issues a grid-proxy-init and enters his certificate's password, getting a valid Globus proxy • User sets up his JDL file. Example: JDL of the “hello world” job Executable = "/bin/echo"; Arguments = "Hello World"; StdOutput = "Messagge.txt"; StdError = "stderr.log"; OutputSandbox = {"Message.txt","stderr.log"};

  12. Running a job • User issues a: dg-job-submit HelloWorld.jdl and gets back from the system a unique Job Identifier (JobId) • User issues a: dg-job-status <JobId> to get logging information about the current status of his Job • When the “OutputReady” status is reached, the user can issue a dg-job-get-output JobId and the system returns the name of the temporary directory where the job output can be found on the UI machine.

  13. George Tsouloupas UCY HPCL Introduction to LCFG

  14. What is LCFG? • Local ConFiGuration system • A tool for installation and configuration of large scale, diverse, dynamically changing systems. • Installs an RPM based Linux system and configures it. • Config of entire site held in source files on central server • Each machine defined by exactly one profile. • “Profile” constructed for each machine • Utilise inheritance • Based on RPM lists, key-value pairs

  15. DHCP Server Provides network information. Software (LCFG) Manages the rpm repository and machine profiles. NFS Server Serves RPM repository. Web Server Server XML machine profiles. LCFG Server Components

  16. Config files XML profiles /etc/shadow LCFG Config Files /etc/services /etc/group Read Profile Load Profile +inet.services telnet login ftp +inet.allow telnet login ftp sshd +inet.allow_telnet ALLOWED_NETWORKS +inet.allow_login ALLOWED_NETWORKS +inet.allow_ftp ALLOWED_NETWORKS +inet.allow_sshd ALL +inet.daemon_sshd yes ..... +auth.users myckey +auth.userhome_mickey /home/mickey +auth.usershell_mickey /bin/tcsh <inet> <allow cfg:template="allow_$ tag_$ daemon_$"> <allow_RECORD cfg:name="telnet"> <allow>192.168., 192.135.30.</allow> </allow_RECORD> ..... </auth> <user_RECORD cfg:name="mickey"> <userhome>/home/MickeyMouseHome</userhome> <usershell>/bin/tcsh</usershell> </user_RECORD> HTTP /etc/inetd.conf /etc/passwd .... mickey:x:999:20::/home/Mickey:/bin/tcsh .... rdxprof ldxprof /etc/hosts.allow in.telnetd : 192.168., 192.135.30. in.rlogind : 192.168., 192.135.30. in.ftpd : 192.168., 192.135.30. sshd : ALL Profile Profile Generic Object Object Make XML Profile Component Web Server Local cache XML Profile inet auth LCFG Objects Client nodes Server How LCFG Works in One Slide. Abstract configuration parameters for all nodes stored in a central repository A collection of agents read configuration parameters and either generate traditional config files or directly manipulate various services

  17. LCFG Installation • Custom install Red Hat 6.2 including • NFS, web server • Install LCFG RPMs and directory tree • Configure web server • Populate RPM repositories • Make root filesystem available over NFS • Configure DHCP server • Create client profiles

  18. Profiles • A profile contains a list of RPMs to be installed and configuration data • Located in /var/obj/conf/profile/source. • Edit site-cfg.h => globus.conf , info-mds.conf, gdmp.conf andothers. • Files testbed001-testbed004 are sample UI, CE, SE and WN. Rename one, e.g. apelatis. • Edit apelatis; hostname and hardware.#include "AIC_ComputingElement-cfg.h"...+update.modlist eepro100+update.mod_eepro100 alias eth0 eepro100+update.rootpwd hj%4aJkf\k45

  19. Profiles (cont) • Contents of "AIC_ComputingElement-cfg.h" #include "ComputingElement-cfg.h" +update.rpmcfg CE_AIC7xxx-rpm +update.disks sda +update.bootdisk sda +update.dopartition_sda yes +update.partitions_sda sda2 sda1 sda3 +update.pdetails_sda1 64 /boot +update.pdetails_sda2 free / +update.pdetails_sda3 1000 swap

  20. Questions ? ...then more examples on prometheus.grid.ucy.ac.cy (our LCFG server)

More Related