1 / 12

CREAM CE: report for the OSCT

CREAM CE: report for the OSCT. Massimo Sgaravatto - INFN Padova On behalf of the CREAM team. Submission through the WMS. WMS. CREAM. CREAM. CREAM. What is CREAM. CREAM service: Computing Resource Execution And Management service

aileen-ramirez
Download Presentation

CREAM CE: report for the OSCT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CREAM CE: report for the OSCT Massimo Sgaravatto - INFN Padova On behalf of the CREAM team

  2. Submission through the WMS WMS CREAM CREAM CREAM What is CREAM • CREAM service: Computing Resource Execution And Management service • Web service for job management operations at the Computing Element (CE) level • Allows to submit, cancel, monitor, … jobs • CREAM can be used: • through the gLite WMS • by a generic client willing to interact directly with the CE • We provide and maintain an “official” CREAM CLI, but users can build their own clients using a Web Service framework EGEE08 conference - Istanbul, Sept 22-26, 2008

  3. What is deployed on a CREAM CE • CREAM • Web application • CEMon with CREAM-job sensor • Web application • Used to notify client (WMS/ICE) about CREAM job status changes • Not strictly needed to have CEMon deployed on the CREAM CE node • Glexec • Used for credential mapping • Uses LCAS and LCMAPS • BLAH • Used for interacting with the batch system • “Server part”: installed on the CE node • BLParser: running where the batch system log files are available • Mysql • Not strictly needed to have Mysql deployed on the CREAM CE node • Other services running as in the LCG CE • LB locallogger • To collect LB LRMS events and send them to the LB server • BDII • To publish Glue schema compliant CE information • gridftpd • Uses LCAS and LCMAPS EGEE08 conference - Istanbul, Sept 22-26, 2008

  4. AuthZ mechanisms • CREAM and CEMon use gJAF • Developed by the EGEE security cluster • “Official” authZ solution for the Java world in EGEE • Glexec and gridftp use LCAS and LCMAPS • We will have a single authorization mechanism when the new Authorization Service is implemented and adopted in all these software components EGEE08 conference - Istanbul, Sept 22-26, 2008

  5. Ports used on a CREAM CE • Port used by the the Web-application container (tomcat) to run both the CREAM and the CEMonitor applications • Default: 8443 • Administrative tomcat ports • Default: 8005 and 8009 • Port used for accepting notifications about job status changes from the job wrapper and from the BLParser • Specified by LRMS_EVENT_LISTENER_PORT in the CREAM config file • Default: 9091 • It is suggested to close this port to all nodes expect the WNs and the one running the BLparser • Port used for the CEMon CREAM-job sensor • Specified by CREAM_JOB_SENSOR_PORT in the CREAM config file • Default: 9909 • It is suggested to close this port to all nodes except the one running CEMon (which by default is the CREAM CE node) EGEE08 conference - Istanbul, Sept 22-26, 2008

  6. Ports used on a CREAM CE • The ports used for gridftp server, • They are the same used in the LCG CE • Default: 2811 (control channel) and range 20000-25000 (data channel) • The ports used for LB local logger • They are the same used in the LCG CE • Default: 9001 and 9002 • The ports used for the BDII running on the CE • They are the same used in the LCG CE: • They are specified in the BDII conf file • Default: • BDII_PORT_READ=2170 • BDII_PORTS_WRITE="2171 2172“ • The mysql port • Default: 3306 EGEE08 conference - Istanbul, Sept 22-26, 2008

  7. Ports used on a CREAM CE • The ports used by the BLAH BLParser, if it is running on the CE node (the parser runs where the batch system log files are) : • PBS • PBS parser listening port(s): • Specified by GLITE_CE_BLPARSERPBS_PORTx in the BLParser conf file • Default 33332 • PBS parser listening CREAM port(s) • Specified by GLITE_CE_BLPARSERPBS_CREAMPORTx in the BLParser conf file • Default 56565 • LSF • LSF parser listening port(s) • Specified by GLITE_CE_BLPARSERLSF_PORTx in the BLParser conf file • Default 33333 • LSF parser listening CREAM port(s) • Specified by GLITE_CE_BLPARSERLSF_CREAMPORTn in the BLParser conf file • Default 56566 EGEE08 conference - Istanbul, Sept 22-26, 2008

  8. Useful logfiles • CREAM and CEMon use log4j, the log can be customized according to the log4j configuration • The default log files are: /opt/glite/var/log/glite-ce-cream.log(.*) /opt/glite/var/log/glite-ce-monitor.log(.*) • glexec logs on syslog and on the file specified in the glexec conf file (/opt/glite/etc/glexec.conf) • default /opt/glite/var/log/glexec_lcas_lcmaps.log • The log files for the container (tomcat) are located in: /usr/share/tomcat5/logs/ • The BLparser log file (if it runs on the CE node) • Location specified in the blparser conf file (/opt/glite/etc/blparser.conf) • Default: /opt/glite/var/log/glite-[lrms]parser.log • On-going improvement of logging stuff (see bug #40821) EGEE08 conference - Istanbul, Sept 22-26, 2008

  9. Control mechanisms • The authZ in CREAM is mainly VOMS-based; a site admin can remove a given VO/Role item from the gridmap file • If it is necessary to ban a single user the site admin must: • insert the following specification in the CREAM configuration file (/opt/glite/etc/glite-ce-cream/cream-config.xml): <plugin name="ban-plugin" classname="org.glite.security.authz.pdp.BlackListServicePDP"> <parameter name="blackListFile" value="/opt/glite/etc/glite-ce-cream/banned.lst" /> </plugin> into the authzchain section just before any other plugin • insert into the file /opt/glite/etc/glite-ce-cream/banned.lst the DN of the user EGEE08 conference - Istanbul, Sept 22-26, 2008

  10. Admin • A user can manage (check status, cancel, etc.) only her jobs • It is possible to define admins for a CREAM CE • An admin of a CREAM CE can manage all (not only her own) jobs submitted to that CE • An admin can disable/re-enable job submissions to that CREAM CE (see next slide) • To specify an admin for a CREAM CE, just add her DN on /etc/grid-security/admin-list EGEE08 conference - Istanbul, Sept 22-26, 2008

  11. Start/stop, disabling of job submissions • To stop CREAM, just stop the container: • /etc/init.d/tomcat5 stop • To start CREAM, just start the container: • /etc/init.d/tomcat5 start • An admin (see previous slide) can decide to disable new job submissions (e.g. because that CE will have to be turned off) • This is done calling a proper operation (provided by the CREAM CLI as glite-ce-disable-submission) • When job submissions have been disabled, the other operations (e.g. job status, etc.) are still allowed • An admin can then re-enable job submissions • This is done calling a proper operation (provided by the CREAM CLI as glite-ce-enable-submission) EGEE08 conference - Istanbul, Sept 22-26, 2008

  12. Proxy renewal • Current approach (implemented in BLAH) • BPRserver process running on the WN for each job • BLAH lets a BPRclient process on the CE contact the correct BPRserver process on the job's WN • Since it cannot predict the port on which the BPRserver listens, it tries all the ports in a small configurable range, until it establishes a secure connection with a BPRserver that identifies itself with a proxy for the same user and with the correct job ID • This approach requires opening a port range on the WN to the CE node • On-going discussion to see if this approach should be replaced with a different one • E.g. the CE could run a standard MyProxy server with a configuration tailored to this limited use case (e.g. the firewall would only need to allow connections from the WN) • Renewed proxies first uploaded into this local MyProxy server • Proxy then downloaded from this MyProxy server to the WN by the job wrapper EGEE08 conference - Istanbul, Sept 22-26, 2008

More Related