1 / 38

An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group

An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group dpturner@nersc.gov 510-486-4027. Topics. Definitions Certificates Acquiring Using Tools and Services Current Soon Future Resources. What is “The Grid”?. The Globus view

sasha-whitaker
Download Presentation

An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group dpturner@nersc.gov 510-486-4027

  2. Topics • Definitions • Certificates • Acquiring • Using • Tools and Services • Current • Soon • Future • Resources

  3. What is “The Grid”? • The Globus view • Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.(http://www.globus.org) • The NERSC implementation • File transfer and data management • Remote job submission • Portals

  4. What is “Globus”? • The Globus Alliance is a research and development project focused on enabling the application of Grid concepts to scientific and engineering computing. • Argonne National Laboratory’s Mathematics and Computer Science Division • University of Chicago’s Distributed Systems Laboratory • University of Southern California’s Information Sciences Institute • University of Edinburgh • Swedish Center for Parallel Computers

  5. What is “Globus”, Really? • The Globus Toolkit is a middleware package from the Globus Alliance • De facto standard for Grid “platform” • Client software • Command-line tools • Application development libraries • Installed on seaborg, escher, newton, and PDSF • Server software • Installed on seaborg, escher, newton, PDSF, HPSS, and web servers • Version 2.4.3 supported on most systems (PDSF is between 2.2.4 and 2.4.3) • Available for “most” desktop systems

  6. Globus Software Tools • Client software • Certificate management tools • grid-cert-info, grid-proxy-init, grid-proxy-info, grid-proxy-destroy • File transfer commands • globus-url-copy, uberftp, pftp_gsi • Server software • GridFTP, Gatekeeper, MyProxy • “Portals” • Combine characteristics of client and server

  7. Using Globus at NERSC • To use client software: % module load globus % echo $GLOBUS_LOCATION /usr/common/globus/gt243 • GridFTP servers run on: seaborg-g1.nersc.gov garchive.nersc.gov escher.nersc.gov pdsfgrid[1-3].nersc.gov newton0[1-4].eth1.nersc.gov

  8. Security Definitions • Authentication • Verifying that someone is who they claim to be • Required to run client software • Based on certificates • Authorization • Determining if an authenticated person has access to a particular resource or service • Typically implemented with a grid-mapfile • Globus security model based on Grid Security Infrastructure (GSI)

  9. Certificate Definitions • A document attesting to the truth of certain stated facts. • A document that is used to certify that a user or organization is who they say they are. They contain information about who it belongs to, who it was issued by, expiry date and information that can be used to check out the contents of the certificate. • Implements ISO X.509 • Public Key Infrastructure (PKI)

  10. Certificate Characteristics • Opaque • Issued by recognized Certificate Authority (CA) • Stored inside a web browser • Netscape/Mozilla Edit->Preferences->Privacy&Security->Certificates->Manage Certificates • Internet Explorer Tools->Internet Options->Content->Certificates • Exported from browser to disk file • Globus tools to query contents of file • Can be exported from browser into file • Move from system to system

  11. Types of Certificates • Personal certificate • a.k.a. client certificate • Host certificate • a.k.a. server certificate • CA certificate • a.k.a. root certificate

  12. Certificates at NERSC • Acquired from a CA • Local institution • DOEGrids (administered by ESnet) • NERSC • Enter Distinguished Name (DN) of Subject and Issuer into NERSC Information Management (NIM) system • Required for authorization (to access servers) • Propagates to local grid-mapfiles • http://nim.nersc.gov

  13. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  14. Getting a DOEGrids Certificate • To allow your browser to trust DOEGrids: http://www.doegrids.org/pages/How-To-Import.html • Follow directions, including restarting browser • To acquire personal certificate: https://pki1.doegrids.org/ • Provide all requested fields, click “Submit” • Wait for email with further instructions • Click on “Import Your Certificate” • Personal Certificate now stored in browser • Valid for 1 year

  15. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  16. Got a Cert; Now What? • Put certificate into “p12” file • Netscape Navigator Edit->Preferences->Privacy&Security->Certificates->Manage Certificates • Select certificate, click “Backup” • Provide file name, password(s) • Internet Explorer Tools->Internet Options->Content->Certificates • Select certificate, click “Export” • Provide file name, password(s) • File name extension: .p12 or .pfx • PROTECT THIS FILE!

  17. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  18. Have “p12” File; Now What? • If necessary, copy file to machine with globus installed • seaborg, escher, newton, pdsf • Extract certificate (which contains public key) openssl pkcs12 -in YourCert.p12-clcerts -nokeys -out ~/.globus/usercert.pem • Extract private key openssl pkcs12 -in YourCert.p12-nocerts -out ~/.globus/userkey.pem • Set permissions to owner read/write chmod go-rwx ~/.globus/user*.pem • Protect these files!

  19. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  20. Querying Your Certificate % grid-cert-info Certificate: Data: Version: 3 (0x2) Serial Number: 1918 (0x77e) Signature Algorithm: sha1WithRSAEncryption Issuer: DC=org, DC=DOEGrids, OU=Certificate Authorities, CN=DOEGrids CA 1 Validity Not Before: Jun 17 17:32:11 2004 GMT Not After : Jun 17 17:32:11 2005 GMT Subject: DC=org, DC=doegrids, OU=People, CN=David Turner 460392 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:f5:b9:2a:d1:e3:89:cb:49:6d:99:00:93:b1:fe: 4a:56:9e:c4:a2:59:00:3e:0f:02:56:c2:5c:2d:ce: bb:4f:f9:44:89:01:9b:42:4b:ad:8f:25:53:07:c1: 8e:88:c2:3d:0c:1e:bd:6b:5f:12:1d:b7:57:bd:df: 22:6e:50:88:63:c5:59:d6:e5:65:9e:34:cd:c9:dd: 00:2e:bd:ef:8d:65:03:20:d0:dc:86:33:b3:d9:2d: 15:cd:20:d3:14:d3:63:05:21:82:e3:ab:b5:12:f4: 2e:45:5b:50:72:a3:71:d6:10:cb:e3:8d:ec:1a:e0: 3c:16:9c:f1:71:4e:45:8c:04:49:9c:4d:eb:be:79: f6:0c:76:cb:66:48:54:a6:d9:94:3c:54:82:c8:46: 72:74:36:99:c1:e4:45:c9:c5:f8:d4:74:a1:6b:fe: ff:2c:ad:43:13:b0:47:76:5d:7f:65:65:b0:b9:02: e2:3f:53:5d:77:cb:30:50:42:1a:dd:3e:df:a3:2f: 49:27:66:d9:63:bf:c5:56:07:c4:8d:de:78:c0:08: fd:f8:5f:b4:ae:5d:61:e4:6d:52:86:d3:cb:d6:76: 28:95:8e:1c:b7:3f:d6:63:87:82:4b:c7:d1:b6:18: b3:76:99:0c:a9:13:d6:2f:20:56:5d:56:a8:dd:87: f6:fd Exponent: 65537 (0x10001) X509v3 extensions: Netscape Cert Type: SSL Client, SSL Server, S/MIME X509v3 Key Usage: critical Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment X509v3 Authority Key Identifier: keyid:CA:19:1D:12:8E:6E:A4:38:5D:42:D4:31:0E:08:DB:D9:8D:17:0D:5D X509v3 Subject Alternative Name: email:dpturner@lbl.gov Signature Algorithm: sha1WithRSAEncryption 16:80:67:ef:45:d3:0d:c8:15:86:64:16:ca:b1:0e:64:e7:46: b6:da:b7:cc:e5:3c:54:3f:c6:7d:62:f9:b4:02:b5:e4:3e:74: ed:5f:61:91:98:df:1f:45:cb:16:2a:17:48:28:8e:56:11:e6: 52:93:73:71:44:bf:7d:24:f8:c8:69:15:e2:7b:dd:38:68:15: a7:de:e4:d6:7a:c7:da:41:c5:3e:dd:a9:b2:15:2b:74:6f:87: 32:4a:e5:38:db:fc:e6:e4:49:b6:95:25:c6:f8:77:32:e2:b2: b7:29:46:21:f9:7d:93:11:3e:97:b7:6c:5b:ac:a1:a8:b0:28: eb:44:99:10:d7:16:6a:39:6a:dc:9e:ef:94:47:2f:ab:1b:02: be:94:96:a8:3a:83:5c:56:58:13:c7:d6:ae:4b:25:49:d1:aa: 20:c4:d8:f2:0a:01:40:67:18:11:11:d3:5b:69:43:4f:5c:61: 71:45:2c:06:fb:95:ae:2d:b9:f7:d1:0f:eb:5b:91:73:7d:8e: 4b:8b:f9:a6:e4:78:c0:51:14:a5:f6:f6:00:db:00:91:75:f9: b5:42:5b:43:46:1a:ca:36:25:21:73:b0:1b:21:bb:35:6f:f6: 3e:21:df:c6:50:96:5b:ac:0f:a5:48:3b:88:68:c6:eb:8d:0a: a5:fe:97:b9

  21. Certificate Information in NIM • Login in to NIM • Click “Grid Certificates” tab in lower frame

  22. Informing NIM of Your Cert % grid-cert-info -subject /DC=org/DC=doegrids/OU=People/CN=David Turner 460392 % grid-cert-info -issuer /DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1 • In NIM, click “Add existing certificate to NIM” • Cut-and-paste Subject and Issuer DN • Click “Add Certificate”

  23. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  24. Have “pem” Files; Now What? • Create “proxy certificate” • Usually has a limited lifetime • Default 12 hours • Maximum 1 year • Creates $HOME/.globus/x509up_u12345 grid-proxy-init [-valid h:m]

  25. DOEGrids Certificate Process usercert.pem Proxy cert CA Browser “p12” file x509up_u12345 userkey.pem NIM Subject DN Issuer DN

  26. Querying Your Proxy Cert % grid-proxy-info subject : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392/CN=proxy issuer : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392 identity : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392 type : full legacy globus proxy strength : 512 bits path : /usr/common/homes/d/dpturner/.globus/x509up_u17931 timeleft : 11:58:33

  27. Proxy Certificate Issues • Managing Grid credential files (“pem” files containing certificate/public key and private key) is a nuisance. • Security • Keys can be stolen if account compromised. • Copies on multiple machines increase exposure. • One solution: MyProxy server

  28. Using MyProxy • To place Grid credentials into MyProxy server: myproxy-init [-t hours] • Default lifetime on server is one week. • Can now remove “pem” files from local file system. • To retrieve proxy delegation: myproxy-get-delegation • To query credentials on server: myproxy-info • To remove credentials from server: myproxy-destroy

  29. Using MyProxy Delegation • A MyProxy delegation is another form of a proxy certificate: grid-proxy-info grid-proxy-destroy • Once delegation in place, user can run Globus client software.

  30. Isn’t There a Simpler Way? • NERSC-managed certificates • Issued by “NERSC CA” (actually “DOEGrids CA 2”) • Created entirely in NIM • Delivered directly to MyProxy server • One-year lifetime • User never handles “p12” file or “pem” files • Not widely trusted (YET!)

  31. Getting a NERSC Certificate • Login to NIM • Click “Grid Certificates” tab • Click “Create a new NERSC-managed certificate in NIM” link

  32. But What Does Globus Do? • GridFTP file transfer commands • globus-url-copy globus-url-copy -nodcau gsiftp://seaborg-g1.nersc.gov/path1/file1gsiftp://garchive.nersc.gov/path2/file2 • uberftp • Complete GridFTP interactive client • Third-party transfers using “lopen” command uberftp -a GSI -P 2811 -H garchive.nersc.gov • pftp_gsi • NERSC-developed pftp client with GSI authentication pftp_gsi garchive.nersc.gov

  33. Other File Transfer Tools • GridFTP API • Transfer data directly from batch jobs to visualization system at PPPL • Grid File Yanker (GFY) portal • Reliable transfer • VisPortal

  34. Globus Future at NERSC • Remote job submission and monitoring • NERSC Portal • Continuing evolution of authentication and authorization • GridLogon

  35. Resources • Web http://www.nersc.gov/ http://www.nersc.gov/nusers/help/access/globus.php http://nim.nersc.gov/ http://www.doegrids.org/ http://www.globus.org/ http://dims.ncsa.uiuc.edu/set/uberftp/ http://www.google.com/ • Human • consult@nersc.gov • 1-800-66-NERSC, option 3 • 1-510-486-8600, option 3

  36. The End This page intentionally left blank.

  37. Still The End This page intentionally left blank.

  38. Still The End, Really This page intentionally left blank.

More Related