1 / 19

Notes from Installing a Mac G5 Cluster at SLAC

Notes from Installing a Mac G5 Cluster at SLAC. Chuck Boeheim SLAC Computing Services. Background. New joint Stanford/SLAC department: Kavli Institute for Particle Astrophysics Choices for compute cluster included joining Opteron purchase, or Mac G5 cluster Astronomers like Macs

toan
Download Presentation

Notes from Installing a Mac G5 Cluster at SLAC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Notes from Installing a Mac G5 Cluster at SLAC Chuck Boeheim SLAC Computing Services

  2. Background • New joint Stanford/SLAC department: Kavli Institute for Particle Astrophysics • Choices for compute cluster included joining Opteron purchase, or Mac G5 cluster • Astronomers like Macs • Compatible with laptops and desktops they were buying

  3. Configuration • 2 Mac G5 file servers • 2 2.5TB Apple XServ Raid units each • 2 Mac G5 interactive servers • 10 Mac G5 compute nodes • No graphics cards • No CD drives • Plus two development servers for us

  4. Physical • Good overall engineering • Good airflow; quiet • Hot swap disks in server nodes • Rails fit standard racks well • Easy to mount

  5. Management • Supports serial console (57,600 baud) • No graphics card in G5 server • Supports serial bios • But must press button to boot in bios • No power management • Supports network install • Initial disk partition and boot a little tricky • Subsequent installs can be fully automated

  6. Installing the first server • “Secret handshake” • Install from CD • Run Server Assistant on another Mac to start install, and again to configure • Run Apple Remote Desktop to get graphic login • Command line alternatives are available, though obscure

  7. The Partitioning Conundrum • System has initial OS with one 250GB partition • Cannot re-partition the boot drive • One solution: • Boot server from CD • Ssh to server • Partition the disk • Diskutil partitionDisk disk0 3 “JournaledHFS+” System 10G “JournaledHFS+” Cache 1G “JournaledHFS+” Work 200G • Netboot may also be possibility • Fastest solution: insert disks in 2nd and 3rd bays of server and format

  8. Reference Server • Install second server • Mac OS X Server 10.3.5 • All patches • Xcode tools • Local directory configuration • AFS • Fink • Shutdown, move hard disk to bay 2 of netboot server

  9. Make a Network Install Master • Netboot serves disk images • Uses DHCP, BSDP1, TFTP, and NFS to serve disk image to target machine • Use Network Image Utility to make image of reference disk • Turn on Netboot in Server Admin • Boot client from network (different “secret handshake”) 1 Boot Server Discovery Protocol

  10. Network Install Sequence DHCP Server Configure with DHCP helper address Router DHCP broadcast DHCP response BSDP broadcast Target Server BSDP response Netboot Server TFTP bootloader NFS disk image

  11. Netboot Issues • Our standard DHCP server worked just fine. • BSDP is an extension of DHCP • We put on same subnet as cluster for simplicity • Should be able to use router helper to put on a different subnet • Network Image Utility had some conflicts

  12. Users: Looks like Unix • Use NIS, LDAP, Kerberos for accounts • Most utilities present: ssh, bash, perl, emacs, X-windows • Fink supplies most gnu tools

  13. Sysadmins: Sorta like Unix • Strong BSD heritage • Some things in different places than Linux • GUI tools for configuring • Most, but not all, can configure a remote server • Serversetup and networksetup commands can configure most settings • Directory Access (setup NIS or LDAP) seems not to be scriptable

  14. Startup Befuddlement • Not inittab or /etc/rc.d based • SystemStarter starts many daemons • Ssh starts out of xinetd • Ypbind started when needed by Directory Services • Watchdog starts server processes • /etc/watchdog.conf • Sorta like inittab • Mac OS X 10.4 (Tiger) is going to introduce yet another way

  15. Authentication Compatibility • Can use NIS, LDAP, Kerberos, Active Directory • Supports PAM, but not everything uses it • LoginWindow (and a few other things) are directly kerberos-aware and don’t use PAM

  16. Infrastructure Fit • Taylor (SLAC’s config tool) ran with little modification • About half the config modules worked with little change • The remainder took completely Mac OS X specific rewrites

  17. Nits • /etc/passwd is there, but not used. Uses netinfo instead for local accounts • Shadow passwords in different place and format than linux, solaris • afs permissions: default is to copy owner mode bits to group and other • Copy file from afs to local, ends up world-readable • Change ‘realmodes’ in /var/db/openafs/etc/config/settings.plist • Mac HFS file system is not case-sensitive: Makefile and makefile are the same!

  18. Conclusions • Good hardware package, but lacking power management • Network install suitable for cluster operation, but still a few wrinkles • Generally good configuration management tools, but some divergence from standard Unix tools

More Related