280 likes | 463 Views
Cluster Boot Issues. Shawn Marriott COSC 3P93. What is a Cluster?. A cluster is a collection of individual autonomous nodes working in concert to create the illusion of a single system. The trouble with nodes.
E N D
Cluster Boot Issues Shawn Marriott COSC 3P93
What is a Cluster? A cluster is a collection of individual autonomous nodes working in concert to create the illusion of a single system.
The trouble with nodes If you must consider the state, and configuration of each node in a cluster the illusion of the single system is lost. As the size of a cluster grows the management of individual nodes becomes tedious, and error prone.
A Paradox How does a node get the information it needs to boot, if it needs to boot to get the information it needs ?
Booting a computer Part of the startup procedure for most computers is to initialize a bootstrap, which in turn starts the operating system. Where to find the bootstrap is generally configure on a per machine basis in the bios. Common places that are searched for the bootstrap are the hard drive(s), optical drives, solid state devices, and floppies.
How do you boot a cluster? Or at least convince the nodes to boot…….
A Sub Ideal Solution: Manually configure every node in the cluster…
An ideal solution: • Unpack the computer. • Plug in the computer. • Turn on the computer. • Done.
How could this be done? If the nodes in our cluster are computers connected to a network. Then when a computer starts up have it use the network to broadcast information about itself to a server, and have the server provide configuration information and a boot strap.
In the beginning…. There was the Reverse Address Resolution Protocol (RARP). RFC 903, 1984.
Next try…… Bootstrap protocol (BOOTP). RFC 951, 1985
Sample BootP config node01:ht=ether:ha=080009030166:ip=15.19.8.2:sm=255.255.248.0:gw=15.19.8.1:bf=/bootloader node02:ht=ether:ha=080009030176:ip=15.19.8.3:sm=255.255.248.0:gw=15.19.8.1:bf=/bootloader node03:ht=ether:ha=080009030186:ip=15.19.8.4:sm=255.255.248.0:gw=15.19.8.1:bf=/bootloader node04:ht=ether:ha=080009030196:ip=15.19.8.5:sm=255.255.248.0:gw=15.19.8.1:bf=/bootloader
Another go around Dynamic Host Configuration Protocol (DHCP). RFC 1531, 1993.
Sample DHCP Configuration subnet 192.168.0.0 netmask 255.255.255.0 { range 192.168.0.10 192.168.0.49; option subnet-mask 255.255.255.0; option broadcast-address 192.168.0.255; option routers 192.168.0.1; filename "pxelinux.0"; next-server 192.168.0.100; }
One more try? In 1999 Intel releases the Wired for Management Framework (WMF) The framework was mostly ignored, but two interesting parts of the specification endure. Preboot Execution Environment (PXE). Boot Integrity Services (BIS).
What is PXE? PXE is a client designed to work with DHCP and TFTP to retrieve boot strap information, it also defines APIs to allow a loaded boot strap to query a local host for additional information
What is BIS? BIS enables a PXE client to examine a digitally signed boot image. This provides a mechanisms to verify the integrity of a supplied boot strap image.
How is PXE implemented? If you want to use a network to provide configuration information for a node, then you better have a network card (NIC). Since you must have a network card, it seems reasonable to put the PXE client on the network card, and have the bios treat network cards as a bootable device.
What is a Network Bootstrap Program(NBP)? • NBP is a binary executable file, specific to a give CPU’s architecture. • They are small, usually less than 512KB in size. • What an NBP does is up to whoever creates it. The PXE specification does not go into any detail on NBP.
What is involved in setting up a PXE environment? • A Node with a PXE client set as the boot device. • A DHCP server • TFTP server
Sample DHCP Configuration subnet 192.168.0.0 netmask 255.255.255.0 { range 192.168.0.10 192.168.0.49; option subnet-mask 255.255.255.0; option broadcast-address 192.168.0.255; option routers 192.168.0.1; }
Sample proxy DHCP config subnet 192.168.0.0 255.255.0.0 { vendor pxe { bootstrapserver 192.168.0.100 # TFTP server ip address. #Type, SystemArch, MajorVers pxebootfile 1 2 1 window.one 1 0 pxebootfile 2 2 1 linux.one 2 3 pxebootfile 1 2 1 hello.one 3 4 client 6 10005a8ad14d { pxebootfile 1 2 1 aix.one 5 6 pxebootfile 2 2 1 window.one 6 7 } } }
What PXE does right • It is a standard feature on modern NICS, and mother boards with integrated NICS • A PXE client knows the architecture of the node it is running on, and can request an appropriate NBP • It builds upon existing technologies (DHCP, TFTP) • It has extensions to authenticate the boot server, and verify the integrity of a downloaded NBP.
The problems with PXE • PXE has limited knowledge of it’s host. • PXE relies on DHCP, which is inefficient on large networks • PXE relies on TFTP, which is impractical for large files, or many concurrent file transfers • If you need different NBPs for different nodes you must uniquely identify each node(By MAC address) and group them accordingly in DHCP or Use a DHCP proxy server and separate the management of network address and images
References Thomas L. Sterling , 1998. Beowulf Cluster Computing with Linux. Cambridge, Massachusetts: The MIT Press Intel Corporation, 1998. Boot Integrity Services Application Programming. Version 1.0. ftp://download.intel.com/design/archives/wfm/downloads/bisspec.pdf Intel Corporation, 1999. Preboot Execution Environment (PXE) Specification. Version 2.1. http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf http://www.ietf.org/rfc/rfc903.txt http://www.ietf.org/rfc/rfc951.txt http://www.ietf.org/rfc/rfc1531.txt http://www.beowulf.org http://en.wikipedia.org/wiki/Preboot_Execution_Environment http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp