1.16k likes | 1.29k Views
Tim Hockin thockin@google.com OLS 2008. Open Source Bring-Up Tools. What is “bring-up”?. What is “bring-up”?. Process of making a new piece of hardware boot $YOUR_OS. What is “bring-up”?. Process of making a new piece of hardware boot $YOUR_OS properly. What is “bring-up”?.
E N D
Tim Hockin thockin@google.com OLS 2008 Open Source Bring-Up Tools
What is “bring-up”? Process of making a new piece of hardware boot $YOUR_OS
What is “bring-up”? Process of making a new piece of hardware boot $YOUR_OS properly
What is “bring-up”? Process of making a new piece of hardware boot $YOUR_OS properly In our case $YOUR_OS == Linux
Tada! Bootdata ok (command line is root=UUID=260F-12F2 ro google) Linux version 2.6.18.5-gg26 (foobar@corp.google.com) (gcc version 4.1.1) #1 BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009c800 (usable) BIOS-e820: 000000000009c800 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 000000007fbe1800 (usable) BIOS-e820: 000000007fbe1800 - 000000007fbe8c00 (ACPI data) BIOS-e820: 000000007fbe8c00 - 000000007fbe9000 (ACPI NVS) BIOS-e820: 000000007fbe9000 - 00000000a0000000 (reserved) BIOS-e820: 00000000f8000000 - 00000000fa000000 (reserved) BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved) BIOS-e820: 0000000100000000 - 0000000480000000 (usable) ACPI: PM-Timer IO Port: 0x4008 ACPI: Local APIC address 0xfee00000 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) Processor #0 15:1 APIC version 16 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) Processor #1 15:1 APIC version 16 ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-23 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
That sounds easy! There are a few catches...
That sounds easy! There are a few catches Nothing ever works the first time
That sounds easy! There are a few catches Nothing ever works the first time Something will never work at all
That sounds easy! There are a few catches Nothing ever works the first time Something will never work at all At least 1 major issue is guaranteed to show up as soon as your customer is looking
Drummond's Law “The universe hates you.”
Bring-up eats time “Is it a hardware problem or a software problem?”
Bring-up eats time “Or both?”
Drummond's Law in action It only happens on the customer's workload It only happens on < 1% of test runs It goes away when instrumented
What can we do? Gather as much data as we can Streamline testing and debugging Learn about the hardware Build tools!
What can we do? Gather as much data as we can Streamline testing and debugging Learn about the hardware Build free tools!
Problem: Console output Many servers do not have VGA
Problem: Console output Many servers do not have VGA Most servers do have serial ports
Problem: Console output Many servers do not have VGA Most servers do have serial ports Linux supports serial console
Problem: Console output Many servers do not have VGA Most servers do have serial ports Linux supports serial console Many BIOSes do not support serial console
Problem: Console output Many servers do not have VGA Most servers do have serial ports Linux supports serial console Many BIOSes do not support serial console Many BIOSes that support serial console consider it a “premium” feature
SGABIOS Tiny real-mode option ROM
SGABIOS Tiny real-mode option ROM Hooks BIOS console interrupts Int 10h, Int 16h
SGABIOS Tiny real-mode option ROM Hooks BIOS console interrupts Int 10h, Int 16h Provides full console redirection
SGABIOS Tiny real-mode option ROM Hooks BIOS console interrupts Int 10h, Int 16h Provides full console redirection Transparent to most BIOSes and legacy apps
Why is it better? Free, in both senses of the word
Why is it better? Free, in both senses of the word Optimized for slow serial links
Why is it better? Free, in both senses of the word Optimized for slow serial links It works!
Features Passes control to VGA, if present Provides hooks for smarter BIOS or EFI logging Supports up to 255x255 consoles Recent-write caching
How can I use it? Talk to your platform or BIOS vendor Look for BIOS-editing tools for $YOUR_BIOS Load it as an option-ROM on a card
Hypothetical Your system boots, but the BIOS did not enable FeatureX
Hypothetical • Your system boots, but the BIOS did not enable FeatureX • You really, really want FeatureX
Hypothetical • Your system boots, but the BIOS did not enable FeatureX • You really, really want FeatureX • CPUCorp tells you that FeatureX is enabled by bit 20 of MSR 0x12345678
Hypothetical • Your system boots, but the BIOS did not enable FeatureX • You really, really want FeatureX • CPUCorp tells you that FeatureX is enabled by bit 20 of MSR 0x12345678 • How do you do it?
You could... Get a new BIOS from Mobo Inc. Slow turnaround time High risk Don't you want to test it first?
You could... Tweak and rebuild a custom kernel Where does a hack like this go? Pretty heavy handed
You could... Write a C/Perl/Python program A much better answer Doesn't scale well for frequent use cases
Enter iotools A simple suite of tools to provide access to hardware registers