Booting plan9 from flash
Changes to the Plan 9 loader (9load) and the Plan 9 kernel to allow booting of Geode nodes without any present BIOS. Hardware used: http://www.advantech.com/products/Model_Detail.asp?model_id=1@30541 Configuration: The configuration uses LinuxBIOS as the initial bootloder, LinuxBIOS does some hardware initialization and jumps to 9load, masqueraded as a linux kernel at address 0x800000. 9load has to run here because the kernel needs to load at 0x100000. It can't load at < 0x100000 because that is owned by linuxbios. We had to make some changes to 9load to make this go (like linking it at 0x800000 for example :) 9load then does standard Plan 9 configuration, finds and initializes the present RTL8139 cards and attempts to download a kernel from what it has configured as ether0. If the download is succesfull 9load jumps to the kernel at 0x100020; otherwise it prompts (on the serial port) for a normal 'boot from:' selection. The Plan 9 kernel (a CPU kernel) then does the usual dance, finding devices, etc, and boots from IL without user intervention. Hard drives are not present and not required. Monitors are optional (depending on hardware they may or may not work, vga hardware initialization is not something we do in Plan 9). Rio will, generally, not work. Currently no work is being done in having Rio start in a bios-less configuration. When necessary, one can always attach a serial console to a node booted in the above configuration and will get a prompt. Changes in general libraries: libboot: modified to accept an extra possible location for nvram, namely the machine's CMOS, aka #r/nvram (see devrtc.c) details and explanation follow (the devrtc section below) factotum: needs to be recompiled (uses above library) and relinked with the kernel. auth/keyfs: needs to be recompiled if one's to utilize the node as a cpu server. Changes in 9load: ether.c: modified to probe for ethernet cards rather than expect it to be configured from plan9.ini (there is no plan9.ini). all ether devices found are given irq 9 (the geode hardwires the INTP lines to 10, but for some reason 10 does not work) l.s: the real mode code is commented out (rather, ifdef'd), 9load is entered in protected mode. code is added to load the GDT. some attempts to fix the IRQ routing problem but nothing is tested extensively. we ended up chasing a bug for a long time with: MOV AX, SS and ended up fixing it somewhat miraculously. it never appears in normal bios boots... load.c: hardcoded to boot off 'ether0' as soon as it has finished finding and configuring the (missing) pci devices and hard drives. if the bootp off ether0 fails (e.g. a bad kernel is given) load.c prompts for another boot device. the 'i'm too big' check is commented out. console support is hardwired (no plan9.ini to turn it on). Changes in the kernel: configuration files -- all unnecessary hardware drivers are removed. this is not really needed, but it's done to get the kernel size down to 365Kbytes (compressed). it also facilitates a bit faster booting, though a lack thereof is not really an issue either. devrtc: changed Nvoff to 0x10 from 128. this allows us to store the cpu server's login information in CMOS, rather than requiring a nvram partition on a floppy or a hard drive, or entering the password every time the machine reboots. the byre configuration is as follows: (bank 1): 0x0 to 0xD real time clock 00E diagnostic status byte 00F shutdown status byte 010 to 07F where NVram lives note: normally one would not like to change the nvram on a machine with a normal BIOS -- some important configuration information is left there (fixed disk info, disk drives info and so on) and the BIOS generally will complain and have you reconfigure it or reload defaults. it's a tight spot to store the nvram in, but it'll do for now. this new stuff is used by libboot. main.c: the kbdinit() call in main() is moved earlier to help with debugging. memory.c: debugging turned on. Plan9 misconfigures its main memory when there's no bios present, causing the Proc structure initialization to overwrite the kernel. some values are hardcoded here for 128 megs of ram (irregardless of how much ram there is) until someone spends enough time to fix it. pci.c: cyrix irq initialization is changed, together with the way p9 sets the interrupts. this is untested on other hardware and is a bit tailored towards our current hardware. needs more work before a patch is submitted, ethernet irqs hardwired to 9, others untouched (it may be a geode bug). uarti82590.c: hardcoded console on eia0 (no plan9.ini to configure it) Conclusions: It could be done. More conclusions: As thorough as Plan9 is to create bug-free code, we found that it is sometimes too trusting in the hardware it runs on. Unfortunately it would require some time to make it work on some of the more unfriendly hardware that some of us would like to see it... Timing: Time to boot from power on to a fully functional kernel is ~30 seconds. It could be made faster by not printing extensive debugging information, by not probing for hard drives or network cards we don't have and so on. We think 30 secs is pretty decent and quite satisfactory. Future tests will have to deal with booting large numbers of nodes and finding out whether the bootp server in P9 can handle larger loads.
Last Modified: Jun 09 2002
andrey@lanl.gov