Installing GRUB on FreeBSD with GPT partitions
NOTE:
These changes were recently incorporated into the ports tree, so it should no longer be necessary to follow these steps. Simply upgrade your ports via portsnap and then build as usual.
GRUB
is the GRand Unified Bootloader. (link) The current version (called grub2) supports gpt partitions but it doesn't quite work with FreeBSD's gpt partitions. The FreeBSD port of grub2 (sysutils/grub2) also contains a bad patch that is likely to cause installation problems for most people. I spent a couple days debugging and patching the FreeBSD grub port and now I want to share what I learned. These instructions will work for FreeBSD and PC-BSD, but PC-BSD users will have to install the ports collection first. The really impatient can skip to installing
Why?
Why do I even want to use GRUB? FreeBSD has a very basic bootloader but it doesn't always meet my needs. I needed to be able to triple-boot PC-BSD, Ubuntu, and Windows 7 for a recent project. The FreeBSD bootloader couldn't see the Ubuntu partition, so I started looking at grub2. And even if the FreeBSD bootloader could see my Ubuntu partiton, GRUB is still a much better bootloader with its support for loadable partition and filesystem modules and its ability to drop to a command-line and tweak kernel options.
What about gpt? GUID Partition Table is a new partitioning scheme set to replace classic MBR partitions. It supports up to 128 partitions of up to 9.4ZB in size, and it provides a special boot partition for embedding the bootloader second stage code vs the classic method of utilizing a limited bit of dead space that follows the MBR. This extra space is especially valuable when booting to ZFS root partitions. As of FreeBSD 8.1, sysinstall still creates MBR partitions. PC-BSD's pc-sysinstall (which was recently merged into FreeBSD-HEAD and may soon replace sysinstall), however, creates GPT partitons by default and you must use a GPT partition in order to boot straight to a ZFS filesystem without a UFS /boot partition. Like it or not, gpt is the way of the future.
How the port is currently broken
As of this time, the grub2 port is broken in a couple of ways. It compiles just fine, but it won't install correctly. Let's take a look at what happens:
[root@freebsd-8797 /usr/ports/sysutils/grub2]# make install
.
.
.
#############################################################
To install GRUB on the master boot record of your hard drive
use 'grub-install <drive-to-install>' command.
A typical menu entry in /boot/grub/grub.cfg for FreeBSD:
menuentry "FreeBSD" {
set root(hd0,1,a)
kfreebsd /boot/loader
}
Or use grub-mkconfig to create the config file.
#############################################################
install-info --quiet /usr/local/info/grub.info /usr/local/info/dir
===> Registering installation for grub2-1.98
[root@freebsd-8797 /usr/ports/sysutils/grub2]#
Compiles just fine, so we go to install to /dev/ad0:
[root@freebsd-8797 /usr/ports/sysutils/grub2]# grub-install /dev/ad0
/usr/local/sbin/grub-probe: error: cannot find a device for /boot/grub (is /dev mounted?).
No path or device is specified.
Try `/usr/local/sbin/grub-probe --help' for more information.
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
[root@freebsd-8797 /usr/ports/sysutils/grub2]#
Hmmm… grub-probe is complaining about not being able to find the device associated with the directory /boot/grub. Why could this be? A look inside the function grub_guess_root_device in /usr/ports/sysutils/grub2/work/grub-1.98/util/getroot.c reveals the problem: this function is supposed to call stat() on the directory to find its parent's Device ID and then recursively search /dev for a character device with the same Device ID. However, instead of searching for the folder's parent ID, the function searches for the folder's own ID instead! That doesn't make sense. Of course searching for /boot/grub in /dev isn't going to yield a result.
Is this a bug a grub itself? Short answer: nope. This bug is introduced by a FreeBSD patch in /usr/ports/sysutils/grub2/files/patch-util-getroot.c. Why was this bad patch introduced? I can't be sure but I think I have a good guess: Somebody was doing a manual install of grub instead of relying on grub-install, and that person must have been invoking grub-probe on a device without passing the -d argument. He then made this patch which effectively turns the -d option on permanently.
OK, so let's remove this patch, rebuild and reinstall grub2, and try to install again:
[root@freebsd-8797 /usr/ports/sysutils/grub2]# rm files/patch-util-getroot.c
[root@freebsd-8797 /usr/ports/sysutils/grub2]# make deinstall clean reinstall
===> Deinstalling for sysutils/grub2
===> Deinstalling grub2-1.98
===> Cleaning for grub2-1.98
.
.
.
#############################################################
To install GRUB on the master boot record of your hard drive
use 'grub-install <drive-to-install>' command.
A typical menu entry in /boot/grub/grub.cfg for FreeBSD:
menuentry "FreeBSD" {
set root(hd0,1,a)
kfreebsd /boot/loader
}
Or use grub-mkconfig to create the config file.
#############################################################
install-info --quiet /usr/local/info/grub.info /usr/local/info/dir
===> Registering installation for grub2-1.98
[root@freebsd-8797 /usr/ports/sysutils/grub2]# grub-install /dev/ad0
/usr/local/sbin/grub-setup: warn: This GPT partition label has no BIOS Boot Partition; embedding won't be possible!.
/usr/local/sbin/grub-setup: warn: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and its use is discouraged..
/usr/local/sbin/grub-setup: error: if you really want blocklists, use --force.
OK, that's progress! grub-probe isn't throwing any errors anymore and grub recognizes this is a GPT partition. But what is this about embedding, blocklists, and a BIOS Boot Partition??
GRUB needs to find a place to install the stage2 boot code to. Ideally this will be a contiguous block of sectors in an area of the disk where the bootloader will be protected–i.e. the boot sector of an MBR disk or the dedicated boot partition of a GPT disk. In this case, GRUB is looking for a GPT partiton of type bios-boot. Where's ours?
[root@freebsd-8797 /usr/ports/sysutils/grub2]# gpart show ad0
=> 34 21180349 ad0 GPT (10G)
34 128 1 freebsd-boot (64K)
162 1048576 2 freebsd-ufs (512M)
1048738 1048576 3 freebsd-swap (512M)
2097314 2097152 4 freebsd-ufs (1.0G)
4194466 16985917 5 freebsd-ufs (8.1G)
Oh there it is, at index 1! But it's type freebsd-boot instead of bios-boot. Now at this point we have two options: Change the partiton type to bios-boot or modify GRUB to recognize freebsd-boot partitions as a valid space to embed to.
Option 1)
gpart doesn't understand the symbolic name bios-boot so you have to tell it the GUID explicitly:
gpart modify -i 1 -t \!21686148-6449-6E6F-744E-656564454649 ad0
Option 2)
I've made two patches that add support for freebsd-boot and solaris-boot partitions to GRUB (patch-include-grub-gpt_partition.h and patch-util-i386-pc-grub-setup.c. Both need to be applied. Copy them to /usr/ports/sysutils/grub2/files and repeat the "make deinstall clean reinstall" command as above.
Installing and Configuring GRUB
Before continuing, be sure to have installed a fixed copy of the grub2 port as explained above (delete the bad patch, copy my two patches into the /usr/ports/sysutils/grub2/files/ directory, cd /usr/ports/sysutils/grub2 && make deinstall clean install). Here's what it might look like:
[grub2]# rm files/patch-util-getroot.c
[grub2]# cp /root/patch-include-grub-gpt_partition.h files
[grub2]# cp /root/patch-util-i386-pc-grub-setup.c files
[grub2]# make deinstall clean install
Now the GRUB userland utilities are installed, but we still need to build the bootloader and embed it into the GPT partition table. We also need to build the part_gpt module into the bootloader:
freebsd-8797# grub-install --modules=part_gpt /dev/ad0
Installation finished. No error reported.
freebsd-8797#
Just change /dev/ad0 to the path of your boot drive, whatever it happens to be.
Now GRUB is installed but the boot menu is not configured. If you were to reboot, you would get a grub prompt waiting for you to tell it how to locate and boot a kernel. GRUB2 is a little different from GRUB1. The boot menu is meant to be generated from config files rather than edited by hand. These config files live in /usr/local/etc/grub.d/
There are two ways to configure GRUB to load FreeBSD: Have GRUB set all the runtime kernel options (including what loadable modules to preload) and invoke the FreeBSD kernel directly, or have GRUB simply invoke /boot/loader and let the loader set all the kernel options and loadable modules. I really prefer the second option for these reasons: It's easier to configure GRUB this way and it doesn't break the /boot/loader.conf functionality. It's better to let BSD be configured and loaded the way it was intended to be rather than put that burden on GRUB and leave a bunch of no longer functional config files in /boot/ to confuse yourself with later.
The default config files will generate a GRUB menu that loads the FreeBSD kernel directly, instead of calling /boot/loader. Run grub-mkconfig without any options and have a look at the 10_kfreebsd section towards the end. It should similar to this:
freebsd-8797# grub-mkconfig
.
.
.
### BEGIN /usr/local/etc/grub.d/10_kfreebsd ###
menuentry "FreeBSD, with kFreeBSD kernel" --class freebsd --class bsd --class os {
insmod ufs2
set root='(hd0,2)'
search --no-floppy --fs-uuid --set 4cb05ace512b3d97
echo Loading kernel of FreeBSD kernel ...
kfreebsd /boot/kernel/kernel
kfreebsd_loadenv /boot/device.hints
kfreebsd_module_elf /boot/kernel/acpi.ko
set kFreeBSD.vfs.root.mountfrom=ufs:/dev/ad0p2
set kFreeBSD.vfs.root.mountfrom.options=rw
}
### END /usr/local/etc/grub.d/10_kfreebsd ###
.
.
.
freebsd-8797#
If that looks good and you want to load the kernel directly, then just run
grub-mkconfig -o /boot/grub/grub.cfg
and reboot. (see the note below about the "set root" command before rebooting)
If you want to be able to invoke the FreeBSD loader, then copy/paste the 10_kfreebsd section of the output of grub-mkconfig into /usr/local/etc/grub.d/40_custom and edit it to look like this:
menuentry "FreeBSD /boot/loader" --class freebsd --class bsd --class os {
insmod ufs2
set root='(hd0,2)'
search --no-floppy --fs-uuid --set 4cb05ace512b3d97
kfreebsd /boot/loader
}
Then run
grub-mkconfig -o /boot/grub/grub.cfg
and reboot. (see the note below about the "set root" command before rebooting)
A note about "set root" and "search"
The values of "set root=" and "search" in the grub.cfg file are critical. Do not just copy and paste mine. If they are set wrong, GRUB won't be able to find the kernel and thus won't boot the system automatically. grub-mkconfig will probably set the "set root" parameter wrong, and the "search" parameter right.
The "set root=" command needs to put into GRUB's language for naming devices. A value of "/dev/ad0p2" is wrong. For a GPT partition, it should be in the form of '(hdX,Y)' where X is the number of the hard drive and Y is the index of the root partition. The commands grub-mkdevicemap and gpart show can be used to determine this information:
freebsd-8797# grub-mkdevicemap -m -
(hd0) /dev/ad0
freebsd-8797# gpart show ad0
=> 34 21180349 ad0 GPT (10G)
34 128 1 freebsd-boot (64K)
162 1048576 2 freebsd-ufs (512M)
1048738 1048576 3 freebsd-swap (512M)
2097314 2097152 4 freebsd-ufs (1.0G)
4194466 16985917 5 freebsd-ufs (8.1G)
freebsd-8797#
The "search" command serves a similar purpose to the "set root=" command, but it works differently. Instead of explicitly stating which drive and partition to set as the root, the search command searches all hard drives for a partition labeled with a unique identifier. It's supposed to be a UUID but FreeBSD doesn't seem to be generating the right labels at this time. In any case, grub-probe will tell you what GRUB thinks the UUID is for a partition:
freebsd-8797# grub-probe -d --target=fs_uuid /dev/ad0p2
4cb05ace512b3d97
What about ZFS?
GRUB2 has an experimental ZFS module that is reported to boot up to ZFS v28. Unfortunately, the module is a moving target with no stable branch. What's worse is that it requires Bazaar to obtain and Ruby and some additional patches in order to compile. I may try to create a patch that adds ZFS support to the existing grub2 port, but that will be the subject of a future entry.
FreeBSD 8.1 amd64AD4S2grub-probe still fails
Tony, I just tested it on my “8.1-RELEASE amd64” system and it worked OK.
First of all, did you update your ports tree before building? It should work without any modifications.
Next you need to use the correct syntax depending on what you’re asking grub-probe to do. For example:
grub-probe –device /dev/ad4s1f
ufs2
grub-probe –device –target=fs_uuid /dev/ad4s1f
46583c3c6eb751b3
grub-probe –target=device /mnt/root
/dev/ad4s1a
grub-probe /mnt/usr
ufs2
When passing a device name, specify –device on the command line. Otherwise grub-probe assumes you are passing a directory.
the first slice works great, its the second slice as windows 7 is installed on the first
Hey, I can’t view your site properly within Opera, I actually hope you look into fixing this.
I viewed it in Opera 10.63 and it looks fine. Perhaps you need to upgrade?
I will have to wait till the quarter is over to try it with the –device option.
Thanks!Good document ! Would you be kind to tell me what should I do to go back to my original boot state if I failed (for example I forgot load gpt when runing grub-install).
I suppose that depends on what your original boot state was 🙂 If you want to restore the MBR for nont-gpt drives, you can use
fdisk -B -b /boot/boot0
If you want to reinstall the FreeBSD bootloader to a gpt boot partition,
gpart bootcode -b /boot/pmbr -p /boot/gptboot -i
Thanks so much for this! I’ve found it incredibly helpful.
Hi,
The last time I tried grub2 on FreeBSD there were some problems arising out of differences b//w the POSIX(?) “sed” available on BSD vs the GNU “sed” GRUB 2.00 is more commonly used with.
Specifically, no usable grub.cfg was generated.
Regards.
Ebon Talifarro…
Installing GRUB on FreeBSD with GPT partitions…
[…] to Rick for his article that covered how to set the bios-boot flag with gpart under FreeBSD. This entry was posted in Uncategorized on January 26, 2015 by […]
[…] to Rick for his article that covered how to set the bios-boot flag with gpart under FreeBSD. Posted on January 26, 2015April 24, 2015Author Jordan BeaverCategories Arch […]
I get errors =>
syntax error.
error: Incorrect command.
error: syntax error.
Syntax error at line 67
Syntax errors are detected in generated GRUB config file.
Ensure that there are no errors in /etc/default/grub
and /etc/grub.d/* files or please file a bug report with
What to do? What is missing?