NOTE:  

These changes were recently incorporated into the ports tree, so it should no longer be necessary to follow these steps.  Simply upgrade your ports via portsnap and then build as usual.

GRUB

is the GRand Unified Bootloader.  (link)  The current version (called grub2) supports gpt partitions but it doesn't quite work with FreeBSD's gpt partitions.  The FreeBSD port of grub2 (sysutils/grub2) also contains a bad patch that is likely to cause installation problems for most people.  I spent a couple days debugging and patching the FreeBSD grub port and now I want to share what I learned.  These instructions will work for FreeBSD and PC-BSD, but PC-BSD users will have to install the ports collection first.  The really impatient can skip to installing

Why?

Why do I even want to use GRUB?  FreeBSD has a very basic bootloader but it doesn't always meet my needs.  I needed to be able to triple-boot PC-BSD, Ubuntu, and Windows 7 for a recent project.  The FreeBSD bootloader couldn't see the Ubuntu partition, so I started looking at grub2.  And even if the FreeBSD bootloader could see my Ubuntu partiton, GRUB is still a much better bootloader with its support for loadable partition and filesystem modules and its ability to drop to a command-line and tweak kernel options.

What about gpt?  GUID Partition Table is a new partitioning scheme set to replace classic MBR partitions.  It supports up to 128 partitions of up to 9.4ZB in size, and it provides a special boot partition for embedding the bootloader second stage code vs the classic method of utilizing a limited bit of dead space that follows the MBR.  This extra space is especially valuable when booting to ZFS root partitions.  As of FreeBSD 8.1, sysinstall still creates MBR partitions.  PC-BSD's pc-sysinstall (which was recently merged into FreeBSD-HEAD and may soon replace sysinstall), however, creates GPT partitons by default and you must use a GPT partition in order to boot straight to a ZFS filesystem without a UFS /boot partition.  Like it or not, gpt is the way of the future.

How the port is currently broken

As of this time, the grub2 port is broken in a couple of ways.  It compiles just fine, but it won't install correctly.  Let's take a look at what happens:

[root@freebsd-8797 /usr/ports/sysutils/grub2]# make install
.
.
.
#############################################################
To install GRUB on the master boot record of your hard drive
use 'grub-install <drive-to-install>' command.

A typical menu entry in /boot/grub/grub.cfg for FreeBSD:
menuentry "FreeBSD" {
        set root(hd0,1,a)
        kfreebsd /boot/loader
}
Or use grub-mkconfig to create the config file.
#############################################################
install-info --quiet /usr/local/info/grub.info /usr/local/info/dir
===>   Registering installation for grub2-1.98
[root@freebsd-8797 /usr/ports/sysutils/grub2]#

Compiles just fine, so we go to install to /dev/ad0:

[root@freebsd-8797 /usr/ports/sysutils/grub2]# grub-install /dev/ad0
/usr/local/sbin/grub-probe: error: cannot find a device for /boot/grub (is /dev mounted?).
No path or device is specified.
Try `/usr/local/sbin/grub-probe --help' for more information.
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
[root@freebsd-8797 /usr/ports/sysutils/grub2]#  

Hmmm…   grub-probe is complaining about not being able to find the device associated with the directory /boot/grub.  Why could this be?  A look inside the function grub_guess_root_device in /usr/ports/sysutils/grub2/work/grub-1.98/util/getroot.c reveals the problem:  this function is supposed to call stat() on the directory to find its parent's Device ID and then recursively search /dev for a character device with the same Device ID.  However, instead of searching for the folder's parent ID, the function searches for the folder's own ID instead!  That doesn't make sense.  Of course searching for /boot/grub in /dev isn't going to yield a result. 

Is this a bug a grub itself?  Short answer: nope.  This bug is introduced by a FreeBSD patch in /usr/ports/sysutils/grub2/files/patch-util-getroot.c.  Why was this bad patch introduced?  I can't be sure but I think I have a good guess:  Somebody was doing a manual install of grub instead of relying on grub-install, and that person must have been invoking grub-probe on a device without passing the -d argument.  He then made this patch which effectively turns the -d option on permanently. 

OK, so let's remove this patch, rebuild and reinstall grub2, and try to install again:

[root@freebsd-8797 /usr/ports/sysutils/grub2]# rm files/patch-util-getroot.c
[root@freebsd-8797 /usr/ports/sysutils/grub2]# make deinstall clean reinstall
===>  Deinstalling for sysutils/grub2
===>   Deinstalling grub2-1.98
===>  Cleaning for grub2-1.98
.
.
.
#############################################################
To install GRUB on the master boot record of your hard drive
use 'grub-install <drive-to-install>' command.

A typical menu entry in /boot/grub/grub.cfg for FreeBSD:
menuentry "FreeBSD" {
        set root(hd0,1,a)
        kfreebsd /boot/loader
}
Or use grub-mkconfig to create the config file.
#############################################################
install-info --quiet /usr/local/info/grub.info /usr/local/info/dir
===>   Registering installation for grub2-1.98
[root@freebsd-8797 /usr/ports/sysutils/grub2]# grub-install /dev/ad0
/usr/local/sbin/grub-setup: warn: This GPT partition label has no BIOS Boot Partition; embedding won't be possible!.
/usr/local/sbin/grub-setup: warn: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and its use is discouraged..
/usr/local/sbin/grub-setup: error: if you really want blocklists, use --force.

OK, that's progress!  grub-probe isn't throwing any errors anymore and grub recognizes this is a GPT partition.  But what is this about embedding, blocklists, and a BIOS Boot Partition?? 

GRUB needs to find a place to install the stage2 boot code to.  Ideally this will be a contiguous block of sectors in an area of the disk where the bootloader will be protected–i.e. the boot sector of an MBR disk or the dedicated boot partition of a GPT disk.  In this case, GRUB is looking for a GPT partiton of type bios-boot.  Where's ours?

[root@freebsd-8797 /usr/ports/sysutils/grub2]# gpart show ad0
=>      34  21180349  ad0  GPT  (10G)
        34       128    1  freebsd-boot  (64K)
       162   1048576    2  freebsd-ufs  (512M)
   1048738   1048576    3  freebsd-swap  (512M)
   2097314   2097152    4  freebsd-ufs  (1.0G)
   4194466  16985917    5  freebsd-ufs  (8.1G)

Oh there it is, at index 1!  But it's type freebsd-boot instead of bios-boot.  Now at this point we have two options:  Change the partiton type to bios-boot or modify GRUB to recognize freebsd-boot partitions as a valid space to embed to.

Option 1)

gpart doesn't understand the symbolic name bios-boot so you have to tell it the GUID explicitly:

gpart modify -i 1 -t \!21686148-6449-6E6F-744E-656564454649 ad0

Option 2)

I've made two patches that add support for freebsd-boot and solaris-boot partitions to GRUB (patch-include-grub-gpt_partition.h and patch-util-i386-pc-grub-setup.c.  Both need to be applied.  Copy them to /usr/ports/sysutils/grub2/files and repeat the "make deinstall clean reinstall" command as above.

Installing and Configuring GRUB

Before continuing, be sure to have installed a fixed copy of the grub2 port as explained above (delete the bad patch, copy my two patches into the /usr/ports/sysutils/grub2/files/ directory, cd /usr/ports/sysutils/grub2 && make deinstall clean install).  Here's what it might look like:

[grub2]# rm files/patch-util-getroot.c
[grub2]# cp /root/patch-include-grub-gpt_partition.h files
[grub2]# cp /root/patch-util-i386-pc-grub-setup.c files
[grub2]# make deinstall clean install

Now the GRUB userland utilities are installed, but we still need to build the bootloader and embed it into the GPT partition table.  We also need to build the part_gpt module into the bootloader:

freebsd-8797# grub-install --modules=part_gpt /dev/ad0
Installation finished. No error reported.
freebsd-8797# 

Just change /dev/ad0 to the path of your boot drive, whatever it happens to be.

Now GRUB is installed but the boot menu is not configured.  If you were to reboot, you would get a grub prompt waiting for you to tell it how to locate and boot a kernel.  GRUB2 is a little different from GRUB1.  The boot menu is meant to be generated from config files rather than edited by hand.  These config files live in /usr/local/etc/grub.d/

There are two ways to configure GRUB to load FreeBSD: Have GRUB set all the runtime kernel options (including what loadable modules to preload) and invoke the FreeBSD kernel directly, or have GRUB simply invoke /boot/loader and let the loader set all the kernel options and loadable modules.  I really prefer the second option for these reasons: It's easier to configure GRUB this way and it doesn't break the /boot/loader.conf functionality.  It's better to let BSD be configured and loaded the way it was intended to be rather than put that burden on GRUB and leave a bunch of no longer functional config files in /boot/ to confuse yourself with later. 

The default config files will generate a GRUB menu that loads the FreeBSD kernel directly, instead of calling /boot/loader.  Run grub-mkconfig without any options and have a look at the 10_kfreebsd section towards the end.  It should similar to this:

freebsd-8797# grub-mkconfig
.
.
.
### BEGIN /usr/local/etc/grub.d/10_kfreebsd ###
menuentry "FreeBSD, with kFreeBSD kernel" --class freebsd --class bsd --class os {
        insmod ufs2
        set root='(hd0,2)'
        search --no-floppy --fs-uuid --set 4cb05ace512b3d97
        echo                    Loading kernel of FreeBSD kernel ...
        kfreebsd                /boot/kernel/kernel
        kfreebsd_loadenv        /boot/device.hints
        kfreebsd_module_elf     /boot/kernel/acpi.ko
        set kFreeBSD.vfs.root.mountfrom=ufs:/dev/ad0p2
        set kFreeBSD.vfs.root.mountfrom.options=rw
}
### END /usr/local/etc/grub.d/10_kfreebsd ###
.
.
.
freebsd-8797#

If that looks good and you want to load the kernel directly, then just run

grub-mkconfig -o /boot/grub/grub.cfg

and reboot.  (see the note below about the "set root" command before rebooting)

If you want to be able to invoke the FreeBSD loader, then copy/paste the 10_kfreebsd section of the output of grub-mkconfig into /usr/local/etc/grub.d/40_custom and edit it to look like this:

menuentry "FreeBSD /boot/loader" --class freebsd --class bsd --class os {
        insmod ufs2
        set root='(hd0,2)'
        search --no-floppy --fs-uuid --set 4cb05ace512b3d97
        kfreebsd                /boot/loader
}

Then run

grub-mkconfig -o /boot/grub/grub.cfg

and reboot.  (see the note below about the "set root" command before rebooting)

A note about "set root" and "search"

The values of "set root=" and "search" in the grub.cfg file are critical.  Do not just copy and paste mine.  If they are set wrong, GRUB won't be able to find the kernel and thus won't boot the system automatically.  grub-mkconfig will probably set the "set root" parameter wrong, and the "search" parameter right. 

The "set root=" command needs to put into GRUB's language for naming devices.  A value of "/dev/ad0p2" is wrong.  For a GPT partition, it should be in the form of '(hdX,Y)' where X is the number of the hard drive and Y is the index of the root partition.  The commands grub-mkdevicemap and gpart show can be used to determine this information:

freebsd-8797# grub-mkdevicemap -m -
(hd0)   /dev/ad0

freebsd-8797# gpart show ad0
=>      34  21180349  ad0  GPT  (10G)
        34       128    1  freebsd-boot  (64K)
       162   1048576    2  freebsd-ufs  (512M)
   1048738   1048576    3  freebsd-swap  (512M)
   2097314   2097152    4  freebsd-ufs  (1.0G)
   4194466  16985917    5  freebsd-ufs  (8.1G)

freebsd-8797# 

The "search" command serves a similar purpose to the "set root=" command, but it works differently.  Instead of explicitly stating which drive and partition to set as the root, the search command searches all hard drives for a partition labeled with a unique identifier.  It's supposed to be a UUID but FreeBSD doesn't seem to be generating the right labels at this time.  In any case, grub-probe will tell you what GRUB thinks the UUID is for a partition:

freebsd-8797# grub-probe -d --target=fs_uuid /dev/ad0p2
4cb05ace512b3d97

What about ZFS?

GRUB2 has an experimental ZFS module that is reported to boot up to ZFS v28.  Unfortunately, the module is a moving target with no stable branch.  What's worse is that it requires Bazaar to obtain and Ruby and some additional patches in order to compile.  I may try to create a patch that adds ZFS support to the existing grub2 port, but that will be the subject of a future entry.