zfs-initramfs mounting wrong filesystem -, have to manually mount

[aarcane]

This is a fresh install on the zfs mailing list, with only minor modifications. Everything works up until zfs-initramfs is supposed to take over and mount /root for me. Then I recieve the following error message:

command: mount -t zfs -o zfsutil - /root
Message: filesystem - cannot be mounted, unable to open the dataset
mount: mounting - on /root failed: no such file or directory
Error: 1

Things I've tried:
Verifying that zfs-initramfs is installed. it is.
Running update-initramfs -c -k all
verifying bootfs and mountpoint are set properly. They are.
just typing in the PROPER mount command, which is:
mount -t zfs -o zfsutil density/ROOT/ubuntu-1 /root; exit
this works perfectly.
Here's all the info I've been able to gather:

root@density:~# grep root /boot/grub/grub.cfg
        set root='(hd0,gpt2)'
        search --no-floppy --fs-uuid --set=root 3eaf4bab1493f9ec
        zfs-bootfs ($root) bootfs
        linux   /ROOT/ubuntu-1/@/boot/vmlinuz-3.2.0-23-generic root=/dev/sda2 ro boot=zfs $bootfs   splash $vt_handoff
        set root='(hd0,gpt2)'
        search --no-floppy --fs-uuid --set=root 3eaf4bab1493f9ec
        zfs-bootfs ($root) bootfs
        linux   /ROOT/ubuntu-1/@/boot/vmlinuz-3.2.0-23-generic root=/dev/sda2 ro single nomodeset boot=zfs $bootfs


root@density:~# sudo blkid
/dev/sda2: LABEL="density" UUID="4516912149432629740" UUID_SUB="9711709925157121020" TYPE="zfs_member"
/dev/sdb2: LABEL="density" UUID="4516912149432629740" UUID_SUB="8526453012990331363" TYPE="zfs_member"


root@density:~# zfs get all density/ROOT/ubuntu-1
NAME                   PROPERTY              VALUE                  SOURCE
density/ROOT/ubuntu-1  type                  filesystem             -
density/ROOT/ubuntu-1  creation              Fri Mar 22  5:32 2013  -
density/ROOT/ubuntu-1  used                  1.74G                  -
density/ROOT/ubuntu-1  available             5.45G                  -
density/ROOT/ubuntu-1  referenced            1.66G                  -
density/ROOT/ubuntu-1  compressratio         1.00x                  -
density/ROOT/ubuntu-1  mounted               yes                    -
density/ROOT/ubuntu-1  quota                 none                   default
density/ROOT/ubuntu-1  reservation           none                   default
density/ROOT/ubuntu-1  recordsize            128K                   default
density/ROOT/ubuntu-1  mountpoint            /                      local
density/ROOT/ubuntu-1  sharenfs              off                    default
density/ROOT/ubuntu-1  checksum              on                     default
density/ROOT/ubuntu-1  compression           off                    default
density/ROOT/ubuntu-1  atime                 on                     default
density/ROOT/ubuntu-1  devices               on                     default
density/ROOT/ubuntu-1  exec                  on                     default
density/ROOT/ubuntu-1  setuid                on                     default
density/ROOT/ubuntu-1  readonly              off                    default
density/ROOT/ubuntu-1  zoned                 off                    default
density/ROOT/ubuntu-1  snapdir               hidden                 default
density/ROOT/ubuntu-1  aclinherit            restricted             default
density/ROOT/ubuntu-1  canmount              on                     default
density/ROOT/ubuntu-1  xattr                 on                     default
density/ROOT/ubuntu-1  copies                1                      default
density/ROOT/ubuntu-1  version               5                      -
density/ROOT/ubuntu-1  utf8only              off                    -
density/ROOT/ubuntu-1  normalization         none                   -
density/ROOT/ubuntu-1  casesensitivity       sensitive              -
density/ROOT/ubuntu-1  vscan                 off                    default
density/ROOT/ubuntu-1  nbmand                off                    default
density/ROOT/ubuntu-1  sharesmb              off                    default
density/ROOT/ubuntu-1  refquota              none                   default
density/ROOT/ubuntu-1  refreservation        none                   default
density/ROOT/ubuntu-1  primarycache          all                    default
density/ROOT/ubuntu-1  secondarycache        all                    default
density/ROOT/ubuntu-1  usedbysnapshots       88.3M                  -
density/ROOT/ubuntu-1  usedbydataset         1.66G                  -
density/ROOT/ubuntu-1  usedbychildren        0                      -
density/ROOT/ubuntu-1  usedbyrefreservation  0                      -
density/ROOT/ubuntu-1  logbias               latency                default
density/ROOT/ubuntu-1  dedup                 off                    default
density/ROOT/ubuntu-1  mlslabel              none                   default
density/ROOT/ubuntu-1  sync                  standard               default
density/ROOT/ubuntu-1  refcompressratio      1.00x                  -
density/ROOT/ubuntu-1  written               472M                   -
root@density:~#


/proc/cmdline:
BOOT_IMAGE=/ROOT/ubuntu-1/@/boot/vmlinuz-3.5.0-26-generic root=/dev/sda2 ro boot=zfs zfs-bootfs=density/21 quiet

grub.cfg:
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
  set have_grubenv=true
  load_env
fi
set default="${saved_entry}"
if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
#  save_env saved_entry
  set prev_saved_entry=
#  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
#    save_env saved_entry
  fi
}

function recordfail {
  set recordfail=1
  #if [ -n "${have_grubenv}" ]; 
  #  then if [ -z "${boot_once}" ]; 
  #    then save_env recordfail; 
  #  fi; 
  #fi
}

function load_video {
  insmod vbe
  insmod vga
  insmod video_bochs
  insmod video_cirrus
}

terminal_input console
terminal_output console
if [ "${recordfail}" = 1 ]; then
  set timeout=-1
else
  set timeout=3
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/05_debian_theme ###
set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
### END /etc/grub.d/05_debian_theme ###

### BEGIN /etc/grub.d/10_linux ###
function gfxmode {
    set gfxpayload="${1}"
    if [ "${1}" = "keep" ]; then
        set vt_handoff=vt.handoff=7
    else
        set vt_handoff=
    fi
}
if [ "${recordfail}" != 1 ]; then
  if [ -e ${prefix}/gfxblacklist.txt ]; then
    if hwmatch ${prefix}/gfxblacklist.txt 3; then
      if [ ${match} = 0 ]; then
        set linux_gfx_mode=keep
      else
        set linux_gfx_mode=text
      fi
    else
      set linux_gfx_mode=text
    fi
  else
    set linux_gfx_mode=keep
  fi
else
  set linux_gfx_mode=text
fi
export linux_gfx_mode
if [ "${linux_gfx_mode}" != "text" ]; then load_video; fi
menuentry 'Ubuntu, with Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os {
    recordfail
    gfxmode $linux_gfx_mode
    insmod gzio
    insmod part_gpt
    insmod zfs
    set root='(hd0,gpt2)'
    search --no-floppy --fs-uuid --set=root 3eaf4bab1493f9ec
    insmod zfsinfo
    zfs-bootfs ($root) bootfs
    linux   /ROOT/ubuntu-1/@/boot/vmlinuz-3.2.0-23-generic root=/dev/sda2 ro boot=zfs $bootfs   splash $vt_handoff
    initrd  /ROOT/ubuntu-1/@/boot/initrd.img-3.2.0-23-generic
}
menuentry 'Ubuntu, with Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os {
    recordfail
    insmod gzio
    insmod part_gpt
    insmod zfs
    set root='(hd0,gpt2)'
    search --no-floppy --fs-uuid --set=root 3eaf4bab1493f9ec
    insmod zfsinfo
    zfs-bootfs ($root) bootfs
    echo    'Loading Linux 3.2.0-23-generic ...'
    linux   /ROOT/ubuntu-1/@/boot/vmlinuz-3.2.0-23-generic root=/dev/sda2 ro single nomodeset boot=zfs $bootfs 
    echo    'Loading initial ramdisk ...'
    initrd  /ROOT/ubuntu-1/@/boot/initrd.img-3.2.0-23-generic
}
### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
if [ "x${timeout}" != "x-1" ]; then
  if keystatus; then
    if keystatus --shift; then
      set timeout=-1
    else
      set timeout=0
    fi
  else
    if sleep --verbose --interruptible 3 ; then
      set timeout=0
    fi
  fi
fi
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

and finally, an e-mail I wrote to the zfsonlinux mailinglist describing my system prior to discovering this issue.

So last night I followed the guide at https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem almost verbatim, and made the following changes, and it worked flawlessly!

Mirror: My root pool is a mirror, so that works.
USB: My root pool is configured as a mirror
ashift=12: Ashift 12 works now for boot!
GPT Partition Tables: I created two partitions on each drive, one for bios_grub and one for the rpool. had to mess with the alignment a bit to get it right.
No /boot partition: I edited /etc/grub.d/00_header and placed a single # at the beginning of each line where the string "save_env" occurs. Everything works, and I can use userspace tools to configure boot behavior.
Had to debug grub install: the *.mod files were never installed, I had to install them manually using the instructions in the Troubleshooting section.
Manually mount the rpool after first boot: The rpool wasn't properly mounted or configured after reboot. I had to manually import it, and will manually update the initramfs before I shut it down. Not sure why, but once I imported and mounted the pool, everything was peachy [--EDIT FOR GITHUB: Obviously everyong was not quite peachy :( --]

[aarcane]

Not sure why grub.cfg comments became headings.. I apologize for any confusion this may cause.

[dajhorn]

@aarcane, I added triple back-ticks (```) so that the pasted information is properly formatted.

Perhaps the GRUB run-parts could be made smart enough to omit save_env when /boot is in a ZFS dataset.

[FransUrbo]

The Debian GNU/Linux Wheezy snapshot/daily packages have had their initrd scripts almost completely rewritten so I can't see this as a problem in Debian GNU/Linux. Tagging appropriately.

[dajhorn]

ZoL initramfs features were deprecated and discontinued in the PPA, and partially continued downstream, so I will close this ticket.