How to fix a broken Linux boot loader

I'm no Linux guru but I'll happily get my hands dirty when it comes to tinkering. I'd like to think I know enough to be dangerous - and that's exactly what happened this time.

After cloning a mate's CentOS server I wanted to make it as generic as possible. Logical Volume Manager (LVM) was used. It had one Volume Group (VG) and two Logical Volumes (LV).

The LVs were named lv_root and lv_swap. No problems there - quite generic.

The VG, however, contained the hostname which is definitely not generic. e.g. vg_myserver02. So I proceeded to change this by running vgrename.

vgrename vg_myserver02 vg_01  

I made some more unrelated changes then rebooted. Then it broke. It wouldn't boot probably because I forgot to update /boot/grub/grub.conf and /etc/fstab. And the root partition is on the LVM volume I had just renamed.

This is how I fixed the broken boot loader.

  1. Boot the server off a live/recovery CD and drop into a shell.

  2. Run fdisk -l to view the partitions found.

    alt

    /dev/sda1 is the boot (/boot) partition. We need to edit grub.conf in this partition.

    /dev/sda2 is the LVM volume in which the root (/) partition. We need to edit /etc/fstab in this partition.

    We will need to mount both partitions in order to edit the required files.

  3. I created two directories under /mnt - root and boot - then mounted /dev/sda1 to boot.

    mount /dev/sda1 /mnt/boot
    
  4. When attempting to mount /dev/sda2 to /mnt/root, we run into a problem - "unknown filesystem type 'LVM2_member'". This is because we can't mount an LVM "container" containing an actual LVM volume.

  5. I noticed that in /dev I couldn't see my LVM VG. There are the dm-0, dm-1 devices which typically map to an LVM volume. You can mount these devices and see what's inside of them. Alternatively you can run vgscan -v followed by vgchange -a y. This will scan, then activate, any discovered LVM VGs and add them to the list in /dev.

    alt

  6. Now we can mount the logical volume itself.

    mount /dev/vg_01/lv_root /mnt/root
    
  7. Edit /mnt/boot/grub/grub.conf and replace all instances of the old VG name with the new.

  8. Edit /mnt/root/etc/fstab and replace all instances of the old VG name with the new.

  9. Reboot the system and with any luck it will boot properly!

After all's said and done, I'm pretty chuffed with myself and I think my Linux-fu just leveled up! :)