I was nervous about replacing the HDD (hard drive) containing BTRFS on root on my main computer with an SSD. However, remembered times when I was able to fix non-booting computers before using guides and sometimes help. I also felt better knowing I had a backup. Unlike Windows, a GNU+Linux operating system is not a “Humpty-Dumpty” system–you can be sure that if you have all of the files (not even boot partition files necessarily), there is a way to put it back together again. I’ve had more than a couple lengthy nightmare scenarios on Windows computers where no amount of refreshing, nor various boot repair commands, nor config file edits from forums including the Microsoft Support website did any good making Windows find the boot device again (even though everything was there!). The cause was usually a hard drive replacement or other system change. Explaining why Linux is better to certain customers or employers has been difficult. They tend to revert to the idea that I like it to “save money” no matter how much I explain it–people tend to erase their own experiences since Windows is popular and has more big companies supporting certain aspects of it. That is a whole separate topic, but it suffices to say that there are paid options for Linux in areas where it is lacking just like there are on Windows (including Windows Server). Since GNU+Linux is not a “Humpty-Dumpty” system, lets learn how to save the day and put a Linux system “back together again.”
If you are replacing your main hard drive which has BTRFS on root but EFI and SWAP are on separate partitions, you need to do some extra steps. Copying the three partitions changing the UID in fstab and grub.conf do not do it all. Updating or reinstalling GRUB won’t do any good if you don’t switch to the EFI version of the grub package (not grub-pc)! I found some advice on setting up my partitions this way, and the Debian 10 (Buster) ISO installed everything correctly (I switched to Devuan Chimera, but everything here still worked using a Debian 10 live CD to complete the setup of the destination drive). As for moving over to a new drive, there wasn’t any direct advice. I used separate guides and some trial and error accomplish the task.
Before starting:
- Follow steps in this article at your own risk. This article has no express or implied warranty or guarantee.
- Make a live DVD or CD (or USB if you don’t have an optical drive) with Debian 10 (If your computer has Debian 10 or Devuan Chimera; otherwise use whatever ISO matches your system best).
- This guide assumes you have three partitions: /boot/efi (msdos), swap (swapfs), and / (BTRFS as the root filesystem–having subvolumes such as for home or other mount points is fine).
- Backup your old drive to a backup (third drive, not the new drive) with enough free space such as via:
-
sudo mount /dev/sdbX /mnt/backup-drivewhere sdbX is the correct backup drive thensudo rsync -avPX --info=progress2 / /mnt/backup-drive/old-drive-slash sudo rsync -avPX --info=progress2 /boot/efi /mnt/backup-drive/old-drive-efi). You can backup your EFI partition and your BTRFS partition. If your backup drive is BTRFS, you can send a remote snapshot which is a nice way to do incremental backups (requires the btrfs-progs package).
-
- Note that experts have mixed opinions on using BTRFS on root, since only the core features of BTRFS are considered ready for daily use according to the kernel.org wiki, and the filesystem could be considered “perpetually half-finished” (whether that is relevant to any features used here is unclear–It seems only relevant to the snapshot and RAID features which aren’t required for this guide). The kernel.org wiki recommends using the latest kernel when using BTRFS.
- (Optional unless you aren’t sure how to ensure you have enough space) You can ensure you have the exact amount of space necessary as follows: Make sure the new drive is the same size as the old drive. Power off. Plug in the new (blank) drive and power on and log into the old OS. Ensure all programs are closed to avoid needing swap space. Open GParted. Note the partition sizes of the old drive. Right-click the old swap partition, click “Swapoff.” Right-click it again, Copy. Select the new drive. If you can’t create partitions and it says to create a filesystem, follow the instructions on screen and choose “GPT”. The mountpoints don’t matter yet–they will be defined in the configuration of the new copy of the drive, not here. Create an EFI partition of the same size (usually an easy even number) as the old drive’s. Paste the swap partition. Create a new partition, choose BTRFS. Go back to the main drive. Right-click swap and choose “Swapon”.
- (Optional) For extra certainty so you can tell which drive is which, you can power off, remove your backup drive then power on.
- To avoid files changing or becoming desynchronized from each other or from databases (usually not a significant problem except on servers, but filesystem- or database-heavy applications such as Nextcloud client may be affected), log into a TTY (text mode) rather than using the GUI login. Press Ctrl+Alt+3 (or change 3 to whatever TTY is unused) and login as root. If you are using a server, stop any services that may write to the disk heavily such as email, Nextcloud (Nextcloud server), web servers (Apache, NGINX), custom web applications, etc. Many of the commands require root privileges. If you are not logged in as root, type “sudo” before each command.
Now that you have both drives installed, the old OS is booted, the new drive has the same partition layout, and you are logged in, you are ready. There is no need to recreate the BTRFS subvolumes since btrfs-replace will copy those automatically.
In each command, change sda, sda3, sdb, sdb3 and ‘/’ to the correct values for your system (old physical drive, old BTRFS partition, new physical drive, BTRFS partition on new drive, and mountpoint to migrate from the old to new partition, respectively). The steps assume you want a GPT filesystem and UEFI: If you use mbr and a BIOS boot partition, you will have to use grub-pc instead of grub-efi-amd64 and change the grub installation steps.
btrfs replace start /dev/sda3 /dev/sdb3
btrfs replace status(Those commands are based on Replacing an online drive with a bigger one from the BTRFS documentation but your new partition can be the same size.)
Wait for the status command to inform you that the operation completed successfully before continuing!
mkdir /mnt/sdb1The btrfs replace start ... subcommand above will automatically change fstab, but assuming you have /boot/efi and swap on separate partitions, more steps are necessary. Even if they are not (swap can be in BTRFS, but must be on a separate subvolume to allow snapshots on the volume since it prevents snapshots on its own volume/subvolume), you still need to install grub on the new drive. Some people use the dd command for that, but I don’t recommend that right now since reinstalling GRUB is easier: Reinstalling grub can detect your operating systems and generate grub.cfg automatically.
sudo mount /dev/sdb1 /mnt/sdb1The trailing slash in the command below is important: You don’t need the copy to have /mnt/sdb1/efi/efi/EFI, but rather /mnt/sdb1/efi/EFI.
sudo rsync /boot/efi/ /mnt/sdb1Change the command below to the existing file. Run ls /mnt/sdb1/efi/EFI to see where your subdirectory is–In my case it was “debian” but it may be “grub” (The new one we’ll install/reinstall will be “grub”).
Boot from your live CD, DVD or USB (See “Before starting” above) to continue the rest of the steps–though if you don’t get the dreaded “Boot Device Not Found” message or a similar error or didn’t reboot you may be able to continue without it. Using it may change the drive letter(s) below (such as sda) so make sure to replace those with the correct ones by listing them with lsblk and blkid first!
sudo apt-get update
sudo apt-get install nano -y
sudo nano /mnt/sdb1/efi/EFI/debian/grub.cfgUsing nano (or another editor), change the grub.cfg to use the UUID of the new drive’s efi partition (sdb1 if using this guide), as listed by the blkid command. Use the short UUID (not the long PARTUUID). Press Ctrl+x to exit nano, then confirm save.
If you did the btrfs replace ... subcommands above correctly, then ‘/’ should now be mounted to the new drive. You must edit fstab to ensure the non-BTRFS partitions get mounted (prevent the dreaded “UUID=… does not exist.” “A file system check failed” error that prevents boot; otherwise use the rescue prompt that error presents and login as root to continue).
sudo cat /etc/fstabCompare the output above to the output of the blkid command. The UUID for the root filesystem “/” should already have been changed automatically by the first btrfs replace ... subcommand when it finished. Take note of the correct UUID for the swap and /boot/efi partitions.
sudo nano /etc/fstabUsing nano (or another editor), set the UUID for /boot/efi and the UUID for swap to match the new UUIDs for the first and second partitions on the new drive, respectively (such as listed for /dev/sdb1 and /dev/sdb3 using the blkid command, or possibly /dev/sda1 and /dev/sda3 if you rebooted without the old drive–the number(s) will differ if using a different layout than this guide uses, and the last letter of each will differ if drives are recognized by linux in a different order).
Now you must install grub onto /dev/sdb (It may be /dev/sda if you are using a removable boot drive. The last letter will differ if drives are recognized by linux in a different order).
sudo umount /mnt/sdb1
sudo rmdir /mnt/sdb1(The set of commands below is based on roadmr‘s Jun 1, 2012 answer edited Jun 2, 2012 on How do I run update-grub from a LiveCD? on StackExchange)
sudo mkdir -p /mnt/new
sudo mount /dev/sdb3 /mnt/new
sudo mount --bind /dev /mnt/new/dev
sudo mount --bind /sys /mnt/new/sys
sudo mount --bind /proc /mnt/new/proc
sudo mount /dev/sdb1 /mnt/new/boot/efiYou may need to install grub (fixes the error grub-install: command not found). You should do sudo apt-get install grub-efi-amd64 (not grub-pc) to work with EFI (according to Rod Smith‘s Apr 18, 2015 answer on grub2-install: “this GPT partition label contains no BIOS Boot Partition” on StackExchange).
According to How to Install GRUB2 with EFI Support by LinuxLink (if your OS is 32-bit see the instructions there for 32-bit which differ!), you can install grub with EFI compatibility with the following two commands. Make sure the mount commands above were done correctly before continuing!
sudo linux64 chroot /mnt/newEnsure the directory “EFI” (capitalized) exists in /boot/efi to ensure it mounted correctly before continuing (unless you are doing a new install or didn’t do the rsync command on it before to preserve old configuration files)!
The drive letters is changed for this guide, but change it to whatever is correct in the command below before running it!
grub-install --target=x86_64-efi --efi-directory=/boot/efi --removable --boot-directory=/boot/efi/EFI --bootloader-id=grub /dev/sdbThe “–removable” option isn’t what it sounds like in this case: It is necessary to avoid writing boot information to the real (non-chroot) system!
The LinuxLink article says that grub.cfg will not have the root device correct due to the bind mounting, but I didn’t have to change anything. See the “Cleanup grub.cfg” section of the article and ensure that information is correct in /boot/efi/EFI/grub/grub.cfg (The set root line may have to be changed to set root='hd0,msdos2' or change 0 to whatever partition number matches your /boot/efi partition as shown by the lsblk command if the set root line is not already correct, but start at 0 not 1 when counting to determine the correct one).
The LinuxLink article says to manually create startup.nsh as follows. I don’t know whether it is necessary but I did it. They provide unclear instructions regarding making and copying the file, so I’ve changed the path for nano below to the place where the file should eventually go, which differs in our case anyway:
sudo nano /boot/efi/startup.nshThen use nano (or another editor) to type the following into the file (If your OS is 32-bit, see “For 32-bit targets” under “Create startup.nsh script” in the article for the correct content instead of the 64-bit content below):
FS0:
cd \efi\grub\x86_64-efi
grub.efiThe case doesn’t seem to matter, as my path is actually \EFI\grub\x86_64-efi uppercase (inside the /boot/efi lowercase directory) but I have used the startup.nsh text above on a working system. Ctrl+x to exit nano and confirm save.
As per roadmr’s instructions back in the StackOverflow answer further up, end by exiting chroot and unmounting everything (mount points are changed to the ones in this guide):
exit
sudo umount /mnt/new/dev
sudo umount /mnt/new/sys
# ^ Failing to unmount sys won't matter since restarting
sudo umount /mnt/new/proc
sudo umount /mnt/new/boot/efi
sudo umount /mnt/newThen shutdown and remove the old drive if not already.
If sudo shutdown 0 doesn’t work (didn’t for me due to missing system files, probably related to the sys mount not unmounting), try sudo reboot (that worked for me) then shut down using the computer’s power button when the BIOS logo appears if you still have the old drive plugged in.
After all that, everything worked for me. Devuan Chimera booted without problems. If you get any of the errors still, see the instructions related to the error if the error is noted on this article.
This method was better for me. Some people may use dd, but that copies the UUIDs and prevents access to the old drive via UUID if you are having trouble (That will most likely only cause issues if you boot with the old drive plugged in or if you try to use this guide to fix the issues it mentions but already instead attempted migration using dd). Note that if you want to dd a whole drive, no matter why you are doing it you should generally use ddrescue instead (usually the package that provides the command is called gddrescue). Not only will it ensure all of the correct options are used to prevent fatally desynchronizing the drives (putting bits in the wrong files if you use the noerror setting without the sync setting!), but also that multiple attempts are made to get as much data as possible if the old drive has problems. However, you don’t need dd nor ddrescue. This guide shows how to copy files instead which will save time and reduce wear on your SSD. After doing all of this and doing some of these steps as a part of solving other issues, I realized that having it all in one place to avoid the errors in the first place and focusing on the EFI way would be helpful to myself later and to people using EFI.