Yrurtj

I have a live btrfs filesystem of 3.7TiB that's >90% full including old snapshots and a fresh 4TB backup harddisk. How to copy all existing snapshots to the backup harddisk?

I tried

# btrfs send home_1 home_2 home_3 share_1 share_2 share_3 ...

but the backup harddisk was full before 2/3 of the snapshots were transmitted. So I did some research:

# ### create test image

# dd bs=1M count=1000 > btrfs_test.dd

# mkfs.btrfs btrfs_test.dd



# ### create snapshots

# btrfs subvol create testvol/

# btrfs subvol snapshot -r testvol/ testvol_0/

# ### (copy some ISO image)

# btrfs subvol snapshot -r testvol/ testvol_1/

# ### (proceed until testvol_3)

My test filesystem then was 91% full with 818MiB used.

# btrfs send testvol_* | wc -c 

At subvol testvol_0

At subvol testvol_1

At subvol testvol_2

At subvol testvol_3

1466441978     # 1398MiB >> 1000MiB

When simply sending all the snapshots with 1 command, data is duplicated and the size of the stream as well as the size of the snapshots at the receiver's end exceed the original used space and the harddisk's capacity.

So the actual question became: How to copy multiple snapshots without duplicating data that is contained in 2 or more of them?

For this simple test case I successfully tried the incremental approach:

# ( btrfs send testvol_0; btrfs send -p testvol_0 testvol_1; btrfs send -p testvol_1 testvol_2; btrfs send -p testvol_2 testvol_3 ) | wc -c 

At subvol testvol_0

At subvol testvol_1

At subvol testvol_2

At subvol testvol_3

838778546    # 800 MiB < 1000MiB

But on the real filesystem there are multiple subvolumes, each of them with multiple snapshots. I can't define any order to use with -p. If a data block is shared among the subvolumes home_1, home_2 and share_3 I want to transmit and store it only once, of course. Any ideas how to do this?

I am experimenting with the -c switch and it looks promising:

# for i in {0..3}; do btrfs send testvol_$i $(ls -d testvol_* | head -n$i | sed 's/^/-c /'); done | wc -c

### btrfs send testvol_0

### btrfs send testvol_1 -c testvol_0

### btrfs send testvol_2 -c testvol_0 -c testvol_1

### btrfs send testvol_3 -c testvol_0 -c testvol_1 -c testvol_2

At subvol testvol_0

At subvol testvol_1

At subvol testvol_2

At subvol testvol_3

838778546    # also 800MiB

I am still not sure if this is what I need. Any comments on this solution?

Update: To test this with my real filesystem I wrote a Perl script to comfortably send a set of subvolumes (creating the lists for -c quickly became tedious) and sent some data to /dev/null:

#!/usr/bin/env perl



use strict;

use warnings;



my @subvols = @ARGV

  or die "Usage: $0 SUBVOLUME ...n";



for(@subvols) {

    -d $_

      or die "Not a directory: $_n";

}



for my $i (0 .. $#subvols) {

    my $subvol = $subvols[$i];

    my @clones = map { ('-c', $_) } @subvols[ 0 .. $i-1 ];

    print "btrfs send $subvol @clonesn";

}

Results:

• 
  btrfs send some-subvolme_* | pv > /dev/null: 24GiB 0:04:17 [95.2MiB/s]


• 
  perl btrfs-send-all.pl some-subvolume_* | bash | pv > /dev/null: 12.7GiB 0:03:58 [54.7MiB/s]

That's not much of a performance gain but nearly a 50% decrease in storage space! I am now trying to run this in real …

Update: I successfully transferred 2 snapshots but for the 3rd one btrfs receive failed with the following error message:

ERROR: unlink path/to/some/file/in/the/snapshot failed. No such file or directory

The named file is present in subvol_2 and subvol_3 but not yet in subvol_1.

I tried to compare the sent and the received snapshots:

# ### sender

# du -s subvol_{1,2,3}

132472304       subvol_1

117069504       subvol_2

126015636       subvol_3



# ### receiver

# du -s subvol_*

132472304       subvol_1

117069504       subvol_2

132472304       subvol_3

The first two snapshots seem to be transferred correctly but subvol_3 to be a clone of subvol_1. It's definitely a clone because the used space on the backup disk is only 39% of the snapshots' combined size—tightly above 1/3 as they share most files.

Why does btrfs send subvol_3 -c subvol_1 -c subvol_2 | btrfs receive not correctly transfer the snapshot but clone subvol_1 and then fail to delete a file that should be kept in subvol_3 and is only present in subvol_2??

Update: Could it be this bug? https://patchwork.kernel.org/patch/10073969/

I am running Debian 9 Stretch with Kernel 4.9 which seems to be older than the patch.