Cleaning up blocks from moved files

Daniel15 · June 8, 2021, 5:42pm

I uploaded a lot of files to one library (e382a2fa-f81e-4d3b-b329-ae97c7139885), then moved them all to another library (f5d4946e-cff9-4fa4-982a-239c3ba06617).

It looks like Seafile kept the blocks in the original library’s storage/blocks directory, and hard linked them in the new library:

daniel@la03:/mnt/seafile-data/storage$ ls blocks/e382a2fa-f81e-4d3b-b329-ae97c7139885/79/ff655bae80d1d69d25ae84728e76754c4856e4 -lah
-rw------- 2 seafile seafile 10M Aug  7  2019 blocks/e382a2fa-f81e-4d3b-b329-ae97c7139885/79/ff655bae80d1d69d25ae84728e76754c4856e4

daniel@la03:/mnt/seafile-data/storage$ ls blocks/f5d4946e-cff9-4fa4-982a-239c3ba06617/79/ff655bae80d1d69d25ae84728e76754c4856e4 -lah
-rw------- 2 seafile seafile 10M Aug  7  2019 blocks/f5d4946e-cff9-4fa4-982a-239c3ba06617/79/ff655bae80d1d69d25ae84728e76754c4856e4

daniel@la03:/mnt/seafile-data/storage$ stat blocks/e382a2fa-f81e-4d3b-b329-ae97c7139885/79/ff655bae80d1d69d25ae84728e76754c4856e4
  File: blocks/e382a2fa-f81e-4d3b-b329-ae97c7139885/79/ff655bae80d1d69d25ae84728e76754c4856e4
  Size: 10485760        Blocks: 20480      IO Block: 262144 regular file
Device: 100059h/1048665d        Inode: 244586351   Links: 2
Access: (0600/-rw-------)  Uid: (  108/ seafile)   Gid: (  114/ seafile)
Access: 2021-01-22 12:28:16.297183803 -0800
Modify: 2019-08-07 16:45:45.571403869 -0700
Change: 2021-06-05 12:39:24.916854809 -0700
 Birth: -

daniel@la03:/mnt/seafile-data/storage$ stat blocks/f5d4946e-cff9-4fa4-982a-239c3ba06617/79/ff655bae80d1d69d25ae84728e76754c4856e4
  File: blocks/f5d4946e-cff9-4fa4-982a-239c3ba06617/79/ff655bae80d1d69d25ae84728e76754c4856e4
  Size: 10485760        Blocks: 20480      IO Block: 262144 regular file
Device: 100059h/1048665d        Inode: 244586351   Links: 2
Access: (0600/-rw-------)  Uid: (  108/ seafile)   Gid: (  114/ seafile)
Access: 2021-01-22 12:28:16.297183803 -0800
Modify: 2019-08-07 16:45:45.571403869 -0700
Change: 2021-06-05 12:39:24.916854809 -0700
 Birth: -

(notice the block in e382a2fa-f81e-4d3b-b329-ae97c7139885 has the same inode as the block in f5d4946e-cff9-4fa4-982a-239c3ba06617).

Is it possible to clean up the blocks in the old library so that they only appear in the new repo? I know they take very minimal space since they’re hardlinks, but it messes with my backup strategy (some libraries have very important files where I take two offsite backups daily, whereas others are less important and I only take one offsite backup weekly)

I tried seaf-gc but it just told me that there’s no blocks to remove:

seafile@la03:~/seafile-server-7.0.5$ ./seaf-gc.sh --dry-run e382a2fa-f81e-4d3b-b329-ae97c7139885

Starting seafserv-gc, please wait ...
[06/08/21 10:27:36] gc-core.c(440): GC version 1 repo Apps(e382a2fa-f81e-4d3b-b329-ae97c7139885)
[06/08/21 10:27:38] gc-core.c(313): GC started. Total block number is 31601.
[06/08/21 10:27:38] gc-core.c(46): GC index size is 15800 Byte.
[06/08/21 10:27:38] gc-core.c(327): Populating index.
[06/08/21 10:27:38] gc-core.c(181): Populating index for repo e382a2fa.
[06/08/21 10:32:22] gc-core.c(234): Traversed 471 commits, 31814 blocks.
[06/08/21 10:32:22] gc-core.c(343): Scanning unused blocks.
[06/08/21 10:32:22] gc-core.c(369): GC finished. 31601 blocks total, about 31814 reachable blocks, 0 blocks can be removed.

rdb · June 8, 2021, 11:36pm

I am pretty sure Seafile is not hard linking.

Have you deleted the original library (e382a2fa) or purged the history of the library? If you haven’t done either, no block is in trash and hence the GC won’t find any blocks to remove.

Daniel15 · June 8, 2021, 11:55pm

It definitely is

github.com

haiwen/seafile/blob/eed7fa2250e7e13e4ac98ec519e044ce892ca508/common/block-backend-fs.c#L349


      
              }
          
          #ifdef WIN32
              if (!CreateHardLinkA (dst_path, src_path, NULL)) {
                  seaf_warning ("Failed to link %s to %s: %lu.\n",
                                src_path, dst_path, GetLastError());
                  return -1;
              }
              return 0;
          #else
              int ret = link (src_path, dst_path);
              if (ret < 0 && errno != EEXIST) {
                  seaf_warning ("Failed to link %s to %s: %s.\n",
                                src_path, dst_path, strerror(errno));
                  return -1;
              }
              return ret;
          #endif
          }
          
          static int

https://manpages.debian.org/stretch/manpages-dev/link.2.en.html

Good catch! I think you’re onto something here. I haven’t deleted the original library (as there’s other files in it that I do want to keep there). I noticed that the files I moved were in the trash. I didn’t expect the “move” operation to leave them in the trash of the original library. I cleaned the trash, however the GC still says “0 blocks are removed.”

Daniel15 · June 10, 2021, 6:07pm

So I figured out that if I:

Clear the trash
Set the history setting for the library to “Don’t keep history” (in the system admin section)

The garbage collector will remove these blocks.