Server 7.0.5: [fs mgr] Failed to read dir

Hi

I’ve just migrated my data storage from one location to another. Now when I run garbage collect, one of the repositories seem broken. Seafile fails to read a certain file/directory:

[11/10/20 02:16:34] gc-core.c(440): GC version 1 repo Pictures(420486bc-98a0-4ed1-a512-fc3da4bda5b4)
[11/10/20 02:16:35] gc-core.c(313): GC started. Total block number is 29807.
[11/10/20 02:16:35] gc-core.c(46): GC index size is 14903 Byte.
[11/10/20 02:16:35] gc-core.c(327): Populating index.
[11/10/20 02:16:35] gc-core.c(181): Populating index for repo 420486bc.
[11/10/20 02:24:28] ../../common/fs-mgr.c(1908): [fs mgr] Failed to read dir 3b80a297f527334e2219846455af1a48c4dfd2f2.
[11/10/20 02:24:28] ../../common/fs-mgr.c(2225): [fs-mgr]get seafdir 3b80a297f527334e2219846455af1a48c4dfd2f2 failed
[11/10/20 02:24:28] ../../common/commit-mgr.c(518): [comit-mgr] CommitTraverseFunc failed
[11/10/20 02:24:28] gc-core.c(234): Traversed 629 commits, 29336 blocks.

However, the file is fine and has correct ownership, permissions and content:

This is from the “new” migrated location.

$ ls -al fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
-rwxr-xr-x 1 1000 1000 1314 Jul  7 17:49 fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
$ md5sum fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
20b65ee1e8a791c0ea659ef14d5a772f  fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2

This is from the “old” location I’m migrating from.

$ ls -al fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
-rwxr-xr-x 1 1000 1000 1314 Jul  7 17:49 fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
$ md5sum fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2
20b65ee1e8a791c0ea659ef14d5a772f  fs/420486bc-98a0-4ed1-a512-fc3da4bda5b4/3b/80a297f527334e2219846455af1a48c4dfd2f2

Any Ideas? I’ve tried to use seaf-fsck. The command crunches the data for an hour and it looks like everything goes well

Starting seaf-fsck, please wait ...

[11/10/20 00:17:35] fsck.c(590): Running fsck for repo 420486bc-98a0-4ed1-a512-fc3da4bda5b4.
[11/10/20 00:17:35] fsck.c(416): Checking file system integrity of repo Pictures(420486bc)...
[11/10/20 01:08:15] fsck.c(654): Fsck finished for repo 420486bc.

seaf-fsck run done

Done.

When I run garbage collection again, I get the same error. If I start seafile-server, it works like normal.

I’m not sure how to investigate this error further. Any help/directions for further investigation?

OK, after reading about seaf-fsck.sh I now understand that it does not do anything unless --repair is specified. I also understand that it does not do anything about the history of the repository, it only reports about errors in the “current” version. So, most likely, I have a corrupt directory in the file history. It is probably corrupt before I migrated from the old storage to the new storage.

The only problem I see with this is that I’m no longer able to run the garbage collector for the damaged repository (because it cannot traverse all blocks for the repository).

So, my idea was to go use seafile web interface to view the history of the damaged repository to see which files were created during 7th of July (the date this file was modified), and then to view a snapshot of how my repository looked like back then. I expected something to go wrong here, i.e that a file or directory could not be found/opened. But nothing broke and everything seemed fine.

Ok, I have found the problem. The issue is not related to seafile itself, but to NFS which I use to mount the storage. I realised that some files timed out when read from the mount when using NFSv3 to mount it, but when mounting it with NFSv4, all files are readable. I have no idea why NFSv3 times out while NFSv4 works as expected though.

Thanks for reporting this and glad to hear that you found the problem.
Could you mark as solved?

1 Like