[SOLVED] Fileserver.log filling up with "cannot find seafdir for origin repo"

Seafile Server
#1

Hello Seafile community !

We recently upgraded our multi-tenant Seafile Pro from 9.0.16 to 10.0.9.
Since then we have noticed that the fileserver log was filling up with those 4 lines :

[2023-10-26 09:58:10] merge repo cec44dd8 failed: failed to get origin repo commit 9d2d15fa-5633-4a94-9874-3f3d47e60979:783e26b5
[2023-10-26 09:58:10] merge repo ecf827a5 failed: failed to get origin repo commit 9d2d15fa-5633-4a94-9874-3f3d47e60979:1c5faa6b
[2023-10-26 09:58:10] merge repo 67b3e90a-d failed: cannot find seafdir for origin repo 9d2d15fa-5 path /path1
[2023-10-26 09:58:11] merge repo 92e20fec-7 failed: cannot find seafdir for origin repo 9d2d15fa-5 path /path2

We ran a seaf-fsck -r on the repo 9d2d15fa but it didn’t fix or find any problem.

Does anyone know what the problem is and how to fix it ?

#2

Hello, the failure to merge libraries should be caused by gc deleting outdated commit and fs objects. Newer versions of seafile gc will keep the commits referenced by the virtual libraries.
You can try to fix this with the virtual library repair script. If the script repair fails, you can try to fix it by adding the --force option.
Please refer to: Seafile Cloud

#3

Hello,
And thank you for your answer Feiniks.
We did use your script on the library but it hasn’t been a huge success :frowning_face:
The errors have evolved to this :

[2023-10-31 14:39:33] merge repo 67b3e90a-d failed: cannot find seafdir for origin repo 9d2d15fa-5 path /path1
[2023-10-31 14:39:33] merge repo 92e20fec-7 failed: cannot find seafdir for origin repo 9d2d15fa-5 path /path2

So it went from

merge repo cec44dd8 failed: failed to get origin repo commit 9d2d15fa-5633-4a94-9874-3f3d47e60979:783e26b5

to

merge repo 67b3e90a-d failed: cannot find seafdir for origin repo 9d2d15fa-5 path /path

We did try to use seaf-fsck after the repair_repo script, but it didn’t correct the problem…

Any idea of what is going wrong ?

#4

Hello @RomainC , did the repair script succeed in fixing this library and are there any error messages?

#5

Hello,
I don’t recall seeing any error or any fix done.
I did launch the script again and the output was this :

sudo -u seafile ./repair_repo.sh -r 9d2d15fa-wwww-xxxx-yyyy-zzzzzzzzzzz--force
mysql+pymysql://user:password@dbIP:3306/seafile-db
2023-11-06 10:07:41,579 Repair repo is finished, 0 repos are corrupt, 0 repos are repaired.

The seaf-fsck script didn’t remove the errors neither…
I still have those errors in loop :

[2023-11-06 10:36:03] merge repo 67b3e90a-d failed: cannot find seafdir for origin repo 9d2d15fa-5 path /Path1
[2023-11-06 10:36:04] merge repo 92e20fec-7 failed: cannot find seafdir for origin repo 9d2d15fa-5 path /Path2

This only happens on one repo.

#6

Hello @RomainC , the libraries that are reporting errors are 67b3e90a and 92e20fec. Did you fix these two libraries with the repair script? The library you are trying to fix in the screenshot should be 9d2d15fa.

#7

Hello,
And thanks again @feiniks for your much appreciated help !
I have just applied the repair_repo script to those libs. There was no error and I will let you know if we have the error again.
I didn’t think of trying this before because those repos seems to be sub-repo of repo 9d2d15fa…
I will post the result tomorrow :crossed_fingers:

#8

Hello, this script can only be used to repair virtual libraries, it does not work on non virtual libraries.

#9

Hello,
Unfortunately, the problem is still there… :frowning_face:
I don’t know what else to do.

#10

Hello，can you show the output of the repair_repo script for the virtual library?

#11

Sure ! Here is the output :

/opt/seafile/seafile-server-latest# sudo -u seafile ./repair_repo.sh -r 67b3e90a

mysql+pymysql://user:password@serverIP:3306/seafile-db
2023-11-07 09:56:36,976 Repair repo is finished, 0 repos are corrupt, 0 repos are repaired.

/opt/seafile/seafile-server-latest# sudo -u seafile ./repair_repo.sh -r 67b3e90a --force

mysql+pymysql://user:password@serverIP:3306/seafile-db
2023-11-07 09:56:43,721 Repair repo is finished, 0 repos are corrupt, 0 repos are repaired.

/opt/seafile/seafile-server-latest# sudo -u seafile ./repair_repo.sh -r 92e20fec --force

mysql+pymysql://user:password@serverIP:3306/seafile-db
2023-11-07 09:57:00,564 Repair repo is finished, 0 repos are corrupt, 0 repos are repaired.
#12

Hello, it seems that you didn’t enter the completed repo_id. you can get the complete repo_id from the VirtualRepo table based on the prefix of the repo_id and then fix it using the script.

#13

Hello @feiniks,
As the repo_id where not visible in SeaHub I thought that virtual repos had shorter IDs…
The repair script did repair the repos and so far, no more error message.
I will post a message to sum up the solution when I will be sure that the problem is gone.
Thanks a lot !

#14

Hello @feiniks,

Thanks again for your help, the problem is solved.
If other users bump into this problem, here is how to solve it.

Problem
Fileserver.log is filling up with this kind of error :

[2023-11-06 10:36:03] merge repo 67b3e90a-d failed: cannot find seafdir for origin repo 9d2d15fa-5 path /Path1

How to solve
1. Find the complete repo_id
Log in to your database and search for the complete virtual repo_id by executing the following query :

SELECT * FROM `seafile-db`.VirtualRepo WHERE repo_id LIKE "%67b3e90a%";

2. Get the repair_repo script
Download it from Seafile Cloud
Copy both files (python and bash) to /opt/seafile/seafile-server-latest (if it is your install directory…)
Make the script executable by using chmod +x repair_repo.sh

3. Repair the virtual repository
Launch the following command onto your Seafile server :
sudo -u seafile ./repair_repo.sh -r 67b3e90a-wwww-xxxx-yyyy-zzzzzzzzzzz
(using your full virtual repo_id)

And that’s it !

I hope the repair script will be integrated in the next release.
An automated way to find the full repo_id would be nice as well :wink: