Seafile Pro 11 on docker: fileserver errors

Seafile Server
1

Hi,
two weeks ago I upgraded our Seafile server from standalone pro 9 to docker pro 11.0.17. Everything seems to be working fine, no users complained, so far. But the logfiles are full of fileserver errors. For example the seafevents.log shows many errors, like

[2025-03-19 15:57:34,509] [ERROR] error when handle msg: Failed to read object 70096414-ba08-4dc1-a1b2-717d72a4f529/16f37230acdc9ca15c145c208846fcd098b194dc: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/70096414-ba08-4dc1-a1b2-717d72a4f529/16/f37230acdc9ca15c145c208846fcd098b194dc'
FileNotFoundError: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/70096414-ba08-4dc1-a1b2-717d72a4f529/16/f37230acdc9ca15c145c208846fcd098b194dc'
seafobj.exceptions.GetObjectError: Failed to read object 70096414-ba08-4dc1-a1b2-717d72a4f529/16f37230acdc9ca15c145c208846fcd098b194dc: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/70096414-ba08-4dc1-a1b2-717d72a4f529/16/f37230acdc9ca15c145c208846fcd098b194dc'

But the files are definitely there (we use NFS for the data):

root@b9c2cc6a7b73:# ls -l /opt/seafile/seafile-data/storage/fs/70096414-ba08-4dc1-a1b2-717d72a4f529/16/f37230acdc9ca15c145c208846fcd098b194dc
-rw-------. 1 112 112 389 Mar 19 15:57 /opt/seafile/seafile-data/storage/fs/70096414-ba08-4dc1-a1b2-717d72a4f529/16/f37230acdc9ca15c145c208846fcd098b194dc

In seafile.log I see the corresponding error:

2025-03-19 15:57:34 ../common/fs-mgr.c(1937): [fs mgr] Failed to read dir 16f37230acdc9ca15c145c208846fcd098b194dc.
2025-03-19 15:57:34 ../common/diff-simple.c(141): Failed to find dir 70096414-ba08-4dc1-a1b2-717d72a4f529:16f37230acdc9ca15c145c208846fcd098b194dc.
2025-03-19 15:57:34 repo-perm.c(1004): Failed to calculate number of new file.
2025-03-19 15:57:34 ../common/fs-mgr.c(1937): [fs mgr] Failed to read dir 16f37230acdc9ca15c145c208846fcd098b194dc.
2025-03-19 15:57:34 ../common/diff-simple.c(141): Failed to find dir 70096414-ba08-4dc1-a1b2-717d72a4f529:16f37230acdc9ca15c145c208846fcd098b194dc.
2025-03-19 15:57:34 size-sched.c(344): [scheduler] failed to do diff.

When I run seaf-fsck -r on the library, no error is shown. What could be wrong? Thanks for any hint.
Dirk

2

Hello @uosseafile , from the output of ls, the owner of the fs file seems to be 112 instead of root, can you try to change the owner of the file to root by chown, and then restart the server to see if there is still a problem?

3

Hi feiniks,

yes, 112 is the uidnumber and gidnumber of the seafile admin we used in the standalone version of seafile 9. The data is on a nfs fileserver, where I defined a user mapping from root to the seafile user. When I create a new library with the docker version, the data belongs to uid/gid 112 on the fileserver:
drwxr-xr-x 4 112 112 40 Mar 19 16:51 a3bda46c-10af-42db-b9f5-ebc2b36364d3.

4

Hello, is the seafevents.log still showing “No such file or directory” errors now even though the file exists locally? Can you send the log for the current time?

5

There are only a handful of libraries where the error occurs, compared to the thousands of libraries stored in seafile. Here is a new error from this morning:

[2025-03-20 08:55:36,388] [ERROR] error when handle msg: Failed to read object 1425d2a9-23f0-415c-b884-72146aeac88d/b92bd46a76d086a2792f6a1e2630308aeef59921: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/1425d2a9-23f0-415c-b884-72146aeac88d/b9/2bd46a76d086a2792f6a1e2630308aeef59921'
Traceback (most recent call last):
  File "/opt/seafile/seafile-pro-server-11.0.17/seahub/thirdpart/seafobj/backends/base.py", line 14, in read_obj
    data = self.read_obj_raw(repo_id, version, obj_id)
  File "/opt/seafile/seafile-pro-server-11.0.17/seahub/thirdpart/seafobj/backends/filesystem.py", line 21, in read_obj_raw
    with open(path, 'rb') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/1425d2a9-23f0-415c-b884-72146aeac88d/b9/2bd46a76d086a2792f6a1e2630308aeef59921'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/seafile/seafile-pro-server-11.0.17/pro/python/seafevents/app/mq_handler.py", line 53, in handle_message
    func(config, session, msg)
  File "/opt/seafile/seafile-pro-server-11.0.17/pro/python/seafevents/events/handlers.py", line 55, in RepoUpdateEventHandler
    renamed_files, moved_files, renamed_dirs, moved_dirs = differ.diff()
  File "/opt/seafile/seafile-pro-server-11.0.17/seahub/thirdpart/seafobj/commit_differ.py", line 68, in diff
    dir2 = fs_mgr.load_seafdir(self.repo_id, self.version, new_id)
  File "/opt/seafile/seafile-pro-server-11.0.17/seahub/thirdpart/seafobj/fs.py", line 203, in load_seafdir
    data = self.obj_store.read_obj(store_id, version, dir_id)
  File "/opt/seafile/seafile-pro-server-11.0.17/seahub/thirdpart/seafobj/backends/base.py", line 20, in read_obj
    raise GetObjectError('Failed to read object %s/%s: %s' % (repo_id, obj_id, e))
seafobj.exceptions.GetObjectError: Failed to read object 1425d2a9-23f0-415c-b884-72146aeac88d/b92bd46a76d086a2792f6a1e2630308aeef59921: [Errno 2] No such file or directory: '/opt/seafile/seafile-data/storage/fs/1425d2a9-23f0-415c-b884-72146aeac88d/b9/2bd46a76d086a2792f6a1e2630308aeef59921'

But when I enter the docker container, the file is there:

root@363b629397d9:/opt/seafile# ls -l /opt/seafile/seafile-data/storage/fs/1425d2a9-23f0-415c-b884-72146aeac88d/b9/2bd46a76d086a2792f6a1e2630308aeef59921
-rw-------. 1 112 112 2408 Mar 20 08:55 /opt/seafile/seafile-data/storage/fs/1425d2a9-23f0-415c-b884-72146aeac88d/b9/2bd46a76d086a2792f6a1e2630308aeef59921

The file has the same timestamp as the error. So maybe the error occurs before the file is written to the nfs storage.
At the same time, the following error occurs in the seafile.log:

2025-03-20 08:55:36 ../common/diff-simple.c(251): Failed to find dir 1425d2a9-23f0-415c-b884-72146aeac88d:b92bd46a76d086a2792f6a1e2630308aeef59921.
2025-03-20 08:55:36 ../common/diff-simple.c(251): Failed to find dir 1425d2a9-23f0-415c-b884-72146aeac88d:b92bd46a76d086a2792f6a1e2630308aeef59921.
6

Hello @uosseafile , in such case, I also suspect that the service attempted to read before the write was persisted to the NFS. What are your current NFS mount options? Could you try modifying the options to enable synchronous writes and see if that helps?

7

I’m using automounter. Maybe that’s the problem. But I used the automounter on the non-docker version for years without problems.

The mount options are:
type nfs4 (rw, relatime, vers=4.0, rsize=1048576, wsize=1048576, namlen=255, hard, proto=tcp, timeo=600, retrans=2, sec=sys, clientaddr=x.x.x.x, local_lock=none, addr=y.y.y.y)

8

Hello, automounter is expected to automatically unmount when there is no access. When the NFS is unmounted, it may cause conflicts with Docker containers, leading to a period during which files cannot be accessed. This differs from operations performed directly on the host machine. When used inside a container, this issue may be amplified. You could try configuring the automounter to prevent automatic unmounting to confirm whether this is the cause of the problem.

9

Good idea. I will try this later.
Thank you, so far.

10

Hi,
I added the nfs mount options “noac” and “sync” to the seafile share and set the timeout to 3600 (1 hour) in autofs.conf. Now the errors have disappeared. :grinning:
Thanks @feiniks for the tip!