Seafile SYNC Client on DFS

[gauburtin]

Hi,

I don’t know if it is related, but i now see conflict reports on my DFS (Microsoft Synchronisation Center).
Somes Seafile files are not synchronized

\Documents\Seafile\accounts.db
\Documents\Seafile\repo.db

Have a look at the image, too.

Then i reboot the PC.
The DFS errors have disappeared.
But the Seafie SYNC process hangs on indexing.

Here is the seafile.log at startup

[11/13/17 17:53:49] seaf-daemon.c(558): starting seafile client 6.1.3
[11/13/17 17:53:49] seaf-daemon.c(560): seafile source code version 5fc440fe04370308d8dd9de4b1c63a388da68278
[11/13/17 17:53:49] ../common/mq-mgr.c(60): [mq client] mq cilent is started
[11/13/17 17:53:50] ../common/mq-mgr.c(106): [mq mgr] publish to heartbeat mq: seafile.heartbeat
[11/13/17 17:53:50] wt-monitor-win32.c(565): GetQueuedCompletionStatus failed, error code 995[11/13/17 17:53:50] wt-monitor-win32.c(565): GetQueuedCompletionStatus failed, error code 87[11/13/17 17:53:52] sync-mgr.c(702): Repo '_Projets' sync state transition from 'synchronized' to 'committing'.

Then I wait 10 minutes.
The SYNC process is complete
The logs finally traces :

    [11/13/17 18:00:45] sync-mgr.c(702): Repo '_Projets' sync state transition from 'committing' to 'uploading'.
    [11/13/17 18:00:45] http-tx-mgr.c(3423): Upload with HTTP sync protocol version 1.
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'init') --> ('normal', 'check')
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'check') --> ('normal', 'commit')
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'commit') --> ('normal', 'fs')
    [11/13/17 18:00:45] sync-mgr.c(702): Repo '_Info' sync state transition from 'synchronized' to 'committing'.
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'fs') --> ('normal', 'data')
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'data') --> ('normal', 'update-branch')
    [11/13/17 18:00:45] http-tx-mgr.c(1132): Transfer repo '022257ce': ('normal', 'update-branch') --> ('finished', 'finished')
    [11/13/17 18:00:45] sync-mgr.c(702): Repo '_Projets' sync state transition from 'uploading' to 'initializing'.
    [11/13/17 18:00:45] sync-mgr.c(1516): Removing blocks for repo _Projets(022257ce).
    [11/13/17 18:01:25] sync-mgr.c(702): Repo '_Info' sync state transition from 'committing' to 'initializing'.
    [11/13/17 18:01:26] sync-mgr.c(702): Repo '_Dossiers' sync state transition from 'synchronized' to 'committing'.
    [11/13/17 18:01:31] repo-mgr.c(3728): All events are processed for repo 4a338242-c020-4285-8e33-592e8f786dd3.
    [11/13/17 18:01:31] sync-mgr.c(702): Repo '_Dossiers' sync state transition from 'committing' to 'initializing'.
    [11/13/17 18:01:31] sync-mgr.c(702): Repo '_Info' sync state transition from 'synchronized' to 'committing'.
    [11/13/17 18:02:08] sync-mgr.c(702): Repo '_Info' sync state transition from 'committing' to 'initializing'.
    [11/13/17 18:02:09] sync-mgr.c(702): Repo '_Projets' sync state transition from 'synchronized' to 'committing'.

Repo _Projets is now synced. But what a mess !

My conclusion is : don’t sync big libraries on DFS !

Files 35422
Size	7,3 Go

Regards,

Gautier

[shoeper]

@gauburtin

Then the process hangs on indexing files so i can set the option (1 second)

I don’t think 1 second is a good idea because it’ll scan the files all the day (so there will be load on the DFS all the time).

[shoeper]

@gauburtin

I don’t know if it is related, but i now see conflict reports on my DFS (Microsoft Synchronisation Center).Somes Seafile files are not synchronized

\Documents\Seafile\accounts.db
\Documents\Seafile\repo.db

It could be. To fix it place the Seafile directory out of DFS. There is a hidden directory which is heavily being used. You can still sync libraries on DFS.

[gauburtin]

@shoeper

It could be.

Yes it does.

The DFS SYNC Process can not acces the files because Seafile Sync porcess is using them.

I’ll try with your local drive option for seafile folder.

Regards

[gauburtin]

@shoeper

To fix it place the Seafile directory out of DFS

Hi,

I did as you told me and placed the Seafile directory on a local drive not SYNCED With DFS.
I do not encounter any problem anymore.
I think that it could be mentioned during the client Installation process and on the manual.
Actually, there is not a lot of information no best pratices about the client.

What is you opinion, @Jonathan ?

Regards,

Gautier

[gauburtin]

@gauburtin

What is you opinion, @Jonathan ?

Hi,

I think that you may implement a defaut sync interval for all libraries, at the top of the client parameters.

It could be useful to prevent sync errors between Seafile and DFS.

It looks like settng a default sync interval is also possible on DFS via Group Policy.

Regards,

Gautier

[shoeper]

@gauburtin

I think that you may implement a defaut sync interval for all libraries,

For all locations where inotify works reliably (which might not be easy to find out automatically) it is a waste of resources.

[gauburtin]

@shoeper

For all locations where inotify works reliably (which might not be easy to find out automatically) it is a waste of resources.

I’m not sure i understood. For our case, All the documents of the Windows logged user are in the DFS Folder, so are the libraries.

[gauburtin]

@shoeper

To fix it place the Seafile directory out of DFS

Hi,

I think i found another issue :

Sync between network share drive (CIFS) and seafile

Hi, In our architecture, our users local documents are located in a CIFS Share (Windows Documents Shortcut points to \\server$homes\user\documents CIFS Path) In this configuration, syncing is very slow. The process begins by indexing during more than 20 mn on a 8Gb Library. The CPU process is very high [image] Regards, Gautier

In our case, syncing huge libraries located in a CIFS/DFS user home folder is hanging a lot.

This clearly makes syncing unusable, whereas the Seafile default install folder is located on a local drive.

regards

[shoeper]

I’m pretty sure the issues involved in such a setup have been discussed multiple times. Inotify (a mechanism to get notified when a file changed) does not work with CIFS/DFS, thus the client needs to rescan the whole library over and over again (using the sync interval). That can be a resource hungry process and is likely going to take more and more time the more files there are.

[gauburtin]

@shoeper

thus the client needs to rescan the whole library over and over again (using the sync interval)

Hi,

This is exactly what happens. And it could also be the reason why the process is so long on huge libraries (>5Go).
Tiis is a major issue for us. I hope this can be fixed in future releases of the client, but i’m not very optimitic.
Are you, @daniel.pan ?

[daniel.pan]

The client has to scan all the folder, checking the timestamp for all the files. It does not read the actual file content. So the performance depends on the number of files. This can’t be improved in the future.

[shoeper]

To solve the problem the only thing you can do is directly run the Seafile client on the fileserver.

[gauburtin]

@shoeper

To solve the problem the only thing you can do is directly run the Seafile client on the fileserver.

But then, the distributed core architecture of Seafile is compromised.

@daniel.pan

So the performance depends on the number of files

The solution would rather be to limit the number of files in a library, which is, in a user perspective, very complicated…

I’m very sorry to learn that

[shoeper]

@gauburtin

The solution would rather be to limit the number of files in a library, which is, in a user perspective, very complicated…

I’m very sorry to learn that

It is only important for the sync interval and remote file systems. I do synchronize more than half a million files from a laptop without issues.

@gauburtin

But then, the distributed core architecture of Seafile is compromised.

I don’t really get what you mean by this. What do you mean by its distributed core architecture?

[gauburtin]

@shoeper

What do you mean by its distributed core architecture

I mean that if clients can’t synchronize but only the server does, the cloud model is gone…

[shoeper]

It is not really the case. It is still the client which is syncing. In the best case synchronization always happens from the machine the disk is in. The client was always developed to synchronize local file systems.

[gauburtin]

@daniel.pan

The client has to scan all the folder, checking the timestamp for all the files. It does not read the actual file content. So the performance depends on the number of files. This can’t be improved in the future.

Hi @daniel.pan @shoeper

I’ve read this on HP Servers documentation

Performance Tuning using Change Notify
This section describes performance tuning using the Change Notify feature and internationalization.
NOTE: Starting with the Samba 3.0.25 version, the Change Notify Timeout feature is deprecated.
The Change Notify Timeout feature is replaced with the Change Notify feature. This new feature
depends on Linux iNotify, which is not available in HP-UX operating systems.
The Samba Server supports a new feature called Change Notify. Change Notify provides the
ability for a client to request notification from the server when changes occur to files or subdirectories
below a directory on a mapped file share. When a file or directory which is contained within the
specified directory is modified, the server notifies the client. The purpose of this feature is to keep
the client screen display up-to-date in Windows Explorer. The result: if a file you are looking at in
Windows Explorer is changed while you are looking at it, you will see the changes on the screen
almost immediately.
The only way to implement this feature in Samba is to periodically scan through every file and
subdirectory below the directory in question and check for changes made since the last scan. This
is a resource intensive operation which has the potential to affect the performance of Samba as
well as other applications running on the system. Two major factors affect how resource intensive
a scan is: the number of directories having a Change Notify request on them, and the size of those
directories. If you have many clients running Windows Explorer (or other file browsers) or if you
have directories on shares with a large number of files and/or subdirectories, each scan cycle
might be very CPU intensive.

I’m very concerned about his issue because most of our users start using the client while their data is on CIFS shares.

Is there any tuning that can be done :

• on the client (Other protocol than Inotifiy ?)
• on the server (CIFS / SMB on a DELL R530 + Windows Server 2016)

[update from SYNC]
https://help.resilio.com/hc/en-us/articles/207755736-Sync-and-SMB-file-shares

Notifications

Sync subscribes to OS notifications to get immediate updates about file changes and start delivering changed files. However, if a synced folder is located on SMB share, it may not be possible to subscribe to file update notifications. Only SMB 3.0 and newer supports notifications (which means, that both your client and server should have SMB 3.0 to get working notifications). If Sync is unable to get notifications, it will only detect file changes during full folder rescan.

Microsoft introduced SMB 3.0 in Windows 8, though it may be installed to your OS with Microsoft Updates (or via any other updates if you are using non-MS OS)

Windows 2016 Server uses SMB 3.1.1, is there any workaround ?

Regards,

Gautier

[daniel.pan]

The client will periodically scan the folder CIFS if sync interval is set. The performance depends on the number of files. If you have only a few thousands of files, you don’t need to worry about the performance.

[gauburtin]

@daniel.pan

The client will periodically scan the folder CIFS if sync interval is set.

Hi, thank you Daniel,

Then i should set Sync interval to 60 seconds and limit the number of files.

Because is depends of user parameters, it may be a potential issue and leads users thinking the client is not working good.

I’ll keep in mind the problem and i also recommand you to think about implemetaing a different event handling process compatible with CIFS, is possible.

Regards,

Gautier