Slow sync of many small files


I’m running a Seafile Pro server with docker behind a nginx proxy. It’s super fast for larger files, but I do have some folders with a few 100.000 files (including some git repositories…) that are terribly slow to sync. I noticed several things:

  • uploading the file list takes quite some time, and half of the time the log shows:
    [02/17/19 09:32:57] http-tx-mgr.c(1070): libcurl failed to POST xxx/seafhttp/repo/88126475-f5b3-423a-9acc-e826278e2fc2/recv-fs/: Timeout was reached.
    [02/17/19 09:32:57] http-tx-mgr.c(3763): Failed to send fs objects for repo 88126475.
    after about 10 minutes of syncing. I tried to find any 10 minute timeout in the nginx or seafile config files, but I can’t find any.
  • when it finally uploads some of the files, it seems to create one http (1.1) request per file, which is really slow. So while I get limited to 10MB/s because of the connection for larger files, for this folder I only get around 100 KB/s.

Is this normal behavior or have I misconfigured something?

Thanks for you help!

For me it looks like the server is too slow. So far I had a good experience syncing many files.

Well, tbh it’s a 1 vCPU with 2GB RAM, but memory and CPU usage stays below 20% at all times…

Only 2 GB RAM could be a problem.

What about the disk load?