Indexing-Threads[Manual]

Hey Guys,

quick question. I’m running a seafile-server of version 6.1.2 on debian 9 with a Xeon 4-Core and 32gb RAM. Nginx only, https configured of course.

I’m into fine-tuning right now and already spotted the “max_indexing_threads = 10” option within the seafile.conf manual page quite often but never thought about modifying it.

Now i’m wondering if raising this amount is advisable and could actually bring me some performance tweaks.

The description of that option reads:

After a file is uploaded via the web interface, or the cloud file browser in the client, it needs to be divided into fixed size blocks and stored into storage backend. We call this procedure “indexing”. By default, the file server uses 1 thread to sequentially index the file and store the blocks one by one. This is suitable for most cases. But if you’re using S3/Ceph/Swift backends, you may have more bandwidth in the storage backend for storing multiple blocks in parallel. We provide an option to define the number of concurrent threads in indexing:

Which Bandwidth we are talking about here?
Ok i’m actually asking this even my system only got 2x Raid-1 HDD’s which are very unhappy with many I/O processes but does anybody here got any experience with that or even played around with it while not having a S3/Ceph/Swift backend?
Any suggestions are welcome :slight_smile:

Thanks!

#Push
No one?

S3/Swift/Ceph are distributed storage systems. Often there is some latency involved in saving an object to disk which is way higher than storing a file on the local disk while the write capacity in total can be very high (e.g. 1000 servers with lots of bandwith and thousands of disks). In such a scenario files uploaded via Seahub could only be indexed/uploaded very slow because most time the server would wait for an ack that the object has been saved before storing the next one and the process can be strongly accerlerated by increasing the amount of threads being used to store the objects. E.g. 100 is going to store 100 objects simoultaniously which would signifcantly improve the speed in such a case as these 100 objects would like be saved to different servers and while some threads do wait for an ack of the backend others actually utilize the available bandwidth to store the next objects.

In your case with raid 1 I’m not sure if it would increase the speed. You could try it with two but a high number would not be of any benefit, but could lead to an unresponsive servers due to waste of resources.

1 Like

As said in the manual, this option is mainly for use with S3 storage backends. In your case, I think increasing max_indexing_threads to 2 may help speed up the indexing, but changing to a larger very will not help.