Quickest way to add large amounts of data into library of a local seafile instance

Hi there,
I am going to setup a Seafile Server locally, so I have physical access.

What would be the quickest way to add large amounts of data to my library (possibly without having to upload / sync it)? Is there any way to make use of HDDs or SSDs that I’d plug into the machine and start a copy process?

Or would it have to be more like creating a local 10Gig+ ethernet link and use the sync client?

Thanks for your ideas!

Hi,
you cannot simply copy the files into the seafile database folder, as seafile uses a special block system to facilitate delta syncing and speed-up syncing in general. That way it is MUCH faster compared to owncloud/Nextcloud as shown in several benchmarks and my personal experience.
So the best way is indeed to use a sync client, using the browser might be more cumbersome in case of browser crash etc. seafile client will continue where it stopped for e.g. during a workstation reboot.

You can also import files directly on the seafile server.
Internally this will probably work like a seafile client, but it might be a bit faster than a seafile client on another computer, because it is running directly on the seafile server.
Maybe someone here has already tested this?

seaf-import.sh is the script for directly importing

How about transferring the files using WebDAV?

I haven’t tested it myself yet, but I can’t really imagine webdav being faster.
Do you have other experiences?

As you suggest the Manual recommends the use the seaf-import.sh script for this.

usage :
seaf-import.sh
-p <import dir path, must set>
-n <repo name, must set>
-u <repo owner, must set>

It looks like a Pro feature, however. Is this a Pro server?

@ sehrgrossermeister, How much data are you talking about?

Hi there,
thanks a lot for your input!
I am talking about 100eds of Gigabytes of large Videofiles.
I am running the Community Edition.
How would transfer with webdav work like exactly?
Cheers and happy holidays!

Since you’re asking, I assume you have not added WebDAV to your server yet. It has to be installed in CE and is described here in the Manual. The exact hookup will depend on the server itself as described in the Manual.

A good general purpose WebDAV client for Mac and Windows is CyberDuck. It is shareware but offers an extended trial period if needed.

Just thinking out loud here, but considering it’s GBs not TBs of data we’re talking about, and the fact that it seems like you might be considering installing WebDAV (which you might not be interested in moving forward), I would run a quick test transferring, say, a tenth of the collection to a suitable Library in Seafile using the standard Desktop sync engine (which is quite excellent).

I would use a wired Ethernet connection if possible and get a fix on how much time and baby-sitting is required to complete the job. Assuming your server is in good working order, it’s quite possible that this offers a no-muss-no-fuss solution that you can turn your back on once started.

Thoughts?

Hi there,
both ways sound like an OK option.
But- I will have to setup the server first :smile: Currently I only have a cloud machine without local access.
So - now that I have 2 options, I might just setup a test server in my office.
After Christmas :wink:
Thanks again for your input. Will get back to this topic…

I recently copied 6TB+ of data from a NAS to a Seafile in anticipation of a 10Gb network. I used smb to reach us and seadrive to access the siafile. synced with the flameless program FreeFileSync, it worked fast enough. I hope my advice will help you, good luck!
*but unfortunately recently my library stopped working, I hope they will help me in the next branch, test the seafile before production

I haven’t tested it myself yet, but I can’t really imagine webdav being faster.