Using Seafile Drive windows client, sync speed is always slower compared to standard network-mapped drive (Samba) transfer. Shouldn't Seafile be just as fast?

recklessnl · February 9, 2019, 1:26am

I used the standard Docker install using the manual online. Everything works perfectly.
I’m running this test on a Proxmox VM host running Debian 9.7. This has Docker installed and also a standard Samba file share in order to compare results. So my server runs Proxmox -> barebones Debian 9 VM --> Docker. This same VM also has a Samba share that I have access to and mapped as a network drive (similar to Seadrive).
Using Wifi on my laptop, when I upload a 2GB file to my Samba share (so from Windows 10 locally to the network mapped drive on the same Proxmox VM that is running Seafile on Docker) I get 55 MB/s sustained write. This is the maximum Wifi speed I get using iperf performance benchmarks, so I know it’s maxing out my Wifi connection.
Using Seafile Drive (latest Windows 10, client from the Seafile website) I get a maximum sync speed of 38MB/s for the same file. Quite a bit slower.

This seems like a pretty big difference and I’m not sure why. It’s the same host, just Seafile vs a standard Samba share. Why is Seafile slower? Is there a server setting I need to tweak to improve the speed or this expected?

shoeper · February 9, 2019, 7:41am

Put 1000 small files in Samba and Seadrive.

tomservo · February 9, 2019, 8:17pm

I wouldn’t expect it to be as fast in that test. They are 2 different technologies designed around solving different, but related problems.

Seafile is doing a lot of stuff behind the scenes that samba isn’t doing. Samba was designed to be fast over local networks just streaming data to and from the server’s disk as quickly as old hardware could go (lots of the protocol was designed over 10 years ago). Samba wasn’t designed to be used on the internet and isn’t safe to expose to the internet.

Seafile is designed with over-the-internet links in mind. So it does things like transfer data over http (optionally https), locally cache files, and chop files up into chunks in a clever and somewhat complicated way, and deduplicate on those chunks. This means that other resources can easily become the bottleneck. The CPU at either end can more easily limit speed because there is so much more CPU work being done in seafile than in samba. But if you edit a large file with a slow network between you and the server (coffee shop wifi for example), seafile might feel much faster.

I tried a similar test with my seafile server which is also a Debian 9 VM, KVM and no Docker. Sorry, I don’t have windows to test from, so this is from my Linux desktop. Everything was capped by my old wifi router, so I did this over Ethernet. For a 2GB file
Smaba gets about 68MB/s.
SeaDrive took close to 100% of one CPU on my desktop while it was writing the file:
dd if=/dev/urandom of=drive_test_random2 oflag=sync bs=1M count=2048
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 86.5861 s, 24.8 MB/s
Then it started to send that to the server, which took 71 seconds, so about 29MB/s.

Now back to my point that the comparison doesn’t reveal that there is a problem. I copied that same file in under a new name on both samba and SeaDrive. Samba got the same 68MB/s. SeaDrive spent 68 seconds processing the file:
dd if=/tmp/drive_test_random2 of=drive_test_random3 oflag=sync bs=1M
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 68.3336 s, 31.4 MB/s
And then in about 5 seconds the file was on the server, which gives an effective speed of about 410MB/s.

How? I am speculating here, but I think it is because the file gets broken into chunks and deduplicated, SeaDrive didn’t have to send any part of the file, just a filename and list of blocks that make up the file. This means that if I edit that file to put different data in the middle it will only have to send that part:
dd if=/dev/urandom of=drive_test_random3 bs=1M skip=512 count=512 conv=notrunc oflag=sync
536870912 bytes (537 MB, 512 MiB) copied, 21.8161 s, 24.6 MB/s
The processing time is about the same, but then it uploaded that 512MB of changes in about 22 seconds, so at about 23MB/s, but 93MB/s if you count that as 2GB uploaded in 22 seconds.

So your speeds don’t seem bad at all. It just different tech with different trade-offs.

recklessnl · February 21, 2019, 6:17am

Just so I’m clear - this is expected behavior? Seafile can’t match Samba speeds for sequential writes?

For bigger file systems this seems like a let-down. I’m sure that it’s faster with smaller files, but I didn’t know it was actually slower compared to Samba using bigger files.

tomservo · March 14, 2019, 6:43am

I am not one of the developers so I could be wrong but; Yes this is expected behavior. Seafile/Seadrive is optimized for bandwidth constrained situations. It has traded spending a lot more CPU time and memory usage to get lower bandwidth and disk space requirements.