Webdav S3-compatibile https backend is not working

bug

#1

Hi,

I run my backend on minio through NGINX.
seafile-pro-server-6.3.4
When on HTTP all is OK (but I obviously do not want it).
However, as soon as I switch to HTTPS, webdaw is reporting Python error page.
the rest seems to be owrking just fine.

A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/flup/server/fcgi_base.py in run(self=<flup.server.fcgi_base.Request object>)
    556         """Runs the handler, flushes the streams, and ends the request."""
    557         try:
=>  558             protocolStatus, appStatus = self.server.handler(self)
    559         except:
    560             traceback.print_exc(file=self.stderr)
protocolStatus undefined, appStatus undefined, self = <flup.server.fcgi_base.Request object>, self.server = <flup.server.fcgi.WSGIServer object>, self.server.handler = <bound method WSGIServer.handler of <flup.server.fcgi.WSGIServer object>>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/flup/server/fcgi_base.py in handler(self=<flup.server.fcgi.WSGIServer object>, req=<flup.server.fcgi_base.Request object>)
   1118                 result = self.application(environ, start_response)
   1119                 try:
=> 1120                     for data in result:
   1121                         if data:
   1122                             write(data)
data undefined, result = <generator object __call__>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/wsgidav_app.py in __call__(self=<wsgidav.wsgidav_app.WsgiDAVApp object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<function start_response>)
    395         # Call next middleware
    396         app_iter = self._application(environ, _start_response_wrapper)
=>  397         for v in app_iter:
    398             yield v
    399         if hasattr(app_iter, "close"):
v undefined, app_iter = <generator object __call__>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/debug_filter.py in __call__(self=<wsgidav.debug_filter.WsgiDavDebugFilter object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<function _start_response_wrapper>)
    152         app_iter = self._application(environ, sub_app_start_response)
    153 
=>  154         for v in app_iter:
    155             # Start response (the first time)
    156             if first_yield:
v undefined, app_iter = <generator object __call__>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/error_printer.py in __call__(self=<wsgidav.error_printer.ErrorPrinter object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<wsgidav.util.SubAppStartResponse object>)
     40                 # Otherwise the we could not catch exceptions here. 
     41                 response_started = False
=>   42                 app_iter = self._application(environ, sub_app_start_response)
     43                 for v in app_iter:
     44                     # Start response (the first time)
app_iter undefined, self = <wsgidav.error_printer.ErrorPrinter object>, self._application = <wsgidav.http_authenticator.HTTPAuthenticator object>, environ = {'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, sub_app_start_response = <wsgidav.util.SubAppStartResponse object>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/http_authenticator.py in __call__(self=<wsgidav.http_authenticator.HTTPAuthenticator object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<wsgidav.util.SubAppStartResponse object>)
    193                 return self.sendBasicAuthResponse(environ, start_response)
    194             elif authmethod == "basic" and self._acceptbasic:
=>  195                 return self.authBasicAuthRequest(environ, start_response)
    196 
    197             util.log("HTTPAuthenticator: respond with 400 Bad request; Auth-Method: %s" % authmethod)
self = <wsgidav.http_authenticator.HTTPAuthenticator object>, self.authBasicAuthRequest = <bound method HTTPAuthenticator.authBasicAuthReq...dav.http_authenticator.HTTPAuthenticator object>>, environ = {'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response = <wsgidav.util.SubAppStartResponse object>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/http_authenticator.py in authBasicAuthRequest(self=<wsgidav.http_authenticator.HTTPAuthenticator object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<wsgidav.util.SubAppStartResponse object>)
    236             environ["http_authenticator.realm"] = realmname
    237             environ["http_authenticator.username"] = username
=>  238             return self._application(environ, start_response)
    239         return self.sendBasicAuthResponse(environ, start_response)
    240         
self = <wsgidav.http_authenticator.HTTPAuthenticator object>, self._application = <wsgidav.dir_browser.WsgiDavDirBrowser object>, environ = {'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response = <wsgidav.util.SubAppStartResponse object>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/dir_browser.py in __call__(self=<wsgidav.dir_browser.WsgiDavDirBrowser object>, environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<wsgidav.util.SubAppStartResponse object>)
    171 #                # sort: 0:"calls",1:"time", 2: "cumulative"
    172 #                profile.print_stats(sort=2)
=>  173             return self._listDirectory(davres, environ, start_response)
    174         
    175         return self._application(environ, start_response)
self = <wsgidav.dir_browser.WsgiDavDirBrowser object>, self._listDirectory = <bound method WsgiDavDirBrowser._listDirectory of <wsgidav.dir_browser.WsgiDavDirBrowser object>>, davres = RootResource('/'), environ = {'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response = <wsgidav.util.SubAppStartResponse object>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/dir_browser.py in _listDirectory(self=<wsgidav.dir_browser.WsgiDavDirBrowser object>, davres=RootResource('/'), environ={'CONTENT_LENGTH': '', 'CONTENT_TYPE': '', 'HTTPS': 'on', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5', 'HTTP_AUTHORIZATION': 'Basic YWRtaW5Aa2lzaWVsLm5ldC5wbDpwcmVkaWN0IHVuYWZyYWlkIHN1cnZpdm9yIGNoYWluIHdyZWNrYWdlIHB1cmlmaWVy', 'HTTP_CACHE_CONTROL': 'max-age=0', 'HTTP_HOST': 'seafile.kisiel.net.pl', 'HTTP_SCHEME': 'https', ...}, start_response=<wsgidav.util.SubAppStartResponse object>)
    261             # No pre-build info: traverse members
    262             dirInfoList = []
=>  263             childList = davres.getDescendants(depth="1", addSelf=False)
    264             for res in childList:
    265                 di = res.getDisplayInfo()
childList undefined, davres = RootResource('/'), davres.getDescendants = <bound method RootResource.getDescendants of RootResource('/')>, depth undefined, addSelf undefined, builtin False = False
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/dav_provider.py in getDescendants(self=RootResource('/'), collections=True, resources=True, depthFirst=False, depth='1', addSelf=False)
    445             res.append(self)
    446         if depth != "0" and self.isCollection:
=>  447             for child in self.getMemberList():
    448                 if not child:
    449                     _ = self.getMemberList()
child undefined, self = RootResource('/'), self.getMemberList = <bound method RootResource.getMemberList of RootResource('/')>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py in getMemberList(self=RootResource('/'))
    494             if len(r_list) == 1:
    495                 repo = r_list[0]
=>  496                 res = self._createRootRes(repo, repo.name)
    497                 member_list.append(res)
    498             else:
res undefined, self = RootResource('/'), self._createRootRes = <bound method RootResource._createRootRes of RootResource('/')>, repo = <pysearpc.client._SearpcObj object>, repo.name = 'photos'
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py in _createRootRes(self=RootResource('/'), repo=<pysearpc.client._SearpcObj object>, name='photos')
    505 
    506     def _createRootRes(self, repo, name):
=>  507         obj = get_repo_root_seafdir(repo)
    508         return SeafDirResource("/"+name, repo, "", obj, self.environ)
    509 
obj undefined, global get_repo_root_seafdir = <function get_repo_root_seafdir>, repo = <pysearpc.client._SearpcObj object>
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py in get_repo_root_seafdir(repo=<pysearpc.client._SearpcObj object>)
    618 
    619 def get_repo_root_seafdir(repo):
=>  620     root_id = commit_mgr.get_commit_root_id(repo.id, repo.version, repo.head_cmmt_id)
    621     return fs_mgr.load_seafdir(repo.store_id, repo.version, root_id)
    622 
root_id undefined, global commit_mgr = <seafobj.commits.SeafCommitManager object>, commit_mgr.get_commit_root_id = <bound method SeafCommitManager.get_commit_root_id of <seafobj.commits.SeafCommitManager object>>, repo = <pysearpc.client._SearpcObj object>, repo.id = u'46584b3b-69bc-4380-9e4c-927d82b0022e', repo.version = 1, repo.head_cmmt_id = u'3c502857ec5a293b0819f83ac2ca804d4bdd09dd'
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/seafobj/commits.py in get_commit_root_id(self=<seafobj.commits.SeafCommitManager object>, repo_id=u'46584b3b-69bc-4380-9e4c-927d82b0022e', version=1, commit_id=u'3c502857ec5a293b0819f83ac2ca804d4bdd09dd')
     55 
     56     def get_commit_root_id(self, repo_id, version, commit_id):
=>   57         commit = self.load_commit(repo_id, version, commit_id)
     58         return commit.root_id
     59         
commit undefined, self = <seafobj.commits.SeafCommitManager object>, self.load_commit = <bound method SeafCommitManager.load_commit of <seafobj.commits.SeafCommitManager object>>, repo_id = u'46584b3b-69bc-4380-9e4c-927d82b0022e', version = 1, commit_id = u'3c502857ec5a293b0819f83ac2ca804d4bdd09dd'
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/seafobj/commits.py in load_commit(self=<seafobj.commits.SeafCommitManager object>, repo_id=u'46584b3b-69bc-4380-9e4c-927d82b0022e', version=1, obj_id=u'3c502857ec5a293b0819f83ac2ca804d4bdd09dd')
     41             else:
     42                 data = self.obj_stores['__default__'].read_obj(repo_id, version, obj_id)
=>   43         return self.parse_commit(data)
     44 
     45     def parse_commit(self, data):
self = <seafobj.commits.SeafCommitManager object>, self.parse_commit = <bound method SeafCommitManager.parse_commit of <seafobj.commits.SeafCommitManager object>>, data = ''
 /mnt/mfs/sfroot/seafile-pro-server-6.3.4/seahub/thirdpart/seafobj/commits.py in parse_commit(self=<seafobj.commits.SeafCommitManager object>, data='')
     44 
     45     def parse_commit(self, data):
=>   46         dict = json.loads(data)
     47         d = {}
     48         for k, v in dict.iteritems():
builtin dict = <type 'dict'>, global json = <module 'json' from '/usr/lib/python2.7/json/__init__.pyc'>, json.loads = <function loads>, data = ''
 /usr/lib/python2.7/json/__init__.py in loads(s='', encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw={})
    337             parse_int is None and parse_float is None and
    338             parse_constant is None and object_pairs_hook is None and not kw):
=>  339         return _default_decoder.decode(s)
    340     if cls is None:
    341         cls = JSONDecoder
global _default_decoder = <json.decoder.JSONDecoder object>, _default_decoder.decode = <bound method JSONDecoder.decode of <json.decoder.JSONDecoder object>>, s = ''
 /usr/lib/python2.7/json/decoder.py in decode(self=<json.decoder.JSONDecoder object>, s='', _w=<built-in method match of _sre.SRE_Pattern object>)
    362 
    363         """
=>  364         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    365         end = _w(s, end).end()
    366         if end != len(s):
obj undefined, end undefined, self = <json.decoder.JSONDecoder object>, self.raw_decode = <bound method JSONDecoder.raw_decode of <json.decoder.JSONDecoder object>>, s = '', idx undefined, _w = <built-in method match of _sre.SRE_Pattern object>, ).end undefined
 /usr/lib/python2.7/json/decoder.py in raw_decode(self=<json.decoder.JSONDecoder object>, s='', idx=0)
    379         try:
    380             obj, end = self.scan_once(s, idx)
    381         except StopIteration:
=>  382             raise ValueError("No JSON object could be decoded")
    383         return obj, end
builtin ValueError = <type 'exceptions.ValueError'>

<type 'exceptions.ValueError'>: No JSON object could be decoded
      args = ('No JSON object could be decoded',)
      message = 'No JSON object could be decoded'

#2

Do you use wsgiDAV or fastcgi?


#3

According to WebDAV config it’s fastcgi
wrong? I think I followed the docu on seafile website


#4

The one is outdated. Not for sure if it’ll solve your S3<->WebDAV problem, but I thin it’s worth to try it. You can find some examples in the forum.

Edit: Search here, in the community manual: https://seafile.readthedocs.io/en/latest/installation/reverseproxy-webserver/#configuring-the-server-for-using-nginx


#5

I am currently trying to move my data from S3 to local storage. Technically the same disk space but available through fuse. Migration is still running.


#6

I can confirm that after switching away of fastcgi all seems to be working.
thank you @bionade24

I guess I am going to stay with minio afterall


#7

No Problem. Ask if you have other problems. And please mark this thread as solved.


#8

I think I was celebrating too early. I forgot to change seafile.conf to point to ssh
it still does not work, although this time, the page in the browser is blank :frowning:

so I was trying to switch to my newly migrated local storage.
It looks like I can read it but cannot write to it.
Trying to play with permissions but it’s 600GB so takes a while
Do I need special permissions for this folder?


#9
  1. Just export seafile-data, not evrything and then change the path for seafile-data. Maybe you did it, but I’m not sure.
  2. You always have to the change the permissions of new file systems in Linux. Make it with chmod a+rwX /mountpoint (You need the big X on some distributions for some fs types, if it gives and error back just type the normal chmod a+rwx.

#10

looks like I run migration script as root.
It was enough to change ownership to my seafile user and it seems to be fine.


#11

So at the moment, I am not using s3 backend via ssh but I believe bug still remains.
Shall I leave this topic open?


#12

Cold you try another tunneling technic? Of course, leave it open. You should search trough the forum and Github for similar issues, if you find nothing you should open an issue on Github.


#13

At the moment I just want for this to start working. I already spent too much time tinkering, so I probably won’t check tunnelling. There are still 10TB photos waiting to be loaded to Seafile.


#14

What’s the pupose for WebDAV? You know how solw WebDAV is and the seafile extra is not like a raw WebDAV service.


#15

in my (not very scientific) tests rclone through webDAV was faster than sync on cli client.
I guess I simply run a couple of webDAV sessions together.
I actually read about the slowness of webDAV later so maybe will give the sync another go.

I liked rsync more because it gives me visual information where I am in my transfer.
And also gives me more control over what is synced where.

What’s the easiest way to see the sync progress and possibly transfer speed, so I can do a better comparison?
Also, is there a way to force one-way sync in the seafile?


#16

rsync(Protocol) > SeaFile(Protocol) > WebDAB

rsync ist just faster with the rsync protocol, maybe just indexing tooks long. And for sure, it’s no good way to load 10TB files into SeaFile over WebDAV. For this, you should rsync them onto your server with rsync protocol, and the use seaf-import.sh script. That’s the only way for so much files.


#17

do I understand correctly that seaf-import.sh script would require 10TB to temporary store the files + 10TB to store them later in Seafile?

at the moment I installed seadrive for linux. Using rclone my upload speed is above 8MByte/s.
Uploaded 30GB with 10GB buffer so I hope I stopped measuring buffer speed already.
That’s better than good, though I have no idea what my speed directly to the storage would be.


#18

direct rclone to the same network storage hoovers around 1MByte/s
I am not sure what’s happening, but I do hope everything works with seadrive :slight_smile:


#19

If you have everything near, you just can mount the drive with the photos into your server. Edit: If not at home with sftp.

BAD idea! It’s beta and has a catastrophically caching.
Maybe you just should describe your problem exactly in a new topic, there will be ideas for you


#20

It’s a distributed storage over a few data centers around Europe, so not really an option to mount locally.
seaf-cli transfer rates hoover around 300KB/s
webDAV is 3-4 times faster via rclone and in the same range as direct rclone to network storage.
seadrive is rubbish, 9MB written out of 30GB claimed.

I wonder what I do wrong.
webDAV is supposed to be slow and so far offers the fastest transfers for me.
I wonder why native seaf-cli is so slow; it’s supposed to be faster from WebDAV