Seafdav - MOVE command causing 502

Hey @saman so yeah, the reverse proxy fix helped. Joplin started working again, but Foldersync on Android was indeed still failing, so something is definitely still off.

My Foldersync sends the complete file with .tacitpart extension just fine, but it fails when it’s trying to remove the extension (the MOVE).

Disabling “Use temp-file scheme” also works for me, but I do not consider this a fix.

A simple rename in Thunar file explorer fails:

Log: x.x.x.x 172.20.0.4 [28/Apr/2020:23:48:28 +0200] "MOVE /seafdav/Android/test.txt HTTP/1.1" 502 435 "-" "gvfs/1.42.2" 0.080

As @Gabri said, there’s definitely an issue with the MOVE operation over webdav.

Yes, MOVE operations are broken. Other operations (e.g. PUT, GET, PROPFIND, HEAD, …) seem to work.

Indeed, that is not a fix. It’s worth noting, again, that this has nothing to do with temporary files or what WebDAV client you are using. The only reason it shows up when using temporary files is because temporary files make heavy use of MOVE commands.

Upgraded to 7.1.4 hoping it could have been fixed (even if not reported in changelog) but without luck :disappointed:

@Gabri @saman My suggestion: SeafDAV is based basically on WebDAV. You could open an issue to check if this problem is already known upstream.

In order to debug this issue, you must run the command from CLI:

Stop seafile && seahub services and then run:

su seafile -c "PYTHONPATH=$PYTHONPATH:/media/sda/seafile/seafile-server-latest/seahub/thirdpart/:/media/sda/seafile/seafile-server-latest/seafile/lib/python3.6/site-packages/ CCNET_CONF_DIR=/media/sda/seafile/ccnet SEAFILE_CONF_DIR=/media/sda/seafile/seafile-data SEAFILE_CENTRAL_CONF_DIR=/media/sda/seafile/conf SEAFDAV_CONF=/media/sda/seafile/conf/seafdav.conf SEAFILE_RPC_PIPE_PATH=/media/sda/seafile/seafile-server-7.1.3/runtime /usr/bin/python3 -m wsgidav.server.server_cli --server gunicorn --root / --log-file /media/sda/seafile/logs/seafdav.log -v --pid /media/sda/seafile/pids/seafdav.pid --port 8080 --host 0.0.0.0 --verbose"

and substitute your system paths accordingly.

The trick is done by the flag --verbose. Check then the seafdav.log

Tried with your command but I get this error:

  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/seafile/seafile-server-latest/seahub/thirdpart/wsgidav/server/server_cli.py", line 44, in <module>
    from wsgidav.dc.domain_controller import SeafileDomainController
  File "/opt/seafile/seafile-server-latest/seahub/thirdpart/wsgidav/dc/domain_controller.py", line 4, in <module>
    from pysearpc import SearpcError
ModuleNotFoundError: No module named 'pysearpc'

Anyway here @Jonathan says that in 7.1 they had done

major upgrade to the webdav functionality

So I guess is something more on Seafile side than wsgidav

What I’m guessing:

The major upgrade means that until R7.1. SeafDAV was based on wsgidav=“1.2.0.pre” and after R7.1. SeafDAV is based on wsgidav=3.0.1. The major changes didn’t happen on Seafie but on the original wsgidav, in which SeafDAV is based on.

/home/seafile # diff seafile-server-7.0.5/seahub/thirdpart/wsgidav/version.py seafile-server-7.1.4/seahub/thirdpart/wsgidav/_version.py
< __version__ = "1.2.0.pre"
...
...

> __version__ = "3.0.1"

And here the whole ChangeLog from wsgidav, where you can observe the release version and date.

So, there are more possibilities than the MOVE issue is comming upstream from wsgidav than from Seafile itself. But like @shoeper already commented, could not be at all a general wsgidav/seafdav issue, instead a specific one.

I also had an issue with Keepass2Android not reliably uploading a changed KDBX to my Seafile, it also ended up with having a 502.
So I played around a little bit and connected to Webdav through Ubuntu’s file manager (Nautilus) and didn’t see any obvious issue with renaming or moving files on Seafile 7.1.4 Pro.
Any specific tests I can do to see if the issue also exists here?

The Keepass issue is somehow related to the builtin okhttp server they use, this has some issues with http2 so I disabled this on my nginx and it’s reliable again.

Hi, just to confirm, I can also see the 502 errors here with the MOVE operations, when my FolderSync is uploading and renaming files. I’m running 7.1.3, but there is nothing in the changelog about WebDav in 7.1.4.

Hi, the corresponding bug in the wsgidav is 183. Unfortunately, I can’t add links here, but it looks like we need to rewrite the Destination header in nginx (or patch wsgidav).

@Erik_Tews A couple of weeks ago I was reading every issue in wsgidav and already applied the workaround, here is how it looks like nginx in my case for seafdav

    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Host $server_name;

    proxy_set_header   X-Forwarded-Proto $scheme; # wsgidav issue 183 https://github.com/mar10/wsgidav/issues/183

    client_max_body_size 0;
    proxy_connect_timeout  36000s;
    proxy_read_timeout  36000s;
    proxy_send_timeout  36000s;
    send_timeout  36000s;
    proxy_http_version                 1.1;

    # This option is only available for Nginx >= 1.8.0. See more details below.
    proxy_request_buffering off;      # works well with mp4
    proxy_buffering off;

But even that, the js could not be served properly. The pictures (wsgidav logo) are missing and not properly displayed the pages in chrome browser. Just going without nginx in between works fine. If someone has hacked that (regex in nginx) to display properly the pages, go ahed pls. and share it.

Hi, I’m sorry, but as far as I understand the source code of wsgidav, the X-Forwarded-Proto header is never evaluated, so adding this header doesn’t help. There was a fix suggested in the bug to also evaluate the X-Forwarded-Proto header, but then another solution (rewriting the Destination) header was proposed, but that is more difficult with nginx.

Hi,
i use apache 2.4.38 as reverse proxy and did also get a 502 error when the webdav client from my KeePass try to initiate a MOVE.

Try to set the X-Forwarded-Proto header in my vhost config, but could not get it to work.

RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}

This is the link to the bug in wsgidav: https://github.com/mar10/wsgidav/issues/183
Anybody else maybe has a hint or idea to fix this or where to raise a bug report?

In reading the issue and examining the code the problem is that WSGIDav examines the requested scheme (http or https) vs the actual scheme. In otherwords, when you contact the server you use https://server.org. This request is SSL terminated by the NGINX reverse proxy which forwards the request as http://127.0.0.1:8080. The headers passed along all indicate a scheme of HTTPS, but the actual communication is on HTTP (non-SSL). The WSSGIDav code sees the incongruity and rejects the request with the 502 error.

This is correct behavior according to the standard. The issue is the code should be checking the X-Forwarded-Proto and then act accordingly, but currently does not. So adjusting the X-Forwarded-Proto will have no affect at the moment.

The only other options are to rewrite the headers so it looks like the original request was HTTP or to enable HTTPS on the WebDav backend - neither of which is trivial. Or you just do not use a reverse proxy.

As I have time I’ll research options and see if a solution presents itself.

Adding something like this in the nginx vhost should rewrite the Destination header :

      set $destination $http_destination;
      if ($destination ~* ^https(.+)$) {
         set $destination http$1;
      }
      proxy_set_header Destination $destination;

If using Apache,

    RequestHeader edit Destination ^https: http:

You will need to enable mod_headers.

I can confirm that modifying the Destination header like so fixes the problem.

Have a tested working copy for Nginx (updated slightly):

add the following outside of the server context:

map $http_destination $nossl_destination {
"~^https:(.+)$" $1;
"~^http:(.+)$" $1;
}

then in location /seafdav add

proxy_set_header Destination "http:$nossl_destination";

Explanation:
The nginx map function operates by “mapping” a value to a variable based on a string (or other variable). In Nginx http headers are stored as variables with prefix $http_ and lowercase name of the header. Hence we are looking at the Destination header and storing a value based on what is there in $nossl_destination. The regex expression inside the map captures the “https:” part of the Destination header (we capture the : so we can evaluate for http as well as https). Everything after https is stored in an accessible variable called “$1” which we indicate as the value to be used for $nossl_destination by adding whitespace and stating it. We also have a value for straight http (based on configuration this may not be necessary but it is a good safety). Now in our location /seafdav we simply substitute a new Destination header with the “proxy_set_header Destination” and add http:$nossl_destination and everything should work.

There are various ways to write the regex and you can also use something akin to what @dani posted.

10 Likes

thanks! this is the only solution that worked for me

Confirmed working with @eruditewriter solution, thank you!