[seafdav] transmitting large file fails

Seeing loads of issues with uploading large (2.5G+) files via webdav client.

At 06:15 a directory sync task is started from an Android 14 client (FolderSync Pro). Smaller files sync fine, large file deletions on seafile side are completed successfully, but 2 large files fail to upload - file nodes are on seafile server, but both have 0 bytes.

In client logs, 502 responses from server are logged exactly 20minutes after the sync task started:

INFO/dk.tacit.android.foldersync 526 Oct 28, 2024 06:35:04 OkHttp: <-- 502 https://seafile.mydomain.eu/seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup (1200952ms)
INFO/dk.tacit.android.foldersync 527 Oct 28, 2024 06:35:04 OkHttp: <-- 502 https://seafile.mydomain.eu/seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup (1200237ms)
INFO/dk.tacit.android.foldersync 528 Oct 28, 2024 06:35:04 OkHttp: server: nginx
INFO/dk.tacit.android.foldersync 529 Oct 28, 2024 06:35:04 OkHttp: server: nginx
INFO/dk.tacit.android.foldersync 530 Oct 28, 2024 06:35:04 OkHttp: date: Mon, 28 Oct 2024 05:35:02 GMT
INFO/dk.tacit.android.foldersync 531 Oct 28, 2024 06:35:04 OkHttp: date: Mon, 28 Oct 2024 05:35:02 GMT
INFO/dk.tacit.android.foldersync 532 Oct 28, 2024 06:35:04 OkHttp: content-type: text/html
INFO/dk.tacit.android.foldersync 533 Oct 28, 2024 06:35:04 OkHttp: content-type: text/html
INFO/dk.tacit.android.foldersync 534 Oct 28, 2024 06:35:04 OkHttp: content-length: 150
INFO/dk.tacit.android.foldersync 535 Oct 28, 2024 06:35:04 OkHttp: content-length: 150
INFO/dk.tacit.android.foldersync 536 Oct 28, 2024 06:35:04 OkHttp: strict-transport-security: max-age=63072000; includeSubDomains; preload
INFO/dk.tacit.android.foldersync 537 Oct 28, 2024 06:35:04 OkHttp: strict-transport-security: max-age=63072000; includeSubDomains; preload
INFO/dk.tacit.android.foldersync 538 Oct 28, 2024 06:35:04 OkHttp: cache-control: no-transform
INFO/dk.tacit.android.foldersync 539 Oct 28, 2024 06:35:04 OkHttp: cache-control: no-transform
INFO/dk.tacit.android.foldersync 540 Oct 28, 2024 06:35:04 OkHttp: content-security-policy: upgrade-insecure-requests; frame-ancestors 'self'
INFO/dk.tacit.android.foldersync 541 Oct 28, 2024 06:35:04 OkHttp: content-security-policy: upgrade-insecure-requests; frame-ancestors 'self'
INFO/dk.tacit.android.foldersync 542 Oct 28, 2024 06:35:04 OkHttp: permissions-policy: interest-cohort=()
INFO/dk.tacit.android.foldersync 543 Oct 28, 2024 06:35:04 OkHttp: permissions-policy: interest-cohort=()
INFO/dk.tacit.android.foldersync 544 Oct 28, 2024 06:35:04 OkHttp: referrer-policy: same-origin
INFO/dk.tacit.android.foldersync 545 Oct 28, 2024 06:35:04 OkHttp: referrer-policy: same-origin
INFO/dk.tacit.android.foldersync 546 Oct 28, 2024 06:35:04 OkHttp: x-content-type-options: nosniff
INFO/dk.tacit.android.foldersync 547 Oct 28, 2024 06:35:04 OkHttp: x-content-type-options: nosniff
INFO/dk.tacit.android.foldersync 548 Oct 28, 2024 06:35:04 OkHttp: x-frame-options: SAMEORIGIN
INFO/dk.tacit.android.foldersync 549 Oct 28, 2024 06:35:04 OkHttp: x-frame-options: SAMEORIGIN
INFO/dk.tacit.android.foldersync 550 Oct 28, 2024 06:35:04 OkHttp: x-ua-compatible: IE=Edge
INFO/dk.tacit.android.foldersync 551 Oct 28, 2024 06:35:04 OkHttp: x-ua-compatible: IE=Edge
INFO/dk.tacit.android.foldersync 552 Oct 28, 2024 06:35:04 OkHttp: x-xss-protection: 1; mode=block
INFO/dk.tacit.android.foldersync 553 Oct 28, 2024 06:35:04 OkHttp: x-xss-protection: 1; mode=block
INFO/dk.tacit.android.foldersync 554 Oct 28, 2024 06:35:04 OkHttp: <-- END HTTP
INFO/dk.tacit.android.foldersync 555 Oct 28, 2024 06:35:04 OkHttp: <-- END HTTP
WARNING/dk.tacit.android.foldersync 556 Oct 28, 2024 06:35:04 OkHttp: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>

Looking at seafdav.log, I can see no errors. But also, there are no mentions of those 2 large files: signal-2024-10-27-05-10-13.backup nor signal-2024-10-28-05-13-07.backup.


Relevant logs from nginx side:
seafdav.access.log:

- client.ip.addy [28/Oct/2024:06:15:00 +0100] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 401 139 "-" "OkHttp" 0.002
- client.ip.addy [28/Oct/2024:06:15:00 +0100] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 207 591 "-" "OkHttp" 0.041
- client.ip.addy [28/Oct/2024:06:15:00 +0100] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 207 2250 "-" "OkHttp" 0.204
- client.ip.addy [28/Oct/2024:06:15:00 +0100] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/ HTTP/2.0" 207 3761 "-" "OkHttp" 0.060
- client.ip.addy [28/Oct/2024:06:15:01 +0100] "PROPFIND /seafdav/phone-bcf01b/config/signal/ HTTP/2.0" 207 1858 "-" "OkHttp" 0.052
- client.ip.addy [28/Oct/2024:06:15:01 +0100] "PUT /seafdav/phone-bcf01b/config/foldersync/2024-10-25_05-10%20-%20foldersync.db.zip HTTP/2.0" 201 372 "-" "OkHttp" 0.448
- client.ip.addy [28/Oct/2024:06:15:01 +0100] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/2024-10-25_05-10%20-%20foldersync.db.zip HTTP/2.0" 207 716 "-" "OkHttp" 0.036
- client.ip.addy [28/Oct/2024:06:15:02 +0100] "PUT /seafdav/phone-bcf01b/config/foldersync/2024-10-27_05-10%20-%20foldersync.db.zip HTTP/2.0" 201 372 "-" "OkHttp" 1.181
- client.ip.addy [28/Oct/2024:06:15:02 +0100] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/2024-10-27_05-10%20-%20foldersync.db.zip HTTP/2.0" 207 716 "-" "OkHttp" 0.033
- client.ip.addy [28/Oct/2024:06:15:02 +0100] "PUT /seafdav/phone-bcf01b/config/foldersync/2024-10-26_05-10%20-%20foldersync.db.zip HTTP/2.0" 201 372 "-" "OkHttp" 1.865
- client.ip.addy [28/Oct/2024:06:15:03 +0100] "PUT /seafdav/phone-bcf01b/config/foldersync/2024-10-28_05-10%20-%20foldersync.db.zip HTTP/2.0" 201 372 "-" "OkHttp" 1.982
- client.ip.addy [28/Oct/2024:06:15:03 +0100] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/2024-10-26_05-10%20-%20foldersync.db.zip HTTP/2.0" 207 716 "-" "OkHttp" 0.033
- client.ip.addy [28/Oct/2024:06:15:03 +0100] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/2024-10-28_05-10%20-%20foldersync.db.zip HTTP/2.0" 207 716 "-" "OkHttp" 0.032
- client.ip.addy [28/Oct/2024:06:35:02 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.932
- client.ip.addy [28/Oct/2024:06:35:02 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.211
- client.ip.addy [28/Oct/2024:06:55:03 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.188
- client.ip.addy [28/Oct/2024:06:55:03 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.189
- client.ip.addy [28/Oct/2024:07:15:05 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.039
- client.ip.addy [28/Oct/2024:07:15:05 +0100] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0" 502 150 "-" "OkHttp" 1200.127

seafdav.error.log:

2024/10/28 06:35:02 [error] 848#848: *807 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup", host: "seafile.mydomain.eu"
2024/10/28 06:35:02 [error] 848#848: *807 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup", host: "seafile.mydomain.eu"
2024/10/28 06:55:03 [error] 851#851: *915 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup", host: "seafile.mydomain.eu"
2024/10/28 06:55:03 [error] 851#851: *915 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup", host: "seafile.mydomain.eu"
2024/10/28 07:15:05 [error] 850#850: *1010 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-28-05-13-07.backup", host: "seafile.mydomain.eu"
2024/10/28 07:15:05 [error] 850#850: *1010 upstream prematurely closed connection while reading response header from upstream, client: client.ip.addy, server: seafile.*, request: "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup HTTP/2.0", upstream: "http://192.168.1.11:8080/seafdav/phone-bcf01b/config/signal/signal-2024-10-27-05-10-13.backup", host: "seafile.mydomain.eu"

Nginx seafile proxy config is here

Note /seafdav/ location defines proxy_read_timeout 1200s; which corresponds to 20min, so my guess is this is the timeout that’s triggering the 502.

But surely if webdav backend worked correctly this timeout should be nowhere near triggering? Also this is the exact timeout value as suggested by the seafile documentation.

Running Seafile 11.0.12

Also note it doesn’t reproduce consistently. Just now manually triggered same task ending up with upload of another 2 files of similar size and task completed successfully in ~12 minutes.

Can someone confirm the nginx proxy config looks OK?

Oh dear, this DAV things was the reason to move away from nextcloud to seafile because of sabredav nextcloud bugs never resolved for years like this Error: Sabre\DAV\Exception\BadRequest: Expected filesize of... · Issue #21382 · nextcloud/server · GitHub

i can’t give you the trick but search for chunk sizes, nginx, webdav, large files, it may helps you find some nginx settings to setup chunk sizes so the connection is transfered in junks. An example from a working Apache2 config, it may helps you find the similar nginx snippets:

  ### Allow huge file size uploads in my seafile environment
  SecRequestBodyLimit 3010720000
  SecRequestBodyNoFilesLimit 3010720000

RequestReadTimeout header=7200 body=7200
ProxyRequests Off
(...)
    #due to upload issues with sesions
    SetEnv proxy-nokeepalive 1
    SetEnv proxy-sendchunks 1

    <Location /seafdav>
        ProxyPass "http://127.0.0.1:10888/seafdav"
    </Location>