Still getting '502 Bad Gateway' and timeouts after upgrading to 6.2.2

Seafile Server
1

Hi. I’m still having trouble getting Seafile server to work after upgrading to 6.2.2 (6.2.x). It’s running on a Raspberry Pi B+, Arch Linux ARM, and has been working and upgrading without major hiccups ever since 5.x.x. The current version of the webserver is nginx 1.12.1.

While the desktop client (on a GNU/Linux laptop) auto-syncs files, it keeps loading the libraries indefinitely, and it’s impossible to access the Seafile server through the website; it either throws a ‘502 Bad Gateway’ error message or also keeps loading indefinitely.
Nginx also serves Radicale, but that website works however.

Since I have been fiddling around with the settings they are not 100% like they were in the previous working state (in the older versions), because I was trying to accomodate the new recommended settings (to use WSGI).

ccnet.conf:

[General]
USER_NAME = ginger
ID = 123456a123456b124356c
NAME = Ginger
SERVICE_URL = https://ginger.bread.com:8001

[Network]
PORT = 10001

[Client]
PORT = 13418

seahub_settings.py:

SECRET_KEY = ‘fwjff(RTJ#(8323r803f38#(=")#(=DJD83483’
FILE_SERVER_ROOT = ‘https://ginger.bread.com:8001/seafhttp’
SEAFILE_VERSION= ‘6.2.2’

/etc/nginx/nginx.conf:

#user html;
user seafile;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/servers-enabled/*;
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       80;
        #server_name  localhost;
        server_name  ginger.bread.com;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }

        # PHP-FPM
        location ~ \.php$ {
             try_files $uri $document_root$fastcgi_script_name =404;
             fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
             fastcgi_index index.php;
             fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
             include fastcgi.conf;
        }



    }

}

/etc/nginx/servers-available/seahub:

server {
    listen 8001; # <--------------------------------------- NGINX PORT
    ssl on; # <-------------------------------------------- SSL
    ssl_certificate /etc/nginx/ssl/seahub.crt; # <--------- SSL
    ssl_certificate_key /etc/nginx/ssl/seahub.key; # <----- SSL
    server_name ginger.bread.com; # <---------------- CHANGE THIS
    error_page 497  https://$host:$server_port$request_uri;

    location / {
	proxy_pass          http://127.0.0.1:8000;
        proxy_set_header    Host $host;
        proxy_set_header    X-Real-IP $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Host $server_name;
        proxy_read_timeout  1200s;

        # used for view/edit office file via Office Online Server
        #client_max_body_size 0;

        access_log      /var/log/nginx/seahub.access.log;
        error_log       /var/log/nginx/seahub.error.log;
    }
    location /seafhttp {
        rewrite ^/seafhttp(.*)$ $1 break;
        proxy_pass http://127.0.0.1:8082;
        client_max_body_size 0;

    }

    location /media {
        root /home/seafile/ginger/seafile-server/seahub; # <-- change: 2015-08-27
        include /etc/nginx/mime.types; # <--- UNCOMMENT THIS IF CSS FILES AREN'T LOADED
    }
}

UPDATE 2019-02-17:
Seems like the culprit was a faulty installation, which might have arisen after the distribution packages changed. After deleting some folders related to seahub and redeploying it, using a script that came with the distribution, seahub finally started working again (!!!). Meanwhile I tried different settings in nginx as well, so now I’m not entirely sure whether they helped or didn’t make any impact. Anyhow, thanks alot for helping out, @jobenvil ! I’m so happy that it’s working again.

2

@Captain_Rage your problem could be here:

location / {
proxy_pass          http://127.0.0.1:8000;
    proxy_set_header    Host $host;

should be changed to

location / {
proxy_pass          http://127.0.0.1:8000;
    proxy_set_header    Host $host:8001;

Because you are listening for seafile in other port than the 80:[quote=“Captain_Rage, post:1, topic:4223”]
SERVICE_URL = https://ginger.bread.com:8001
[/quote]

A detailed explanation could be found here.

Edit: Insert also the directive proxy_http_version 1.1;
inside location / and location /seafhttp block

3

Thanks for the suggestion. I tried it, but unfortunately to no avail. :disappointed: Today I went through different iterations and permutations of all configuration files I could find but the results ended up inconclusive. I tried to install php-fpm but that didn’t help either (assuming I configured it properly).

This is what shows when systemd starts seafile-server:

 CGroup: /system.slice/system-seafile\x2dserver.slice/seafile-server@gingerbread.service
           |-16989 seafile-controller -c /home/seafile/gingerbread/ccnet -d /home/seafile/gingerbread/seafile-data -F /home/seafile/gingerbread/conf
           |-16990 ccnet-server -F /home/seafile/gingerbread/conf -c /home/seafile/gingerbread/ccnet -f /home/seafile/gingerbread/logs/ccnet.log -d -P /home/seafile/gingerbread/seafile-data/pids/ccnet.pid
           |-16992 seaf-server -F /home/seafile/gingerbread/conf -c /home/seafile/gingerbread/ccnet -d /home/seafile/gingerbread/seafile-server/runtime/seahub.conf -b 0.0.0.0:8000
           |-17008 python2.7 /usr/lib/seafile/seafileenv/bin/gunicorn seahub.wsgi:application -c /home/seafile/gingerbread/seafile-server/runtime/seahub.conf -b 0.0.0.0:8000
           |-17192 python2.7 /usr/lib/seafile/seafileenv/bin/gunicorn seahub.wsgi:application -c /home/seafile/gingerbread/seafile-server/runtime/seahub.conf -b 0.0.0.0:8000
           |-17193 python2.7 /usr/lib/seafile/seafileenv/bin/gunicorn seahub.wsgi:application -c /home/seafile/gingerbread/seafile-server/runtime/seahub.conf -b 0.0.0.0:8000
           `-17194 python2.7 /usr/lib/seafile/seafileenv/bin/gunicorn seahub.wsgi:application -c /home/seafile/gingerbread/seafile-server/runtime/seahub.conf -b 0.0.0.0:8000


Oct 13 22:50:52 Unicorn systemd[1]: Starting Next-generation open source cloud storage with advanced features on privacy protection and teamwork....
Oct 13 22:51:46 Unicorn seafile-admin[16984]: Starting seafile-server...
Oct 13 22:51:46 Unicorn seafile-admin[16984]: Starting seahub...
Oct 13 22:51:46 Unicorn seafile-admin[16984]: Seahub running on port 8000
Oct 13 22:51:46 Unicorn seafile-admin[16984]: Done

It claims it runs on port 8000, although I set the port to 8001 in ccnet.conf and seahub_settings.py. If I recall systemd has always been displaying 8000, so it is probably not relevant.

Not sure what I should be trying next. Maybe other users that are utilizing a Raspberry Pi B+, and nginx as web server, could post their ccnet.conf, seahub_settings.py, nginx.conf and systemd configuration files? It might be helpful for troubleshooting. Thanks.

4

Please post the output of netstat -tulpen

Why you don’t change your configuration to work on 443? Do you need to run your seafile server on port 8001? This is not really needed if you don’t have anything running on port 443. Due to the fact that many people configured it on 8001 instead of 443, is not really a security issue. Security through obscurity is here not achieved.

5

I see something weird:

gunicorn should be delivered by thirdpart libraries directory:

python2.7 /media/sda/seafile/seafile-server-6.2.2/seahub/thirdpart/gunicorn seahub.wsgi:application -c /media/sda/seafile/seafile-server-6.2.2/runtime/seahub.conf -b 0.0.0.0:8000 --preload

6

The only reason is that I don’t know any better because I’m a profound novice when it comes to these things (webservers and such, that is). :disappointed: Do you mean deleting ‘:8001’ from ccnet.conf and seahub_settings.py altogether and changing something to ‘443’ in nginx.conf?
Until now I’ve mostly been comparing my configuration files with the ones of other users and trying to deduct how things work.

Here is the output from the netstat command:

[root@Pony nginx]# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name    
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN      195        12153      239/systemd-resolve 
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      0          24134      1217/nginx: master  
tcp        0      0 192.168.0.20:5232       0.0.0.0:*               LISTEN      992        13277      285/python          
tcp        0      0 192.168.0.20:6544       0.0.0.0:*               LISTEN      994        12811      231/taskd           
tcp        0      0 0.0.0.0:10001           0.0.0.0:*               LISTEN      0          16555      409/ccnet-server    
tcp        0      0 0.0.0.0:8082            0.0.0.0:*               LISTEN      0          16573      411/seaf-server     
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      0          12133      238/vsftpd          
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          12170      240/sshd            
tcp       17      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      0          16674      427/python2.7       
tcp        0      0 0.0.0.0:8001            0.0.0.0:*               LISTEN      0          24133      1217/nginx: master  
tcp        0      0 0.0.0.0:12001           0.0.0.0:*               LISTEN      0          16570      411/seaf-server     
tcp        0      0 0.0.0.0:8200            0.0.0.0:*               LISTEN      997        12739      232/minidlnad       
tcp6       0      0 :::5355                 :::*                    LISTEN      195        12156      239/systemd-resolve 
tcp6       0      0 :::22                   :::*                    LISTEN      0          12177      240/sshd            
tcp6       0      0 :::64738                :::*                    LISTEN      122        12770      229/murmurd         
udp        0      0 239.255.255.250:1900    0.0.0.0:*                           997        12738      232/minidlnad       
udp        0      0 192.168.0.20:123        0.0.0.0:*                           0          12510      251/ntpd            
udp        0      0 127.0.0.1:123           0.0.0.0:*                           0          12508      251/ntpd            
udp        0      0 0.0.0.0:123             0.0.0.0:*                           0          12504      251/ntpd            
udp        0      0 192.168.0.20:35758      0.0.0.0:*                           997        22889      232/minidlnad       
udp        0      0 0.0.0.0:5355            0.0.0.0:*                           195        12152      239/systemd-resolve 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           195        12183      239/systemd-resolve 
udp        0      0 192.168.0.20:68         0.0.0.0:*                           193        22884      199/systemd-network 
udp6       0      0 fe80::ba27:ebff:fe7:123 :::*                                0          12514      251/ntpd            
udp6       0      0 ::1:123                 :::*                                0          12512      251/ntpd            
udp6       0      0 :::123                  :::*                                0          12501      251/ntpd            
udp6       0      0 :::64738                :::*                                122        12773      229/murmurd         
udp6       0      0 :::5355                 :::*                                195        12155      239/systemd-resolve 
udp6       0      0 fe80::ba27:ebff:fe7:546 :::*                                193        11171      199/systemd-network

EDIT:
When looking through the nginx error logs:

[root@Pony nginx]# tail /var/log/nginx/seahub.error.log
2017/10/15 00:40:06 [error] 8956#8956: *16 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.14, server: ginger.bread.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "192.168.0.20:8001"
7

don’t worry, we will reconfigure it. The old configuration had the :8001 at the end, but later it was changed to be more easy and work all the traffic over 443 or 80, both seahub and seafile.

Where you have :8001 in ccnet.conf and seahub_settings.py, you should delete it. Then change the 8001 by 443 in nginx. Don’t forget to delete the :8001 in the seafile blocks.

There is something wrong with your seaf-server over port 12001, this should not be there. Can you post seafile.conf and seahub_settings.py without the passwords? The other seaf-server running on 8082 is correct.

Comment # the [Network] and PORT = 10001 in ccnet.conf because this is not anymore necessary.

8

That’s reassuring, thanks. :slight_smile: And thanks for the explanations!

ccnet:

[General]
USER_NAME = ginger
ID = 499445b4b5dd7bade6m78m23v2
NAME = Ginger
SERVICE_URL = https://ginger.bread.com

seahub_settings.py:

SECRET_KEY = '-%#C""¤C¤6ju&k+5%hV%V%V&3m6^9&&BQB'
FILE_SERVER_ROOT = 'https://ginger.bread.com/seafhttp'
SEAFILE_VERSION= '6.2.2'

seafile.conf:

[network]
port=12001

[fileserver]
host=0.0.0.0
port=8082

# Set maximum upload file size to 4000M.
max_upload_size=4000

# Set maximum download directory size to 4000M.
max_download_dir_size=4000

/etc/nginx/servers-available/seahub:

server {
    listen 443; # <------------
    ssl on; # <-------------------------------------------- SSL
    ssl_certificate /etc/nginx/ssl/seahub.crt; # <--------- SSL
    ssl_certificate_key /etc/nginx/ssl/seahub.key; # <----- SSL
    server_name ginger.bread.com; # <---------------- CHANGE THIS
    error_page 497  https://$host:$server_port$request_uri;

    location / {
	proxy_pass          http://127.0.0.1:8000;
        proxy_set_header    Host $host;
        proxy_set_header    X-Real-IP $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Host $server_name;
        proxy_read_timeout  1200s;
	proxy_http_version 1.1;

        # used for view/edit office file via Office Online Server
        #client_max_body_size 0;

        access_log      /var/log/nginx/seahub.access.log;
        error_log       /var/log/nginx/seahub.error.log;
    }

    location /seafhttp {
        rewrite ^/seafhttp(.*)$ $1 break;
        proxy_pass http://127.0.0.1:8082;
        client_max_body_size 0;
	proxy_http_version 1.1;

    }

    location /media {
        root /home/seafile/gingerbread/seafile-server/seahub; # <-- change: 2015-08-27
        include /etc/nginx/mime.types; # <--- UNCOMMENT THIS IF CSS FILES AREN'T LOADED
    }

    #location ~ \.php$ {
    #    try_files $uri $document_root$fastcgi_script_name =404;
    #    fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
    #    fastcgi_index index.php;
    #    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    #    include fastcgi.conf;
    #}

}

Once I change the port to 443 (from 8001), will I have to change the address in the clients to “adress:443” instead of the old “adress:8001”, since it says in the clients to also provide adress with the port?

Now I also realize that the way Seahub is provided changed a while ago (on Arch Linux you used to have to download run the upgrade scripts manually, but now Seahub is provided as a standalone package in the AUR; Arch User Repository), and now I’m not sure which one the system uses since I have Seahub installed both explicitly as a package as well as the old folders.
This is most certainly bound to cause problems, and that is sooner rather than later. Too bad I didn’t think about this earlier. :frowning:

I probably should not have ran seahub-preupgrade manually ever since Seahub also got installed through the package manager.

9

here is something wrong. I don’t know why you have this configuration but this is the error that I already observed with netstat. You must delete the port=12001 and the [network] clausel. You can put a route (#) in front of host=0.0.0.0 because this is the default. Only if you want to exchange it to other, you should enable it with new IP.

Yes, this is probably why I observed that you use django not from the thirdpart directory, instead from a lib directory.

Edit: The manual is your best friend :slight_smile:

1 Like