Build Seafile RPI 7.1.x with Seahub starting problems

Seafile Server
, ,
1

I want to make a build for Ubuntu-Server 20.04 for Raspberry PI arm64, as the [relaeses (7.1.4)](github .com/haiwen/seafile-rpi/releases) haven’t worked.

The build runs without errors: github .com/haiwen/seafile-rpi/blob/master/build3.sh
But when I want to start seahub (e.g. with ./seahub.sh start-fastcgi 8081), I get this error:

File "/home/pi/seafile/seafile-server-7.1.4/seahub/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/thirdpart/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/thirdpart/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/thirdpart/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/thirdpart/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/thirdpart/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/seahub/base/models.py", line 13, in <module>
    from seahub.utils import calc_file_path_hash, within_time_range, \
  File "/home/pi/seafile/seafile-server-7.1.4/seahub/seahub/utils/__init__.py", line 25, in <module>
    from django.core.urlresolvers import reverse

ModuleNotFoundError: No module named 'django.core.urlresolvers'

I looked up the used sources and ther seahub-6.0.1 is used, where [this fix](github .com/haiwen/seahub/pull/4620) isn’t present and therefore throws the error.

Then I tried to set the [seahub version](github .com/haiwen/seafile-rpi/blob/master/build3.sh#22) in build3.sh to the current 7.1.4, but that did’t update the resources.

Any ideas, how to make seahub working, preferably by upgrading seahub to 7.1.4?

Related topics:
forum.seafile .com/t/error-seahub-failed-to-start/2101/5
forum.seafile .com/t/seahub-startet-nicht/12012
github .com/haiwen/seafile-rpi/issues/64

2

From Seafile ChangeLog:

In version 6.3, Django is upgraded to version 1.11. Django 1.8, which is used in version 6.2, is deprecated in 2018 April.

With this upgrade, the fast-cgi mode is no longer supported. You need to config Seafile behind Nginx/Apache in WSGI mode.

The way to run Seahub in another port is also changed. You need to modify the configuration file conf/gunicorn.conf instead of running ./seahub.sh start <another-port> .

Do you want to install seafile or do you need to update from lower versions?

3

Sorry I am confused as hack. I noticed, that Seahub 7.1.4 is already used, despite the version in built-seafile-sources which states 6.0.1
But anyways Seahub 7.1.4 doesn’t include upgrade of Django to 2.2. So I tried to build with Seahub 7.1.5, which was’t successful for my system.

Then I tried installing Django 1.11 via pip3, cause I thought it would remove the ModuleNotFoundError: No module named 'django.core.urlresolvers' for 7.1.4, but didn’t. How can I resolve this error?

BTW: I used fast-cgi mode only for getting the error message. Is there another method to debug?

And even more confusing: I checked that the port is defined in gunicorn.conf, but it further showed the default port 8000 and only switched it with this style: ./seahub.sh start <another-port>.

Background: I ported my RaspberryPI 3 from arm32 to arm64, as some packages aren’t supported in 32 bit, I have to use. Before I ran 7.0.5 seafile-rpi-server, the upgrade to 7.1.4 already was successful, and seafile is running. Only seahub is making problems somehow…

Thank you for your patience and your help :smiley: Best regards.

4

I now modified my gunicorn.conf.py to show errors with daemon = False. The error was now displayed with ./seahub.sh start but the message kept the same, so don’t think it has sth to do with fast-gci :slight_smile:

5

Yes, this is already known. Even if you compile seafile v 7.1.4 you get the v6.0.1 zip/tar file. That’s a bug in the packaging.It’s already commented on the build3.sh . Don’t worry, the produced name of the package is wrong (6.0.1) but the content is the correct (7.1.4).

That’s again a good catch, and again, you are right with your assumptions. I strugled last week already with the same issue, ending up with your issue django.core.urlresolvers . How?:I tried to build Seafile 7.1.5 and use the master branch for Seahub, instead of use the tag v7.1.5, since this has Django 2.2 from last commits. This could work in other case, BUT, the problem are the wrong or unprecisely requirements.tx, that are resulting in an impossible situation, after the compilation. Compiling master tag in seahub and requirements.txt from master ended up in such an error: Missing staticfiles manifest entry for '%s'" % clean_name

When you compile the thirdpartmodules you get such a warning:

djangorestframework 3.12.1 requires django>=2.2, but you'll have django 1.11.29 which is incompatible.

This is because the seahub master branch has between other requirements:

Django==2.2.14
...
djangorestframework==3.11.1

but in 7.1.5 they are:

Django==1.11.29
...
djangorestframework   # here no version specification

If you don’t specify the lower version in djangorestframework (3.11.1), pip will take the upper version (3.12.1) which only works with django 2.2, but the code is still for django until 1.11.29.

The solution is to force pip to install djangorestframework==3.11.1 and Django==1.11.29. I put these requirements here on my repo.

This is again the same problem with the requirements.

Yes, I imagined it, therefore I asked you for the update or new installation. Thanks for clarifying. Your seahub issue has nothing to do with fast-cgi

Try to stick to the tag v7.1.5 when compiling and pull my seahub requirements.txt

6

Wow that’s lots of information! Thank you very much and helped me a lot understanding what’s going on.
First I tried compiling with 7.1.5 tag and your seahub requirements, but that didn’t eliminate the Django error.

The compiling worked then with these constants in build3.sh and seahub is starting somehow (with timeout). Edit: forgot to set back to daemon = True in gunicorn.conf.py

VERSION=7.1.5
VERSION_TAG=v$VERSION-server
PYTHON_REQUIREMENTS_URL_SEAHUB=https://raw.githubusercontent.com/jobenvil/rpi-build-seafile/main/seahub_requirements_v7.1.5.txt
PYTHON_REQUIREMENTS_URL_SEAFDAV=https://raw.githubusercontent.com/jobenvil/rpi-build-seafile/main/seafdav_requirements_v7.1.5.txt

VERSION_TAG=master Edit: 7.1.5 is sufficient.

PYTHON_REQUIREMENTS_URL_SEAFDAV=seafdav/requirements_SeafDAV-7.1.5.txt at master · jobenvil/seafdav · GitHub
 Edit: used not the raw file :see_no_evil:

But then I wanted to try a new setup, before migrating and the old image. But then seafile/bin/seaf-server-init was missing from package. I think you experienced a similar issue here:

How did you solve it? Greets ;D

Edit: seafile/pids/seafdav.pid is also missing :confused:

7

After using python virtual environment with python 3.6, I can compile the 7.1.5 version without using master, but seafile/bin/seaf-server-init is further missing. Seahub is partially starting but then freezes after the log:

Oct 13 14:20:38 ubuntu systemd[1]: seahub.service: start operation timed out. Terminating.
Oct 13 14:20:38 ubuntu seahub.sh[2142]: [2020-10-13 14:20:38 +0000] [2142] [INFO] Worker exiting (pid: 2142)
Oct 13 14:20:38 ubuntu seahub.sh[2138]: [2020-10-13 14:20:38 +0000] [2138] [INFO] Handling signal: term
Oct 13 14:20:39 ubuntu seahub.sh[2138]: [2020-10-13 14:20:39 +0000] [2138] [INFO] Shutting down: Master
Oct 13 14:20:40 ubuntu systemd[1]: seahub.service: Failed with result 'timeout'.
Oct 13 14:20:40 ubuntu systemd[1]: Failed to start Seafile hub.

The logs/seahub.log is completely empty, and the other logs don’t have any information relevant for Seahub in my opinion.

Edit: Never mind, forgot to set back daemon = True, so process will continue in terminal.

8

Now I get in both versions master and 7.1.5 the error you told me, although I used your resources in the PYTHON_REQUIREMENTS_URL_SEAHUB variable.

2020-10-13 15:27:01,769 [ERROR] django.request:222 log_response Internal Server Error: /accounts/login/
Traceback (most recent call last):
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/utils/decorators.py", line 142, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/seahub/auth/views.py", line 197, in login
    return render(request, template_name, {
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/shortcuts.py", line 36, in render
    content = loader.render_to_string(template_name, context, request, using=using)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 171, in render
    return self._render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/loader_tags.py", line 150, in render
    return compiled_parent._render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/templatetags/static.py", line 106, in render
    url = self.url(context)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/templatetags/static.py", line 103, in url
    return self.handle_simple(path)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/templatetags/static.py", line 118, in handle_simple
    return staticfiles_storage.url(path)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/contrib/staticfiles/storage.py", line 153, in url
    return self._url(self.stored_name, name, force)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/contrib/staticfiles/storage.py", line 132, in _url
    hashed_name = hashed_name_func(*args)
  File "/home/pi/seafile/seafile-server-7.1.5/seahub/thirdpart/django/contrib/staticfiles/storage.py", line 420, in stored_name
    raise ValueError("Missing staticfiles manifest entry for '%s'" % clean_name)
ValueError: Missing staticfiles manifest entry for 'scripts/lib/jquery.min.js'
9

@jobenvil Do I have to compile the thirdpartmodules in a separate process? And if so: how?

Sorry I don’t know how to resolve this staticfiles issue.

Although I found an interesting solution, but it requires rewriting the lib:


Can you publish the build3.sh for 7.1.5 on your repository?
Or is there any other way to get Seafile-RPI 7.1.4 or 7.1.5 for arm64? Any help is appreciated. Thx

10

After many frustrating hours I was able to build a running server application (Ubuntu 20.04 Focal Fossa arm64).

@jobenvil Thank you for the guidance.

11

:v:I know very well that feeling. For the last release migration I spent many hours fixing here and there…

@Gustl22 why you don’t contribute here? You could upload your release there. I plan to upload the armv7 releases (stretch, buster, bionic, focal) tomorrow . What do you think?

12

Basically the build3.sh script should do the job, but due to the requirements issue that you already know, I use the following one to compile succesfully seafile-server. I run that script in the containers to build seafile.

You can create x4 lxc containers (stretch, buster, bionic focal) and run the script inside.

For creation:

lxc launch images:ubuntu/20.04/armhf rpi-focal
lxc launch images:debian/9/armhf rpi-stretch
lxc launch images:debian/10/armhf rpi-buster
lxc launch images:ubuntu/18.04/armhf rpi-bionic

later on, on each container do:

lxc start rpi-buster  # rpi-bionic, rpi-focal, etc
apt install inetutils-ping wget git bash-completion
useradd -m -s /usr/bin/bash seafile
visudo
add the line to the bottom:
seafile ALL=(ALL) NOPASSWD: ALL
su - seafile
./build3.sh   bzw. ./build3-7.1.5.sh
1 Like
13

@Gustl22 I also plan to change the release name adopting the stablished naming schema convention for releases. Here is a good example how will be the schema in the future:

-rw-rw-r-- 1 desigual desigual 71308604 Okt  3 12:16 seafile-server-7.1.5-bionic-armv7.tar.gz
-rw-r--r-- 1 desigual desigual 73459568 Okt 16 15:36 seafile-server-7.1.5-buster-armv7.tar.gz
-rw-rw-r-- 1 desigual desigual 74599781 Okt 16 19:31 seafile-server-7.1.5-focal-armv7.tar.gz
-rw-r--r-- 1 desigual desigual 71299971 Okt 16 17:20 seafile-server-7.1.5-stretch-armv7.tar.gz

On the past I got a lot of questions arising from the kind of ARM hardware in which I build the seaifle versions. Note that compiling on armv7 could not work on armv6 and such things. Therefore is important to be more explicit releasing version and the schema must be consistent. I told you that, because I’m confident you will contribute for arm64 architectures :stuck_out_tongue_winking_eye:

14

@jobenvil I would be happy to help you to release the armv8 builds. Unfortunantely your link to the updated build script is broken, to ensure I really use the right configuration as the other builds. Further I never did builds with containers before, so I can’t guarantee any success or if I have the time for it in future releases. Also not quite fixed the issue with python3 dependency (manually have to link from python3.8 to python3.6 with ln -s)
Further wouldn’t it be good to update the requirements links in build3.sh, so others can build on their own, if necessary?

Not quite sure, how I can upload the armv8 builds in the releases section or as Merge-Request? Think that has to do the maintainer? Best regards ;D

15

Hi @Gustl22 I delete the link one hour ago. Here it is again

The build script is here. on the GitHub.

It’s the same build script, but since the requirements from Seahub with Tag v7.1.5 are broken (I realized myself after the compilation with that script and you as well) I had to change one of them to djangorestframework==3.11.1. Since I cannot modify the requirements.txt from the official Seafile repo, I cloned it myself and made that change in my repo. Later on I pulled that requierement file instead of the official seafile. In normal case it should work the official requirements file.

Fair enough. I’m tinking how can be managed that. I will write you by PN.

This point is very easy to overcome until Seafile devs change to just python3 or python.

This is done in 5 minutes x container. I already posted the commands.

16

It’s now available on GitHub. Your sweat has paid off :ok_hand:

2 Likes
17

@jobenvil Now I was able to build for the other arm64 architectures. The biggest problem were the python versions and dependencies, which take hours to build without python wheels.
Here are the 7.1.5 builds (I assume they all work, although bionic, buster and stretch aren’t tested in production).

Additional a guidance to avoid the pitfalls:

18

  1. Launch images (Terminal 1)

    lxc launch images:<image-name> <container-name>

    1.a Build for armhf for 32 bit (armv7l):

    lxc launch images:ubuntu/21.04/armhf armhf-hirsute
lxc launch images:ubuntu/20.04/armhf armhf-focal
lxc launch images:ubuntu/18.04/armhf armhf-bionic

lxc launch images:debian/11/armhf armhf-bullseye
lxc launch images:debian/10/armhf armhf-buster
lxc launch images:debian/9/armhf armhf-stretch

    1.b Build for arm64 for 64 bit (aarch64):

    lxc launch images:ubuntu/21.04/arm64 arm64-hirsute
lxc launch images:ubuntu/20.04/arm64 arm64-focal
lxc launch images:ubuntu/18.04/arm64 arm64-bionic

lxc launch images:debian/11/arm64 arm64-bullseye
lxc launch images:debian/10/arm64 arm64-buster
lxc launch images:debian/9/arm64 arm64-stretch

    Optionally can start or stop container with:

    lxc start <container-name>
lxc stop <container-name>

  2. Do for each container:

    Start another terminal (Terminal 2)!

    2.1 Log in container (Terminal 2)

    lxc exec <container-name> -- /bin/bash

    2.2 If no internet connection / change domain name system (Terminal 2)

    vim /etc/netplan/10-lxc.yaml
    Then add:

    eth0:
   nameservers:
       addresses: [8.8.8.8, 8.8.4.4]

    Then netplan apply

    2.3 Install missing dependencies (Terminal 2)

    apt install sudo // If not installed

    2.4 Add seafile user (Terminal 2)

    useradd -m -s /bin/bash seafile
visudo

    Add the line:

    seafile ALL=(ALL) NOPASSWD: ALL

    Change visudo with commands: Insert-Key to edit, Esc to exit edit mode, :w to save, :q to quit.

    2.5 Push build file to container (Terminal 1)

    lxc file push build.sh <container-name>/home/seafile/

    For stretch remove this string from build.sh beforehand or edit with vim:

    --no-warn-script-location

    2.6 Run build (Terminal 2)

    su - seafile
time bash ./build.sh -D -A -v 9.0.1

    Wait for build to finish…

    If no success sudo bash ./build.sh -D -A -v 9.0.1

    2.7 Download the built package (Terminal 1)

    lxc file pull <container-name>/home/seafile/built-seafile-server-pkgs/seafile-server-9.0.1-<distro-name>-<architecture>.tar.gz ./

  3. To clean the distro containers and images run (Terminal 1):

    lxc list
lxc delete <container-name>

lxc image list
lxc image delete <image-fingerprint>

  4. Test and fix your build

    4.1 Start seafile and seahub

    tar -xf seafile-server-9.0.1-<distro-name>-<architecture>.tar.gz -C ./

    For debugging, set daemon = False in conf/gunicorn.conf.py

    seafile-server-latest/seafile.sh start
seafile-server-latest/seahub.sh start

    4.2 Fix bugs

    If you get the error ModuleNotFoundError: No module named 'Image' most likely in (stretch and bionic), you have to install pillow again (see here):

    pip3 install -U Pillow -t ./seafile-server-9.0.1/seahub/thirdpart
2 Likes
19

Yes, they should work. Tipp: You can even test your own produced packages when these have the libc6 (GLIB_*_*) library inferior than your host Distro. I assume your host distro is Ubuntu Focal.

Furthermore, since we are bulding the packages inside distro specific containers, the packages are getting up-to-date (distro depending) native libraries. The backwards compatibility must not be garanted anymore.

Doing that, we benefit that our shipped third party python modules in our packages, will work out the box, without to be installed by pip again. Containers are really a great think, when it cames to build binaries.

I uploaded all your distros to GitHub.

1 Like