"About the future of Seafile" --- repository packages

Hello, dear Seafile developers!

I am not here to comment or participate in the communitie’s speculations on what happend between the two Seafile companies. Instead let me first express my gratitude for the seafile community version, which was created by you. Once i had it running, I have been using it happily and for more than a year now. It has made my life easier and (overall) I love it!

I am a little appalled by the installation process and the way Seafile is packaged together with its various dependency libs, though. Don’t be offended, I am probably more of a purist person with respect to that and I know it is hard to do this properly. I did notice however that Alexander Jackson said something about that in the lengthy “About the future of Seafile” thread, in the second last paragraph of this post (in German, among some lament about lack of cooperation):
https://forum.seafile.de/t/zur-zukunft-von-seafile-about-the-future-of-seafile/5451/14?u=leo
It roughly translates into the codebase being in a somewhat scruffy state and that it has issues with license violations/incompatibilities. That is why they plan to mainly focus on those problems in their work first with the goal of getting Seafile into the Debian repositories.

I cannot judge on whether or not they have the manpower and expertise to achieve this, but I would be really pleased with having Seafile in one of the big distribution’s repos. Does Seafile Ltd. also have such plans (prioritizing codebase maintenance and getting these issues mentioned above sorted out)?

I have also been wondering about the questions raised in this post on the old forum, which somehow seems to have gone completely unnoticed:
https://forum.seafile.de/t/a-few-misc-questions/2809

What is the position of Seafile Ltd. on this? Or have things already improved in the meantime without me noticing?

I am asking all these questions because in the roadmap so far I have not seen anything concerning these things. I had been wondering for quite a while already, but the current event probably provides a good opportunity to talk about the future course of development. Since version 6 will hopefully be finished soon, is getting Seafile ready for mainline repository quality requirements, thus easing the installation and update process, something the community can hope for in Seafile 7?

Thanks a lot for everything and greetings from Munich!

PS: so far I am only aware of the roadmap document at seacloud.cc. I completely lost track of which domain/website belongs to whom. If it is not yours, where will I find your roadmap in the future?

3 Likes

Hi,

Thank you for opening the discussion!

Yes we’ve heard quite a few requests for improving our release method for Linux (both clients and servers). I’ll first try to list all the requirements we’ve heard and then explain the current status and our plan.

Requirements:

  • The Linux clients (both GUI and CLI clients) should be able to installed from a debian repository. It would be even better if official RPM packages are available too.
  • Linux clients should get into the official Debian repositories. Similar for RPM packages.
  • The Linux server packages should be available in Deb and RPM format. And they should be installed from a repository too.

The current status:

  • Currently Linux clients can only be downloaded as a single deb package, not from a repository. RPM packages are not officially maintained. But there is an non-official package maintained by contributor: https://seacloud.cc/group/3/wiki/seafile-clients-linux.md. But it’s not always up-to-date.
  • The debian package of Seafile isn’t in Debian official repositories, due to strict copyright requirements in Debian. You can see the discussions https://github.com/haiwen/seafile/issues/666. The core issue is that we use some code from Git project (GPLv3). We also use OpenSSL. However, Debian rules doesn’t allow GLP software to link with OpenSSL. This is not a general copyright issue, since Git itself links with OpenSSL too. So by default they’ve added exception to allow their GPL code to link with OpenSSL.
  • Seafile server is not a single-component system daemon like MySQL. It’s an application server contains both daemons written in C and web front-end written in Python. (Even more components for the Pro Edition.) Installing python web application in standard Debian/RPM way is troublesome. So if we would provide a deb or rpm package, the file system layout after installation would still be different from traditional ones.

Our plan:

  • We’ll provide a debian/ubuntu repository for installing seafile client. This may be based on this contribution: https://github.com/haiwen/seafile/pull/1686. But we haven’t decided about the package organization yet (single package like it is now, or multiple packages). We want to listen to the users about their opinion.
  • The next step is to resolve the copyright issue for entering debian official repositories.
  • We’ve been undecided about using deb package to release servers, due to the concerns stated above. But we’ve seen a lot requests for it. It does makes Seafile more “standard”. Now we’ve decided to approach this goal in multiple steps and solve issues we may meet piece by piece.

We want to listen to your opinions.

2 Likes

That last line is what will make people stick with you!!!
The day when that mentality starts to decline at seafile because “it would be to hard to implement” or etc. will be the day Seafile begins to decline and loose customer base to the company that does cater to the customers.

1 Like

Hi,

thanks a lot for this extensive reply about the current status and for the link to the discussions on Github! I think this is a very valuable post on this forum! I had actually not been aware that there had already been a lot of progress made during the past year. So the way I see it now is, that the client needs to drop support for pre-3.0 libraries before it can enter the official Debian repositories? Interesting, that was not obvious to me before. Well, this is certainly a step that should be well-thought-out and (if it happens at all) announced quite a while in advance before it actually happens!

The plan you outlined seems reasonable to me personally, so thanks for sharing it. But I still believe that it is also something which should be reflected in your roadmap document. I have no clear preference of a single .deb vs. multiple ones, though modular approaches tend to have advantages in general, if well-designed.

For the seafile server I would see some room for improvement indeed. Before I comment on that though, maybe you can elaborate a little more on why Seafile’s peculiar directory layout is a necessity and what is hard about doing it the Debian/RPM way? I am not at all experienced in developing Python web applications, so I am not sure whether I understand that paragraph correctly…

Thank you again, I really appreciate the work going on here!

I’ve followed the packaging situation with Seafile for a while, and that was always my mayor concern for deploying/using Seafile in a production environment. In the past new features always seemed to take preference over consolidating the code-base by Seafile Ltd. But Seafile GmbH wasn’t much better in my opinion, I was very disappointed when the installer scripts were written, because I thought this is effort that should have gone into planning and moving to a packaged code structure, instead of writing scripts to install Seafile on one particular platform, in one particular way. Which is not good for anybody whose infrastructure is setup differently, or who uses different technologies. As a sysadmin, I would have never used those scripts, even if I could have, in my environment. Might have had some value for home-users, but not much else IMHO. Whereas, having official, standard OS packages would have helped everybody, regardless of whether they use Docker, lxc, Puppet, ansible etc for deployment.

That being said, since then there have been a few improvements in that space. Most importantly, the config and application files are now somewhat properly separated. So, I think doing packages now should be easier than it was in the past.

I, personally, wouldn’t be too worried about Seafile not being in the official Debian (or whatever else distribution) repositories, as long as there is an ‘official’ (Seafile) repository. Those packages would always be quite out of date anyway, considering how often there is a new server release. If it’s in there, fine, but I’d probably use the Seafile channel to install anyway. Having a testing/stable version wouldn’t hurt though, for the more conservative use-cases where new features are way less important than a working service.

Also, I agree that Seafile is not a straight-forward app to package, but it’s not the most complicated either.
I guess the main problem is that it can be deployed in a few different ways. For example using SQLite or Mysql or Mariadb as a db backend (and potentially PostgreSQL). Also, there are a few different configurations for webservers (none, Apache, Nginx, …).

Having to sort all this out is cumbersome, but in my experience at least, it also helps you understand your codebase and it’s problems: where your abstractions don’t really work, where you have assumptions that you shouldn’t have, that sort of thing. Either way, I do think native packages would greatly benefit the adoption of Seafile, much better than any new features would.

So, if I was the person responsible, this is a quick outline what I would do:

  • look for a few other applications that are similar to Seafile, see how they did the packaging/deployment
  • decide on which packages I want to create (I’d recommend deb as well as rpm, ideally debs would work for both the latest Debian and Ubuntu versions and rpms for RedHat/CentOS/Fedora – whether that is possible is to be determined)
  • if you think you can only support one package format reasonably, that is fine too. Just make sure to document the work you do properly, and put the scripts you use somewhere prominent. That way it’s much more likely that the community will pick up packaging for other targets
  • decide how many packages you want to create (common, client, ccnet, seafile, seahub, server-community, server-pro, …), make them independent/dependent on each other as much as possible (server-pro depends on server-community, for example, if appropriate) – this might require re-factoring of some components, but, as I and others already said, this exercise it usually worth it in the long term
  • put config files in the appropriate directories (under /etc)
  • make a list of dependencies that can be sourced from the host distro, and ones that need to be included in the packages
  • check out ‘fpm’ ( https://github.com/jordansissel/fpm ) if you haven’t already, and if you don’t have experience in packaging applications. I’ve made good experiences with it – even if you just use it for prototyping and to get your head around the whole thing
  • decide on a ‘most simple’ install scenario (probably community version, using sqlite and no external webserver), use that as a starting point – if somebody installs the package, they get a running seafile server, even if it uses an ‘ugly’ port and ‘only’ sqlite
  • other setup scenarios can be done (initally at least) by the user manually, but should be documented properly (but that work is mostly done already – I think the manual is quite good in that regard) – this is something that is acceptable in my opinion, I have to do that for a lot of other packages as well (e.g. put Nginx in front of a web-app). In the future, other ‘default’ scenarios could be included, e.g. let the user choose on setup whether to use mysql or sqlite and install the right dependency if a local mysql server is used. Remains to be seen whether that is feasable or not though, I wouldn’t worry about it too much at first
  • include example config files for as many scenarios as possible in the packages (in /usr/share/doc/seafile or so)
  • now, the main problem is updating the server, here is where you need to make sure you take into account custom setups:
    • if in doubt, the installer should not go ahead with an update
    • create and document scenarios supported by the packages (mysql, sqlite, …)
    • make sure you have test environments and a set of automated tests ready for every scenario you ‘officially’ support
    • once those basic packages work, you should have enough experience and a good overview what is involved in packaging the ‘pro’ packages

I’m sure I forgot a lot of things, and some of what I’ve written is not possible for some reason or other. Just consider it an opinion of a user who is quite fond of Seafile (the application), and who wants to see Seafile Ltd to succeed.

Good luck, either way!

2 Likes

Hi @Leo

After more thought, I also think the directory layout of Seafile server can also be mostly compliant to the rules of most distros. But that would take long to implement. We’ll do that step by step. The first step would be a deb package without changing the directory layout. Then we move binary, config, data and scripts directories to standard locations later.

The issue with Seahub is that there is no standard location for django application (or python applications). But we can keep them under /opt for convenience.

In my opinion, ‘opt’ is a perfectly fine location for Seafile (or parts of Seafile)…

1 Like

One thing that could be improved about the directory structure at the very moment is the runtime folder, which contains seahub.conf and seahub.pid outside of the conf and pids directories. But this is now a very specific concern and maybe even a little off-topic.

I agree with what makkus said on the installer script situation, stable/testing branches in a custom repository and his packaging advice.
As far as django in Debian packages is concerned, i am afraid I do not have the knowledge to help out. All I know is my (former) colleagues saying “if there are no packages yet, throw dh-virtualenv at it”. Since for the server, it is still a very long way, maybe this might be a solution for an intermediate step? There is also a rather young project called py2deb (check https://py2deb.readthedocs.io/en/latest/comparisons.html for an overview), but that is just me taking a stab in the dark, not sure it can solve your issues.

I was on vacation while all of this happened. Now I’m back and see that there’s a Ubuntu repo. Great news. This will help me a lot. As for the Linux server, in my opinion the installation and upgrade process is already rather on the easy side. I’m also no fan of installing server software like Seafile via the servers package management.

On a side note: I read all of the posts and discussion and as you probably don’t follow German IT news and follow German forums (like Heise News), I wanted to let you know that the German (vocal) user base seems to be behind you. That doesn’t surprise me much. Every seafile admin who’s had technical questions had them answered by you and not the German marketing team. So rock on @daniel.pan & co!

1 Like

For people interested, I’ve done some proof of concept packaging of seafile-server for Debian. I’de love some feedback.

1 Like