Any chance? Upgrade/Migration from 2.15 server to latest version?


#1

Hello,
I recently found on a new customers undocumented site a still running and obviously still in use 2.15 Seafile server installation.

Is there any chance to migrate the data to a fresh 6.x server installation?

I tried google, wiki and this forum :wink:

Regards

Loetmann


[SOLVED] "Don't support syncing old version libraries" after upgrade to Ubunto 18.04 LTS
#2

it would be easier to set up a brand new server and manually import the files into NEW libraries.

In version 3 the library format did change drastically and there is no import or upgrade funktion for this.


#3

If you upgrade by every single release, it wil work. But how @Andre sad you CAN have some problems with old libraries but upgrade script should work. For best result is better export all data from seafile for backup, make upgrade and check libraries if.

New installation is not nessesary but recommended.


#4

yes, it is possible. But really not recommended. I was a bit harsh in my first post.
You run into multiple problems if you do.


#5

Hello,
thanks for that topic. I have an really similar situation. An old 4.3.2 server with librarys from a 2.x Server (i guess that because I can’t see filedates in old librarys, but in new librarys they a präsent).

So here are my questions:

  1. How can I prove which librarys are affected in detail?
  2. Is there a way to migrate to the new library cheme without interaction from my users? How is the mentioned above import exactly working?

Thanks in Advance


#6

Hello,
after waiting for replys :sweat: i tried it myself and found a copy of an old post with an script that can do the magic part of the conversion. But there are many other instructions to do and all off these for every single library. so i wrote my own additional script for doing the other stuff besides the “magic”

In this copied post you find a description of the script for the conversion and which steps you have to do manually (for these steps you find my additional script below)

I finished converting my complete seafile installation a few days before and polished my program a little as well.
Manual
What my program can do:

Converting an unencrypted library from the „V0“ to the „V1“ library format while preserving the version history

What my program cannot do:

Converting an encrypted library (I do not need this myself)
Brew some coffee
Everything else

Please note:

While I successfully converted my own unencrypted libraries I cannot guarantee that it won't delete all your precious data, keep backups and play safe.
You have to shutdown seafile before starting a conversion (or at least ensure that the library that's being converted is not modified).
It is pretty slow (this is an inherent property of the conversion process). Depending on the exact size of structure of the library it may take a few hours or even days.

How to use?

Find the UUID of the library you want to convert (it's in the address bar of the web interface)
Shut down seafile
Find the root commit via: SELECT commit_id FROM Branch WHERE repo_id = '…' (insert your UUID here)
Run ./seaf-fix /srv/seafile/seafile-data/storage /srv/seafile/seaf-fix/target/ UUID commit_id (you'll know that your paths are)
Wait
Wait a little bit more until something like this appears: Done, your new root is: b0d58206779cc1299a4dde13950dfe091cc1a704
Ensure your backup is good, once again
UPDATE Branch SET commit_id = 'b0d58206779cc1299a4dde13950dfe091cc1a704' WHERE repo_id = 'UUID' (adapt as needed)
Repeat step 4 to 8 for every sub library: SELECT repo_id FROM VirtualRepo WHERE origin_repo = 'UUID'.
You have to symlink /srv/seafile/seafile-data/storage/fs/SUBLIBRARY_UUID/ to /srv/seafile/seafile-data/storage/fs/UUID/ (as both share the same FS directory)
rm -rf /srv/seafile/seafile-data/storage/commits/UUID/ (for every library you converted)
rm -rf /srv/seafile/seafile-data/storage/fs/UUID/ (for every library you converted)
Move every folder from /srv/seafile/seaf-fix/target/commits/ to /srv/seafile/seafile-data/storage/commits/
If you have sublibraries: Merge the contents of the /srv/seafile/seaf-fix/fs/ folders into the folder with the UUID of the root library
If you have sublibraries: Fix the base_commit inside VirtualRepo (you have to search the output of the parent library for that, sorry).
Move /srv/seafile/seaf-fix/target/fs/UUID to /srv/seafile/seafile-data/storage/fs/UUID.
Run seafile-fsck.sh (it should not find any issues)
Run seafile-gc.sh --dry-run (it should not find any issues either)
Start seafile again and check whether your library looks fine.
Party

License?

MIT License.
Where to get it?

It comes in three prebuilt flavors. One version for servers with lots of RAM to spare (it uses up to 25 GB of RAM on my box, YMMV), another one for smaller boxes (about 2 GB of RAM, YMMV) and a third with only the minimal amount of RAM. You can adapt it yourself in the source code on this line: https:// github. com/TimWolla/seaf-fix/blob/d5c362c34eb4f6c5bd88cf58d0d9eef3b137043b/seaf-fix.hs#L279 (6 Millionen = 25 GB, 400k = 2 GB). Obviously the more RAM you have to spare the faster it is. The binaries are static binaries built using GHC 7.10.1.20150710 inside a Docker container (Debian Jessie), they should work on every Linux out there, though.

Both the git tags, as well as the binaries can be verified using this GPG key:

pub 4096R/094168AF 2013-10-04
Key fingerprint = 3033 55CF 8E33 D090 86CB 28F6 8FF7 5566 0941 68AF
uid Tim Düsterhus (git signing) timwolla[@] bastelstu.be
sub 4096R/127D597F 2013-10-04

Git repository: https://github.com/TimWolla/seaf-fix

25G RAM version: https://github.com/TimWolla/seaf-fix/releases/download/0.1.0.0/seaf-fix-6000000
2G RAM version: /seaf-fix-400000
Minimal version: /seaf-fix-0

Hashes:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

6166144eb0eb6b68f7b94aa6ea5e7904c1494d5f72c3dde97a4c67e2cb8e9a97 seaf-fix-0
f131ad42c8940e9e4bdbc4814420e0ecc0876d70cf700cc5143b25f994b3246f seaf-fix-400000
c44818d20cec4622bf04a59ce5f1fa6e5f235c1703dc4242f9eea381b33ebb4e seaf-fix-6000000
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJWZNkyAAoJEI/3VWYJQWiv5W0P/ifns/qmvXWZ/tZr7zsYlkgG
mwAnh7iqeWrwFw0EAS9F2LwvVSffTdJenG6tSUsC1SQ5NnD7JI89zUxk1wN13Y+3
kO3bFaq5UE6GQbkpPVtSK9zi1F6TYWNdmkygkTE3L1/3wqIKRvuP2hGvyoI8ydY3
xqExbnpwSd6kI2+yy1OONJVR+A1ZmK6ZNdROP6AOcfxvPtYbqN8Zd6qptMeHbfQ0
4ujnwXtPB+zvleT7KfrfWyTZbEW6BL624Ox/FQzu9Dvn9S/uDQZPxQL5uMtfvWY2
A3OaZFCPqaZ+pyJYHz6cp/1iuaPQT94gV2BFEHXXgJNqtCc/5CS99Y3VAhDxpJ6U
aCKQEeRKkkjJXkBE2edd+KCOqnw+FJlgIzi4yTagziZ73Gtq+br8Dpv9raw2ypQv
xTE0dYR6zjKMvhPKKZ+59FwbZfWLfpqfyJnXSudnAN66jJyJr4RtGKZT/WF047Pf
Nm7HP5W3mJ7ftXZFqAi9TEbsdOdrd/DE1GA0KVVwC4wUJKvXq3Nq091C4/uL4Rkt
tpJQ0B+GVoYMYpBoJeZqUO5lZs2Mv4QjL8/DRF2GEl0FRDTfjC9bGv1YWfywCHob
guI1JEKY1WkWchMavOTO2jIZoLZZokhZq2RQDdaaBVODFnPkW4e/O4blD7cVD0xe
+xS1IZQOQQy1Oqp/TI3S
=v3hu
-----END PGP SIGNATURE-----

To save you from the manual work, you are free to use my script in addition. But please keep in mind that my script uses the “magic” script, so the same limitations are applying. I have not coded the case of sublibarys into the script. There is only a test, an if there are any sublibarys the script ends without finishing the job.

TAKE BAKUPS FIRST AN USAGE IS AT YOUR OWN RISK - obviously I’m not a developer ^^ (but for me it worked like a charm :slight_smile: )

Maybe you have to edit the parts regarding to the Database (SELECT and UPDATE) if you are using an other DB than MySQL.
I translated the script descriptions into english (I tried, ^^) and put a few more variables for you inside. So put everthing inside a xyz.sh file and maybe you have more time for more interesting things.

#!/bin/bash

##############
#declarations#
##############
#
#user defined variables
#
db_user=""
db_pwd=""
db="seafile-db"

path_storage="/usr/seafile/seafile-data/storage"
path_fix="/usr/seafile/seaf-fix/target"
path_log="/usr/seafile/convert_v0_to_v1.log" #path has to already be present but the file will be created while running the script

script_seaf_gc="/usr/seafile/seafile-server-latest/seaf-gc.sh"
script_seaf_fix="/usr/seafile/seafile-server-latest/seaf-fix-400000 $path_storage $path_fix/"
script_seaf_fsck="/usr/seafile/seafile-server-latest/seaf-fsck.sh"

owner_files="seafile:seafile" #owner user:group of the library files. This is needed if the script is running with other user rights e.g. root. (look into your storage path to find out)
user_seafile="seafile" #user for running seafile... when running script e.g. as root and seafile server as an other user this variable is neede for runngin seaf-fsck.sh as seafile user

#
#other used variables (already declarated for better overview
#
response=""
response_convert="y"
response_convert=${response_convert,,}
UUID=""
commit_id=""
sublibrarys=""
new_root_id=""
echo_script=""

#
#Funktionen
#
funct_response_ok(){
    echo "----------"
    read -r -p "Is everthing ok till now? [y/N]: " response
        response=${response,,}
        if [[ "$response" =~ ^(no|n)$ ]]
            then
                echo "CANCELED BY USER"
                exit 0
        fi
    unset response
}



#########################################
#preparations incl. garbage collection#
#########################################

#check if log file already created
if [ ! -f $path_log ]
  then
     echo -n > $path_log
fi

clear
service seafile-server stop

$script_seaf_gc --dry-run|grep 'can be' #shows all librarys an if there are old unused parts

#
#do you want to cleanup
#
read -r -p "do you want to cleanup? [y/N]: " response
    response=${response,,}
    while [[ "$response" =~ ^(yes|y)$ ]]
        do
            $script_seaf_gc
            $script_seaf_gc --dry-run|grep 'can be'
            read -r -p "do you want to cleanup? [y/N]: " response
            response=${response,,}
    done
unset response

######################################################
####LOOP for collecting information and converting####
######################################################
while [[ "$response_convert" =~ ^(yes|y)$ ]]
    do
        #######################################
        #get UUID, root commit_id, sublibrarys#
        #######################################
        #
        #using script_seaf_gc for showing UUIDs of all old librarys and then cut this output to use the first of these UUIDs
        #
        echo_script=$($script_seaf_gc --dry-run|grep 'GC version 0')
        echo "$echo_script"
        echo_script=${echo_script#*repo}
        echo_script=${echo_script%%)*}
        UUID=${echo_script#*(}
        unset echo_script
        echo "$UUID"
        
        #
        #usind UUID for collecting root commit_id and sublibrarys from SQL_DB
        #
        commit_id=$(mysql -u $db_user -p$db_pwd -D $db -N -B -e "SELECT commit_id FROM Branch WHERE repo_id = '$UUID';")
        sublibrarys=$(mysql -u $db_user -p$db_pwd -D $db -N -B -e "SELECT repo_id FROM VirtualRepo WHERE origin_repo = '$UUID';")
        
        #showing output on screen and write also to log
        echo "----------"
        echo "UUID:		$UUID" | tee -a $path_log
        echo "commit_id:	$commit_id" | tee -a $path_log
        echo "Sublibrarys:" | tee -a $path_log
        echo "				$sublibrarys" | tee -a $path_log
        echo "----------"

        read -r -p "sublibrarys or more than one commit_id found? [y/N]: " response
            response=${response,,}
            if [[ "$response" =~ ^(yes|y)$ ]]
                then
                    echo "ATTENTION: THIS CASE IS NOT CONSIDERED WHILE WRITING THIS SCRIPT"
                    exit 0
            fi
        unset response

        ############################
        #copy & convert old cibrary#
        ############################
        echo_script=$($script_seaf_fix $UUID $commit_id) #run script_seaf_fix and write output to variable
        echo "$echo_script"
        new_root_id=${echo_script##*' '} #cuts output to show new_root_id
        unset echo_script
        echo "----------"
        echo "CHECK, is that the above mentioned ROOT_ID?: $new_root_id"

        funct_response_ok
        
        echo "new_root_id:	$new_root_id" >> $path_log #write info to log
        echo "----------" >> $path_log #write info to log

        mysql -u $db_user -p$db_pwd -D $db -e "UPDATE Branch SET commit_id = '$new_root_id' WHERE repo_id = '$UUID';" #updates DB
        
        #
        #removing old files and moving converted library files to old location and changing owner
        #
        rm -rfv $path_storage/commits/$UUID
        rm -rfv $path_storage/fs/$UUID
        chown -Rv $owner_files $path_fix
        mv -v $path_fix/commits/* $path_storage/commits/
        mv -v $path_fix/fs/* to $path_storage/fs/

        #################
        #Nachbearbeitung#
        #################
        unset UUID
        unset commit_id
        unset sublibrarys
        unset new_root_id
        unset echo_script
        
        read -r -p "Do you want to search for an other library for conversion? [y/N]: " response_convert
        response_convert=${response_convert,,}
done
unset response_convert
clear
######################################################
####                LOOP finished                 ####
######################################################

#
#checking librarys
#
su $user_seafile -c "$script_seaf_fsck"
$script_seaf_gc --dry-run
service seafile-server start


echo "check of librarys finished, when there are any problems then use $script_seaf_fsck and $script_seaf_gc to repair and check again"
echo "#####"
echo "FINISHED :-)"
echo "#####"