Migrate a single library to another storage backend

Seafile Server
1

Thanks to a hint from haikuo I now have a working multiple storage backend setup.

Now the next question is how do I migrate existing repos from one backend to another. Our Seafile system contains about 25 TB of data, which means that offline migration is out of the question - the downtime is prohibitive. Is there a recommended way to online-migrate a single (or all) repo(s) to another storage backend?

2

Ping @scheff - he has written a (sophisticated) script for exactly that. Maybe he is willing to strip it from assumptions about our ecosystem and publish it.

3

Hi,
I’m happy to share the script. Give a day or two to prepare it.

Best regards

Thomas

4

Thanks a lot! I’ve some ideas how to do that, but it would be a lot of work.

Christian

5

Hi,

you can download the script from our Seafile server:

https://seafile.rlp.net/d/a80f38655dbe4f658556/

Description:

This script migrates all libraries of a user from one storage backend to another

usage:

./moveLibraryToStorageBackend --mode info    --library UUID 
./moveLibraryToStorageBackend --mode migrate --library UUID --source STORAGE --target STORAGE
./moveLibraryToStorageBackend --mode cleanup --library UUID --source STORAGE

optional parameters

mode info:    [--config FILE] 
mode migrate: [--config FILE] [--dry-run] [--verbose] [--safety-period SECONDS] [--up-to SIZE]
mode cleanup: [--config FILE] [--dry-run] [--verbose]

Parameters:

Called without any parameters this script just displays this help text

-c, --config
a config file; defaults to moveLibraryToStorageBackend.conf if this parameter is omitted

-d, --dry-run
don’t actually migrate or clean up libraries, just show what would to be done

-h, --help
display this help text

–info
display this help text and additional information about this script

-m, --mode MODE
Set the main mode of this script; mandatory; valid modes are:

   - info       : Shows basic information like name, type, storage backend of libraries
   - migrate    : Migrates libraries from one to another storage backend
   - cleanup    : Deletes already migrated libraries on the old storage backend

-l, --library UUID
The UUID of a library that should be migrated; which can be one of the following:

   - UUID                 - to select a single library 
   - oldest-unused:NUMBER - to select the number of libraries that are unused, i.e. not modified longest
   - user:EPPN            - to select all libraries of a user

-p, --safety-period SECONDS
seconds that a library has to be unchanged before it is migrated

   if omittet this parameter defaults to 3600 seconds

-s, --source STORAGE
the storage-id of the current storage, mandatory for modes migrate and cleanup

   valid storage ids are: nfs_data and nfs_001 to nfs_008

-t, --target STORAGE
the storage-id of the target storage, mandatory for mode migrate

   valid storage ids are: nfs_data and nfs_001 to nfs_008

-u, --up-to SIZE
maximum size of all migrated libraries in MByte to avoid consuming the entire space
of the target partition

-v, --verbose
verbose output

Best regards

Thomas

1 Like
6

Hey @scheff ,
I am interested in your script.
Unfortunately the link no more works.
Could you please create a new one?

7

In 7.1.4 Pro edition, we’ll officially support migrating library from one backend to another. The documentation is on its way.

2 Likes
8

Hey @Jonathan , any news about the documentation for migrating libraries?
Im interested in migrating libraries from s3 to local storage.

9

The procedure has been updated to the multiple storage backend documentation.

10

Unfortunately the documentation still cannot be found using Google.

https://download.seafile.com/published/seafile-manual/deploy_pro/multiple_storage_backends.md