Multiple seafile-data folders?

It appears Seafile Server supports only one seafile-data folder. It would be wonderful to add support for many seafile-data folders. And the admin should be able to specify which libraries or groups or users are saved in which data folder. This is needed to be able to mount different data folders for different groups and to backup the data folders differently.

We have plan to support more than one storage folders in the future. But the purpose is only to let admins increase storage easily, not to map libraries to storage in a more complex way. Mapping libraries into multiple storage folders would introduce more database accesses. This will slow down Seafile’s syncing.

But your idea is interesting. We’ll keep it in mind.

6 Likes

Multiple backends would require storing it in the database on per library base anyway, wouldn’t it? At least that would be what I’d expect to be the solution with the best performance. Only down I could think of is that once one backend is almost full some libraries would need to be moved to another backend. New libraries could be stored in the default backend (allow setting multiple default backends could also be useful for large setups). Another option would be a flag per backend if library creation is allowed or not and calculate how much libraries we have per backend and choose the one with most GB (total space) per library.

This feature request could be implement on top by adding something like “library policies”, where the policy defines where a library should be stored.

2 Likes

want to know whether this feature added to new 7.0.x version…i also want this feature

Multiple storage backend support was added in Seafile 6.3. Check this out: https://blogs.seafile.com/2018/07/30/seafile-professional-server-6-3-now-production-ready/

It’s a Seafile PE feature though.

Thanks your help…
but mutiple storage backend is different to the requirement, I think.
Actually the author of this topic and myself just want to store the file in local drives. Let’s say in different volumes(/dev/sdb,sdbc) or disks(D drive and E drive). and different users store the fs, commit, blocks file in different seafile-data folder.
No need to integrate with other cloud based storage.

I think this feature is required by most of the small company if they have more than 2 teams, eg, accounting and IT teams

I am using mutiple seafile-data folders which are mounted to different linux disks. However seems only working for what put in side ccnet/seafile.int
or maybe my conf is wrong… hahahaha:rofl:

[
{
“storage_id”: “seafile-data”,
“name”: “seafile-data”,
“is_default”: true,
“for_new_library”: false,
“fs”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”},
“commits”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”},
“blocks”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”}
},

{
“storage_id”: “new_storage”,
“name”: “new_storage”,
“for_new_library”: true,
“is_default”: false,
“fs”: {“backend”: “fs”, “dir”: “/storage/seafile/newseafile-data”},
“commits”: {“backend”: “fs”, “dir”: “/storage/seafile/newseafile-data”},
“blocks”: {“backend”: “fs”, “dir”: “/storage/seafile/newseaflle-data”}
}

]

Do you use Seafile PE or CE? Support for multiple storage backends is a PE feature.

The feature does not care if you use two filesystem-based backends or S3 + Swift or whatever.

I had a quick look at your config. Looks fine to me, but I am not a multi-backend expert. There is one thing though that you might want to change:

seaflle should be seafile.

Do these folders exist?

thanks your help.haha yes, i think that is my typO :stuck_out_tongue: error…but i manually create those folder before I changed seafile.ini content.

I am reinstalling pro version 7.0.5 now and see whether it works…soon update here…heheh
I am curious whether it works as search a lot but no any final solution yet.

I tested again…still failed.
I read both Chinese and English github manual again for this multiple storage backend setup and reinstall new seafile pro 7.0.5 version.
I followed exactly the “user chosen” option:

User Chosen
This policy lets the users to choose which storage class to use when creating a new library. The users can select any storage class that’s been defined in the JSON file.

To use this policy, add following options in seahub_settings.py:

STORAGE_CLASS_MAPPING_POLICY = ‘USER_SELECT’

If you enable storage class support but don’t explicitely set STORAGE_CLASS_MAPPING_POLIICY in seahub_settings.py, this policy is used by default.

and Here is the storage class I defined even though all local file system??? here is my doubt, if I use different seafile-data folder will it support in this mutiple backend storage portion?

[root@app01 conf]# cat seafile_storage_classes.json
[
{
“storage_id”: “seafile-data”,
“name”: “seafile-data”,
“is_default”: true,
“fs”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”},
“commits”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”},
“blocks”: {“backend”: “fs”, “dir”: “/seafile/seafile-data”}
},
{
“storage_id”: “cold_storage”,
“name”: “cold_storage”,
“is_default”: false,
“fs”: {“backend”: “fs”, “dir”: “/storage/seafile/cold_seafile-data”},
“commits”: {“backend”: “fs”, “dir”: “/storage/seafile/cold_seafile-data”},
“blocks”: {“backend”: “fs”, “dir”: “/storage/seafile/cold_seafile-data”}
},
{
“storage_id”: “new_storage”,
“name”: “new_storage”,
“is_default”: false,
“fs”: {“backend”: “fs”, “dir”: “/storage/seafile/new_seafile-data”},
“commits”: {“backend”: “fs”, “dir”: “/storage/seafile/new_seafile-data”},
“blocks”: {“backend”: “fs”, “dir”: “/storage/seafile/new_seafile-data”}
}
]"

cat seahub_settings.py
here is the seahub_setting.py

}
ENABLE_SYS_ADMIN_VIEW_REPO = True
MULTI_INSTITUTION = True
EXTRA_MIDDLEWARE_CLASSES += (
‘seahub.institutions.middleware.InstitutionMiddleware’,
)
ENABLE_STORAGE_CLASSES = True
STORAGE_CLASS_MAPPING_POLICY = ‘USER_SELECT’
#STORAGE_CLASS_MAPPING_POLICY = ‘REPO_ID_MAPPING’
#STORAGE_CLASS_MAPPING_POLICY = ‘ROLE_BASED’
ENABLED_ROLE_PERMISSIONS = {
‘default’: {
‘can_add_repo’: True,
‘can_add_group’: True,
‘can_view_org’: True,
‘can_use_global_address_book’: True,
‘can_generate_share_link’: True,
‘can_generate_upload_link’: True,
‘can_invite_guest’: True,
‘can_connect_with_android_clients’: True,
‘can_connect_with_ios_clients’: True,
‘can_connect_with_desktop_clients’: True,
‘storage_ids’: [ ‘new_storage’, ‘cold_storage’,‘seafile-data’], <---- add the storage_id here
}
}
"
as the “for_new_library”: true, option is only mentioned in libarary ID mapping option so i removed it from my json file.

However I tested, frontend portal when I create new libarary by using new user account, still everything goes to old seafile-data folder. There is no place to “CHOSEN” the storage class.

A quick update here… I am able to make it work by doing some manual work.

i changed policy to repo id mapping
STORAGE_CLASS_MAPPING_POLICY = 'REPO_ID_MAPPING’
firstly, i create new library from web, and there is a folder created in old seafile-data commit folder.

I manually changed database storage_id for the newly created library. can see the new id is “new_storage” which point to “fs”: {“backend”: “fs”, “dir”: “/storage/seafile/new_seafile-data”},

image
then I copy old commits folder(located at old seafile-data folder,as i didnt upload anything yet,so no folder under “fs” and “bloack”) to this new commit folder(commits": {“backend”: “fs”, “dir”: “/storage/seafile/new_seafile-data”},)
then wola… all working …
I am able to upload files to this new library and at backend all files are stored in new folder. and other libaries no impact.

but it is really not a good way even can make it work.

So my guessing is, need to populate storage_id information before you config the storage class.

If anyone has any finding or can contact the dev team. Please help me to ask for it.

thanks :stuck_out_tongue: