It appears Seafile Server supports only one seafile-data folder. It would be wonderful to add support for many seafile-data folders. And the admin should be able to specify which libraries or groups or users are saved in which data folder. This is needed to be able to mount different data folders for different groups and to backup the data folders differently.
We have plan to support more than one storage folders in the future. But the purpose is only to let admins increase storage easily, not to map libraries to storage in a more complex way. Mapping libraries into multiple storage folders would introduce more database accesses. This will slow down Seafileās syncing.
But your idea is interesting. Weāll keep it in mind.
Multiple backends would require storing it in the database on per library base anyway, wouldnāt it? At least that would be what Iād expect to be the solution with the best performance. Only down I could think of is that once one backend is almost full some libraries would need to be moved to another backend. New libraries could be stored in the default backend (allow setting multiple default backends could also be useful for large setups). Another option would be a flag per backend if library creation is allowed or not and calculate how much libraries we have per backend and choose the one with most GB (total space) per library.
This feature request could be implement on top by adding something like ālibrary policiesā, where the policy defines where a library should be stored.
want to know whether this feature added to new 7.0.x versionā¦i also want this feature
Multiple storage backend support was added in Seafile 6.3. Check this out: https://blogs.seafile.com/2018/07/30/seafile-professional-server-6-3-now-production-ready/
Itās a Seafile PE feature though.
Thanks your helpā¦
but mutiple storage backend is different to the requirement, I think.
Actually the author of this topic and myself just want to store the file in local drives. Letās say in different volumes(/dev/sdb,sdbc) or disks(D drive and E drive). and different users store the fs, commit, blocks file in different seafile-data folder.
No need to integrate with other cloud based storage.
I think this feature is required by most of the small company if they have more than 2 teams, eg, accounting and IT teams
I am using mutiple seafile-data folders which are mounted to different linux disks. However seems only working for what put in side ccnet/seafile.int
or maybe my conf is wrongā¦ hahahaha:rofl:
[
{
āstorage_idā: āseafile-dataā,
ānameā: āseafile-dataā,
āis_defaultā: true,
āfor_new_libraryā: false,
āfsā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā},
ācommitsā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā},
āblocksā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā}
},
{
āstorage_idā: ānew_storageā,
ānameā: ānew_storageā,
āfor_new_libraryā: true,
āis_defaultā: false,
āfsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/newseafile-dataā},
ācommitsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/newseafile-dataā},
āblocksā: {ābackendā: āfsā, ādirā: ā/storage/seafile/newseaflle-dataā}
}
]
Do you use Seafile PE or CE? Support for multiple storage backends is a PE feature.
The feature does not care if you use two filesystem-based backends or S3 + Swift or whatever.
I had a quick look at your config. Looks fine to me, but I am not a multi-backend expert. There is one thing though that you might want to change:
seaflle should be seafile.
Do these folders exist?
thanks your help.haha yes, i think that is my typO errorā¦but i manually create those folder before I changed seafile.ini content.
I am reinstalling pro version 7.0.5 now and see whether it worksā¦soon update hereā¦heheh
I am curious whether it works as search a lot but no any final solution yet.
I tested againā¦still failed.
I read both Chinese and English github manual again for this multiple storage backend setup and reinstall new seafile pro 7.0.5 version.
I followed exactly the āuser chosenā option:
User Chosen
This policy lets the users to choose which storage class to use when creating a new library. The users can select any storage class thatās been defined in the JSON file.
To use this policy, add following options in seahub_settings.py:
STORAGE_CLASS_MAPPING_POLICY = āUSER_SELECTā
If you enable storage class support but donāt explicitely set STORAGE_CLASS_MAPPING_POLIICY in seahub_settings.py, this policy is used by default.
and Here is the storage class I defined even though all local file system??? here is my doubt, if I use different seafile-data folder will it support in this mutiple backend storage portion?
[root@app01 conf]# cat seafile_storage_classes.json
[
{
āstorage_idā: āseafile-dataā,
ānameā: āseafile-dataā,
āis_defaultā: true,
āfsā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā},
ācommitsā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā},
āblocksā: {ābackendā: āfsā, ādirā: ā/seafile/seafile-dataā}
},
{
āstorage_idā: ācold_storageā,
ānameā: ācold_storageā,
āis_defaultā: false,
āfsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/cold_seafile-dataā},
ācommitsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/cold_seafile-dataā},
āblocksā: {ābackendā: āfsā, ādirā: ā/storage/seafile/cold_seafile-dataā}
},
{
āstorage_idā: ānew_storageā,
ānameā: ānew_storageā,
āis_defaultā: false,
āfsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/new_seafile-dataā},
ācommitsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/new_seafile-dataā},
āblocksā: {ābackendā: āfsā, ādirā: ā/storage/seafile/new_seafile-dataā}
}
]"
cat seahub_settings.py
here is the seahub_setting.py
}
ENABLE_SYS_ADMIN_VIEW_REPO = True
MULTI_INSTITUTION = True
EXTRA_MIDDLEWARE_CLASSES += (
āseahub.institutions.middleware.InstitutionMiddlewareā,
)
ENABLE_STORAGE_CLASSES = True
STORAGE_CLASS_MAPPING_POLICY = āUSER_SELECTā
#STORAGE_CLASS_MAPPING_POLICY = āREPO_ID_MAPPINGā
#STORAGE_CLASS_MAPPING_POLICY = āROLE_BASEDā
ENABLED_ROLE_PERMISSIONS = {
ādefaultā: {
ācan_add_repoā: True,
ācan_add_groupā: True,
ācan_view_orgā: True,
ācan_use_global_address_bookā: True,
ācan_generate_share_linkā: True,
ācan_generate_upload_linkā: True,
ācan_invite_guestā: True,
ācan_connect_with_android_clientsā: True,
ācan_connect_with_ios_clientsā: True,
ācan_connect_with_desktop_clientsā: True,
āstorage_idsā: [ ānew_storageā, ācold_storageā,āseafile-dataā], <---- add the storage_id here
}
}
"
as the āfor_new_libraryā: true, option is only mentioned in libarary ID mapping option so i removed it from my json file.
However I tested, frontend portal when I create new libarary by using new user account, still everything goes to old seafile-data folder. There is no place to āCHOSENā the storage class.
A quick update hereā¦ I am able to make it work by doing some manual work.
i changed policy to repo id mapping
STORAGE_CLASS_MAPPING_POLICY = āREPO_ID_MAPPINGā
firstly, i create new library from web, and there is a folder created in old seafile-data commit folder.
I manually changed database storage_id for the newly created library. can see the new id is ānew_storageā which point to āfsā: {ābackendā: āfsā, ādirā: ā/storage/seafile/new_seafile-dataā},
then I copy old commits folder(located at old seafile-data folder,as i didnt upload anything yet,so no folder under āfsā and ābloackā) to this new commit folder(commits": {ābackendā: āfsā, ādirā: ā/storage/seafile/new_seafile-dataā},)
then wolaā¦ all working ā¦
I am able to upload files to this new library and at backend all files are stored in new folder. and other libaries no impact.
but it is really not a good way even can make it work.
So my guessing is, need to populate storage_id information before you config the storage class.
If anyone has any finding or can contact the dev team. Please help me to ask for it.
thanks
I can confirm that this feature is working as intended now, in version 10 of pro edition (probably earlier). Iām doing a Docker setup, and following the docs exactly does actually work (assuming you donāt mess up a docker volume path string or something like that). And if you do mess that up (like I did) I donāt think itās always clear what you should/should not do while youāre troubleshooting and fixing it, so Iām planning to open a docs PR soon to help improve it.
For example, if you destroy/recreate any of the containers along the way, you cannot simply copy/paste entire config files from backup and expect it to work after you restart the containers, because there are credentials/keys to the database and stuff in there that get regenerated when you recreate the containers. If you do that, youāll likely see errors about DB authentication in the logs, and youāll get stuck on the 502 Bad Gateway screen forever.
This is the test that seems to be working for me:
I added the following config to seafile.conf
:
[storage]
enable_storage_classes = true
storage_classes_file = /shared/seafile/conf/seafile_storage_classes.json
[memcached]
memcached_options = --SERVER=memcached --POOL-MIN=10 --POOL-MAX=100
Then defined this storage class config file just for testing (Iām going to replace the backend dirs with the system mount points of multiple external disks now that I have observed this working properly):
[
{
"storage_id": "vdisk1",
"name": "vDisk 1",
"is_default": false,
"fs": {
"backend": "fs",
"dir": "/shared/vdisk1"
},
"commits": {
"backend": "fs",
"dir": "/shared/vdisk1"
},
"blocks": {
"backend": "fs",
"dir": "/shared/vdisk1"
}
},
{
"storage_id": "disk2",
"name": "vDisk 2",
"is_default": false,
"fs": {
"backend": "fs",
"dir": "/shared/vdisk2"
},
"commits": {
"backend": "fs",
"dir": "/shared/vdisk2"
},
"blocks": {
"backend": "fs",
"dir": "/shared/vdisk2"
}
}
]
Finally I added this to the bottom of seahub_settings.py
for the UI:
ENABLE_STORAGE_CLASSES = True
STORAGE_CLASS_MAPPING_POLICY = 'USER_SELECT'
I recreated the seahub docker container ONLY (running in gunicorn mode) to reload the backend and UI apps: docker compose up -d --force-recreate seafile
I wait about 15sec or so for the container to come up again, and I can see that it has created the vdisk1 and vdisk2 directories on my filesystem.
# tree -L 3
// This is the directory mounted to the seafile container @ /shared
.
āāā vdisk1
ā āāā storage
ā ā āāā blocks
ā ā āāā commits
ā ā āāā fs
ā āāā tmpfiles
āāā vdisk2
āāā storage
ā āāā blocks
ā āāā commits
ā āāā fs
āāā tmpfiles
In the UI, when I create a new library, I get this option now: