Getting "no space left", but over 75% space left on drive

I have seafile server running on Debian Jessie on ext4 and the seafile-data on btrfs through VirtualBox.

Seafile client on my 3 computers will not sync. I check server log files and keep getting:

tail /opt/seafile/logs/seafile.log

[02/18/2017 07:26:29 AM] …/common/obj-backend-fs.c(233): [obj backend] Failed to open tmp file /mnt/seafile-store/seafile-data/storage/commits/7bbe8c18-f3da-4513-ac69-ddf4a6baaf79/65/e65091f96e704fa0b1afa5eafd9cdc6d85a0b2.9DALVY: No space left on device.

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.3G 3.4G 5.5G 39% /
udev 10M 0 10M 0% /dev
tmpfs 403M 5.7M 397M 2% /run
tmpfs 1006M 0 1006M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1006M 0 1006M 0% /sys/fs/cgroup
/dev/sdb1 80G 18G 62G 23% /mnt/seafile-store
tmpfs 202M 0 202M 0% /run/user/1000

# btrfs filesystem df -h /mnt/seafile-store/
Data, single: total=76.97GiB, used=15.34GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.50GiB, used=1.12GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=368.00MiB, used=0.00B

# df -ih
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 612K 115K 497K 19% /
udev 249K 344 249K 1% /dev
tmpfs 252K 506 251K 1% /run
tmpfs 252K 1 252K 1% /dev/shm
tmpfs 252K 4 252K 1% /run/lock
tmpfs 252K 13 252K 1% /sys/fs/cgroup
/dev/sdb1 0 0 0 - /mnt/seafile-store
tmpfs 252K 4 252K 1% /run/user/1000

I’ve read it could be an inode issues, but looks like I have inodes available and also read BTRFS doesn’t run out of inodes and/or have this issue.

I’ve ran
# /opt/seafile/seafile-server-lastest/seaf-gc.sh
and
# /opt/seafile/seafile-server-lastest/seaf-fsck.sh

After running garbage collection it seems to work for an hour and then back to not working saying I’m out of space.

I’m lost and don’t know what to do. I feel I’m looking at the wrong thing since I believe I still have plenty of space left on the drives.

could you check what is reported by ‘btrfs filesystem usage /mnt/seafile-store’ ?

are you able to do writes there?

I pasted “btrfs filesystem df -h /mnt/seafile-store/” in the OP. Is there another command I should run to get that info?

Is this what you mean? I just found this online.

btrfs filesystem show

Label: ‘seafile-store’ uuid: 5a68818f-3b4c-4130-a963-d12956944aa4
Total devices 1 FS bytes used 16.47GiB
devid 1 size 80.00GiB used 80.00GiB path /dev/sdb1

Btrfs v3.17

I do see devid saying 80.00GiB used. Is that my issue? What does that mean so I understand it if that’s the issue?

I think it’s a BTRFS issue and I’m focusing on that.

https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_get_.22No_space_left_on_device.22_errors.2C_but_df_says_I.27ve_got_lots_of_space

I’ll post if I need more help unless someone knows of this BTRFS issue and a resolution?

Was the btrfs metadata eating all the reserved space and had to run command to fix.

btrfs fi balance start -dusage=5 /mnt/seafile-data

yes, I did not ask for df -h but for usage.
That command would have shown the high metadata usage

But good you sorted it out

I use this scripts on my Server: https://github.com/kdave/btrfsmaintenance

Very good for doing scrub/balance one a week per cronjob to avoid such problems with btrfs.

Thanks for that link. I was going to try to make my own scripts, but that will work way better with my limited skills.