Error generating thumbnails for large JPGs

Seafile Server
,
1

I’ve been having issues with thumbnails for JPG files with a large resolution and/or file size (mostly panorama’s made with my phone). I found a workaround for it and like to share it with you, hoping to find a permanent fix.

After uploading these large JPG files, the thumbnails do not get generated and the full image does not load on the web front-end or the Android app.

seahub.log shows the following error:
[ERROR] seahub.thumbnail.utils:149 generate_thumbnail broken data stream when reading image file

I tried fixing it with the following attempts:

  1. Increasing thumbnail size limits in the seahub_settings.py configuration file:
    FILE_PREVIEW_MAX_SIZE = 100 * 1024 * 1024
    THUMBNAIL_IMAGE_SIZE_LIMIT = 100 # MB

  2. (Re)installing and upgrading Pillow:
    apt install zlib1g-dev libjpeg-dev
    apt remove python3-pil # removes Pillow 7.0.0
    pip3 install --upgrade Pillow # installs Pillow 8.0.1
    source: Seahub "Failed to create thumbnail"

I did some debugging/research and found the following upstream bug report:

As suggested in that thread I could work around my issue with the following patch in Seafile:

--- seafile-server/seahub/seahub/thumbnail/utils-orig.py        2021-01-02 02:57:37.337404053 +0000
+++ seafile-server/seahub/seahub/thumbnail/utils.py     2021-01-02 02:29:50.892236856 +0000
@@ -12,6 +12,7 @@
 except:
     from urllib.request import urlretrieve
 
+from PIL import ImageFile
 from PIL import Image
 from seaserv import get_file_id_by_path, get_repo, get_file_size, \
     seafile_api
@@ -217,6 +218,7 @@
 
     `fp` can be a filename (string) or a file object.
     """
+    ImageFile.LOAD_TRUNCATED_IMAGES = True
     image = Image.open(fp)
 
     # check image memory cost size limit

Does anyone have a suggestion on what might be the root-cause or how to fix this properly?

I am running Seafile Server 7.1.5 on Ubuntu 20.04.

The smallest file in my collection with this issue is 16.6MB.
The smallest resolution is 5296x3872 pixels.
There are many other (smaller) pictures that work just fine.

2

This seems to be a similar problem that I am having. Previews are generating for smaller images, but when I upload large resolution images the only way to view the files is to download. Did you manage to get a fix in place?

3

I experienced this issue, and while this didn’t provide the fix, it pointed me in the right spot.

The THUMBNAIL_IMAGE_ORIGINAL_SIZE_LIMIT setting is misleading to me.

In the seafile-server/seahub/seahub/thumbnail/utils.py script it is calculated as:

    # check image memory cost size limit
    # use RGBA as default mode(4x8-bit pixels, true colour with transparency mask)
    # every pixel will cost 4 byte in RGBA mode
    width, height = image.size
    image_memory_cost = width * height * 4 / 1024 / 1024
    if image_memory_cost > THUMBNAIL_IMAGE_ORIGINAL_SIZE_LIMIT:
        return (False, 403)

I have a JPEG image that is 15.1 MB in size on disk. It is 12,000 pixels by 5596 pixels though. So by the calculation above, it’s memory cost is ~256MB.

So I had to set the THUMBNAIL_IMAGE_ORIGINAL_SIZE_LIMIT setting much higher than I thought in order for my thumbnails to display.

Hope this helps others!