Input file has corrupt header

[maxbec]

The bug

I get corrupt header errors on a lot of images after mass upload. manual thumbnail creation does not fix it.

This is the error on the example image:

[Nest] 6  - 11/28/2024, 8:32:12 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:539: Failed to decode DCT block

[Nest] 6  - 11/28/2024, 8:32:12 AM   ERROR [Microservices:JobService] Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:539: Failed to decode DCT block

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:158:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:108:25)

    at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:151:28)

    at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:122:13)

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 6  - 11/28/2024, 8:32:12 AM   ERROR [Microservices:JobService] Object:

{

  "id": "2c6eb1f0-2de2-4abc-9230-f15506c59a04"

}

This is one example image:

The OS that Immich Server is running on

Unraid 7

Version of Immich Server

v1.121.0

Version of Immich Mobile App

latest

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich-server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro

      # Custom mounts for external libraries
      - ${UPLOAD_LOCATION}/external-libraries:/mnt/media/nas/external-libraries:ro
    env_file:
      - stack.env
    ports:
      - '2284:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
    networks:
      - default
      - external

  immich-machine-learning:
    container_name: immich-machine-learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - stack.env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich-redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich-postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command:
      [
        'postgres',
        '-c',
        'shared_preload_libraries=vectors.so',
        '-c',
        'search_path="$$user", public, vectors',
        '-c',
        'logging_collector=on',
        '-c',
        'max_wal_size=2GB',
        '-c',
        'shared_buffers=512MB',
        '-c',
        'wal_compression=on',
      ]
    restart: always

volumes:
  model-cache:

networks:
  external:
    external: true

Your .env content

UPLOAD_LOCATION=/mnt/remotes/nas/immich
DB_DATA_LOCATION=/mnt/user/appdata/immich
IMMICH_VERSION=release
DB_PASSWORD=**************
DB_USERNAME=**************
DB_DATABASE_NAME=immich
TZ=Europe/Berlin

Reproduction steps

1. Mass upload images with immich-go
2. Run Thumbnail creation manually

Relevant log output

[Nest] 6  - 11/28/2024, 8:22:20 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:499: Skipped 8763971 bytes before marker 0xd9

VipsJpeg: ./lib/jpegli/decode.cc:645: jpegli_read_header: unexpected EOI marker.

[Nest] 6  - 11/28/2024, 8:22:20 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:499: Skipped 8763971 bytes before marker 0xd9

VipsJpeg: ./lib/jpegli/decode.cc:645: jpegli_read_header: unexpected EOI marker.

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:158:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:108:25)

    at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:151:28)

    at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:122:13)

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 6  - 11/28/2024, 8:22:20 AM   ERROR [Microservices:JobService] Object:

{

  "id": "bfe765cf-34cf-4796-9833-f456e9303aad"

}

[Nest] 6  - 11/28/2024, 8:22:21 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: magickload: Magick: ImproperImageHeader `upload/library/max/2015/Jan/07/P1000843.JPG' @ error/tga.c/ReadTGAImage/221 (null)

[Nest] 6  - 11/28/2024, 8:22:21 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: ImproperImageHeader `upload/library/max/2015/Jan/07/P1000843.JPG' @ error/tga.c/ReadTGAImage/221 (null)

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:158:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:108:25)

    at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:151:28)

    at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:122:13)

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 6  - 11/28/2024, 8:22:21 AM   ERROR [Microservices:JobService] Object:

{

  "id": "dd0a19d4-0bb1-49b5-b175-b8422ae9d3fa"

}

Additional information

No response

[bo0tzz]

Sounds like your files have corrupt headers ;)

[maxbec]

How can i check that? And how can i fix this... Because as you can see the file shows fine here in github and on my pc?

[johny-mnemonic]

Seems like I have the same issue.
Not sure with which version it started, but seems like the thumbnail generator has issue working with some files which are perfectly OK for other stuff.
immich errors
No clue how to find which are the problematic files as the error log doesn't contain filenames 🙁

@bo0tzz would you advice how to debug this?