extract full exif and xml metadata from jxl images with metadata compressed

[amitrea]

Add support to extract full EXIF and XML metadata from JPEG XL(JXL) files. For a detailed description of the current issue please see issue.

Currently, those that have all or the majority of the images in JPEG XL format will not be able to see/edit any EXIF/XML metadata (GPS data, original datetimes), searching images by metadata and geo tagging information.

[jrasm91]

Can you update the description to describe how you have tested this and what impact it has to the final size of the base image?

[amitrea]

For image size impact:

• checkout commit 13464ad. This commit is the last from the main branch (upstream).
• build dev and prod images:

$ cd server/
$ docker build --target dev -t docker.svc.local/private/base-server-dev:20231124 .
$ docker build --target prod -t docker.svc.local/private/base-server-prod:20231124 .
$ docker images

REPOSITORY                                   TAG                 IMAGE ID      CREATED         SIZE
docker.svc.local/private/base-server-prod    20231124            68a6f980c95f  27 seconds ago  820 MB
docker.svc.local/private/base-server-dev     20231124            29d4a7a14462  3 minutes ago   1.7 GB

Prune all dangling images and delete:

• docker.svc.local/private/base-server-dev:20231124
• docker.svc.local/private/base-server-prod:20231124

$ docker image prune
$ docker image rm docker.svc.local/private/base-server-prod:20231124
$ docker image rm docker.svc.local/private/base-server-dev:20231124

Git swich to my fix

$ git switch fix/extract-full-exif-from-jxl

Build dev and prod images from scratch:

$ docker build --target dev -t docker.svc.local/private/base-server-dev:20231124 .
$ docker build --target prod -t docker.svc.local/private/base-server-prod:20231124 .
$ docker images

REPOSITORY                                        TAG                 IMAGE ID      CREATED             SIZE
docker.svc.local/private/base-server-prod         20231124            8f4bf913ab6c  13 seconds ago      821 MB
docker.svc.local/private/base-server-dev          20231124            cdda36347447  About a minute ago  2.27 GB

Prod image with the fix contains 1 additional MB, and dev image with the fix contains 570 additional MB.

Testing approach

I provided already in the issue URL from this PR description how to create a JXL image from an existing JPEG and also how to test without the fix and with the fix but I will try to make it a bit simpler, if it doesn't look like so.

Testing without the fix

Alternative 1 - Test against immich-app/base-server-dev or immich-app/base-server-prod

1. Take one of your JPG images that have GPS data (could be without GPS as well), an image that was taken in the past and put it your preferred host directory: for instance ~/media/test/1/image_name.jpg. If you already have an image in JXL format then put it into ~/media/test/1/image_name.jxl and skip step 4. Be sure that this JXL image has EXIF and/or XML metadata.
2. Run Immich base server prod

$ docker run -it -d --name base-server-prod -v "~/media/test/1:/media:rw" ghcr.io/immich-app/base-server-prod:20231109

3. Install libjxl tools and wget

$ apt update
$ apt install libjxl-tools wget

4. Convert JPEG file to JXL file

$ cjxl --lossless_jpeg=1 /media/image_name.jpg /media/image_name.jxl

5. Check the structure of JXL image and look for "Brotli-compressed Exif metadata" - something similar like the output below.

$ jxlinfo /media/image_name.jxl

JPEG XL file format container (ISO/IEC 18181-2)
JPEG XL image, 4608x3456, (possibly) lossless, 8-bit RGB
Orientation: 3 (Upside down)
Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative
JPEG bitstream reconstruction data available
Brotli-compressed Exif metadata: 20000 compressed bytes

6. Install Exiftool (you can change version of exiftool to 12.67)

$ cd ~
$ mkdir tmp
$ wget https://exiftool.org/Image-ExifTool-12.69.tar.gz
$ tar xvf Image-ExifTool-12.69.tar.gz
cd Image-ExifTool-12.69/

7. Test if Exiftool supports reading JXL, compressed EXIF and XML metadata. Search for a warning line: "Install IO::Uncompress::Brotli ...". In output below is line 2. There are no other dates other than the File:System datetimes, which is today at the time I converted the JPEG file into JXL. Also no GPS data or other metadata of the original JPEG file.

$ ./exiftool -a -u -G0:1 -s -ee /media/image_name.jxl

[ExifTool]      ExifToolVersion                 : 12.69
[ExifTool]      Warning                         : Install IO::Uncompress::Brotli to decode Brotli-compressed metadata
[File:System]   FileName                        : image_name.jxl
[File:System]   Directory                       : /media
[File:System]   FileSize                        : 4.0 MB
[File:System]   FileModifyDate                  : 2023:11:24 17:41:14+00:00
[File:System]   FileAccessDate                  : 2023:11:24 17:52:26+00:00
[File:System]   FileInodeChangeDate             : 2023:11:24 17:41:14+00:00
[File:System]   FilePermissions                 : -rw-r--r--
[File]          FileType                        : JXL
[File]          FileTypeExtension               : jxl
[File]          MIMEType                        : image/jxl
[File]          ImageWidth                      : 4608
[File]          ImageHeight                     : 3456
[Jpeg2000]      MajorBrand                      : JPEG XL Image (.JXL)
[Jpeg2000]      MinorVersion                    : 0.0.0
[Jpeg2000]      CompatibleBrands                : jxl
[Composite]     ImageSize                       : 4608x3456
[Composite]     Megapixels                      : 15.9

Alternative 2 - Test against immich-app/immich-server, the version 1.87.0 or later.

Do steps 1 to 5, inclusive, and step 2 exclusive from Alternative 1.

2. Run immich server

docker run -it -d --name immich-server -v "/mnt/data/media:/media:rw" --entrypoint bash ghcr.io/immich-app/immich-server:release

6. Check exiftool version that comes with exiftool-vendored. It should be 12.67 or later.

$ /usr/src/app/node_modules/exiftool-vendored.pl/bin/exiftool -ver

12.67

7. Extract EXIF and XML metadata from JXL using the exiftool that comes with exiftool-vendored, the library that is used by immich server and immich microservices. I suupose that immich microservices is in our interest.

$ /usr/src/app/node_modules/exiftool-vendored.pl/bin/exiftool -a -u -G0:1 -s -ee /media/image_name.jxl

The output is exactly like in Alternative 1, step 7.

Alternative 3 - Test against Immich browser app, the version 1.87.0 or later.

You need to have the JXL file (check Alternative 1).
Upload the JXL image to the immich server using immich-cli or immich-go or whatever you consider working.
Locate the uploaded JXL image, click on it and then on info and then check if it has the original JPEG metadata (datetimes or GPS), the metadata that Immich is exposing currently to the user. Check if the image appear on map.

Testing with the fix

Alternative 4 - Test against immich-app/base-server-dev or immich-app/base-server-prod that contains the fix. See at the beginning how I built them in order to see the impact of the fix on the size of the image.

1. Idem as in step 1, Alternative 1.
2. Run immich server

$ docker run -it -d --name immich-server-with-fix -v "/mnt/data/media:/media:rw" docker.svc.local/private/base-server-prod:20231124

Steps 3 to 6, inclusive should be the same.

7. Test if Exiftool supports reading JXL, compressed EXIF and XML metadata.

$ ./exiftool -a -u -G0:1 -s -ee /media/image_name.jxl

You should be able to see the full EXIF data of your original JPEG file

Alternative 5 - Test against immich-app/immich-server with the fix, similar with Alternative 2.

Alternative 6 - Test against Immich browser app, using images with the fix. You have to build yourself all 3 images (immich-server, immich-microservices and immich-machine-learning). For this you have to modify server/Dockerfile from immich repository:

• Search for "FROM ghcr.io/immich-app/base-server-dev" and replace with the base-server-dev you build and which contains the fix.
• Search for "FROM ghcr.io/immich-app/base-server-prod" and replace with the base-server-prod you build and which contains the fix.

Buuld images, deploy, upload the JXL file and check the JXL metadata by pointing to the image and then info. It should have the right datetimes and also GPS. The image should also appear on the map.

[bo0tzz]

@amitrea kudos for the most comprehensive PR writeup we've ever seen 😄

[amitrea]

I appreciate all your feedbacks.
Thank you.