I maintain a plugin that depends on ruby-vips, and I'm having trouble with dependency managment

[rbuchberger]

Hey all, firstly I really appreciate this project. I maintain jekyll_picture_tag, which is a jekyll plugin that takes the pain out of responsive images. I'm using vips via ruby-vips to handle all the image generation, and when it works it works really well. Build times are 1/5 of what they were when we were using imagemagick, and I really didn't have to write much code to interface with it. Thanks a ton for that!

The problems crop up when we try to deploy with it. Netlify builds fail on PNG images, and it looks like mac builds are completely broken. I know how to operate a package manager, but beyond that I'm not very smart on dependency management.

Netlify has beta support for homebrew, but brew install libpng webp && build_site didn't work, and brew install vips && build_site tried to compile about half an OS from source and ran over the build time limit.

I don't even know where to start troubleshooting the MacOS issues, seeing as I don't have a mac to test on.

I'd like to offer guidance on how to compile & use a static binary with all dependencies included, but I don't have the slightest idea how to do that. Do y'all have any advice? Is that a good solution, or is there a better way to handle environment-agnostic deployment?

[jcupitt]

Hi @rbuchberger, your project sounds very cool!

libvips recently switched to libspng for PNG loading, perhaps this is causing problems? It tends to be stricter about PNG conformance and rejects some files which the older libpng accepted.

On macos, libvips 8.10.6 (pushed to homebrew last night) tries to relax libspng rules a bit and should load most PNGs that libpng can load. I would try updating to that. More recent libpng versions are better with out of spec PNGs as well.

On the VipsOperation: class "magicksave" not found in {file path} error, is that for GIF save? You'll need imagemagick development headers if you are building your own libvips.

Generally for deployment, I would use the platform libvips. Most linuxes have a reasonable version in their package managers, and building your own binary is annoying.

If the platform libvips is too old, you must either build it yourself (argh), or deploy on something like docker.

It's possible to build locally and then distribute the libvips binary as a tarball, but it would take a bit of skill to make it work well.

[jcupitt]

Here's libspng:

https://libspng.org/

It's a modern, clean, fast implementation of PNG load. libpng (the usual PNG load library) is hard to use, slow and has had almost endless security problems, so libspng ought to be step forward.

[rbuchberger]

Thanks! I didn't know about the libspng difference, maybe that's what was breaking the netlify builds! I'll give that another shot. I'll also pass on the advice about updating the latest libvips version with homebrew.

The magicksave error is because of my questionable error handling; whenever vips raises an error, we try again with magicksave:

    def write(image)
      image.write_to_file(base.absolute_filename, **write_opts)

      rescue Vips::Error
        opts = write_opts.transform_keys do |key|
          key == :Q ? :quality : key
        end

        image.magicksave(base.absolute_filename, **opts)
    end

I really should do a better job with that.

Anyway, I was afraid that packaging it ourselves would be painful. I was under the impression that most linux distros ship reasonable versions of vips, that's half the reason I switched away from imagemagick!

Thanks for the advice, I'll report back with results.

[jcupitt]

Ah OK. libvips already falls back to imagemagick if its own loaders fail. I would remove that step I think, unless you have a very specific reason to do it.

On macos, you need to use homebrew ruby, I don't know if that could be part of the problem. Apple ship their own /usr/bin/ruby, but it's not very useful, unfortunately. Just brew install ruby before you start.

[jcupitt]

libvips image loader detection works like this:

1. Every loader is a subclass of VipsForeignLoad. The base class has fields called priority and a virtual method called is_a.
2. The is_a method in each loader sniffs the first few byes of the file and tries to decide whether or not it's a file it can handle. Usually the first four byes is enough, but some formats (like CSV or SVG) are a bit more complex.
3. The imagemagick loader has a sniffer of sorts, but is very slow, unfortunately. IM supports things like AVI video files and they take an age to detect (up to several seconds).
4. On new_from_file, libvips finds all subclasses of VipsForeignLoad, sorts by priority order (fast detectors, and detectors for common formats have high priority) and runs the is_a sniffers in order.
5. The first is_a to signal that it can handle the file gets to do the load.

The imagemagick loader has the lowest priority and will run if everything else fails. There are two shortcuts for ICO and BMP to get magickload going quickly on them.

You can see the class hierarchy with vips -l, eg.:

$ vips -l fileload
      VipsForeignLoad (fileload), file loaders, priority=0
        VipsForeignLoadCsv (csvload_base), load csv, priority=0
          VipsForeignLoadCsvFile (csvload), load csv (.csv), priority=0, get_flags, get_flags_filename, header, load
          VipsForeignLoadCsvSource (csvload_source), load csv, priority=0, is_a_source, get_flags, header, load
        VipsForeignLoadMatrix (matrixload_base), load matrix, priority=0
          VipsForeignLoadMatrixFile (matrixload), load matrix (.mat), priority=0, is_a, get_flags, get_flags_filename, header, load
          VipsForeignLoadMatrixSource (matrixload_source), load matrix, priority=0, is_a_source, get_flags, header, load
...

So the CSV loader has no is_a method, but the stream CSV loader does, and both have "neutral" priority.

[rbuchberger]

That makes a lot of sense, thanks for the breakdown. Do savers work the same way? vips -l filesave shows magicksave for .gif and .bmp, but not jp2, which is why I added the rescue. If I remove it, attempting to convert jpeg to jp2 on my particular installation results in a 'no known saver' error. (jp2 support is something I'd like, because safari doesn't support webp or avif.)

That blanket rescue is also suppressing the original error that's breaking other builds, which is spectacularly unhelpful, so I'll at minimum be adding an option to disable it. I'm feeling a bit foolish that I didn't think to take this step first; without the original error message it's obvoiusly going to be difficult to troubleshoot build failures.

[jcupitt]

Savers are just done on suffix: every subclass of VipsForeignSave has a priority and set of supported suffixes. The first saver with a matching suffix gets the job.

There's nice jp2 load/save coming in 8.11, fwiw.

[rbuchberger]

Ok, makes sense. I'll write an update to handle this better. Glad to hear jp2 is incoming!

Will report back with results, might take a few days.

[rbuchberger]

Update: I have the original error message!

(snip)
2:10:17 PM: Generating new image file: projects/screenshot-mtns-mobile-400-7355c16b0.png
2:10:17 PM:   Liquid Exception: unable to call VipsForeignSavePngFile: unknown option Q in projects.html
2:10:17 PM:                     ------------------------------------------------
2:10:17 PM:       Jekyll 4.2.0   Please append `--trace` to the `build` command 
2:10:17 PM:                      for any additional information or backtrace. 
2:10:17 PM:                     ------------------------------------------------
2:10:17 PM: /opt/build/cache/bundle/ruby/2.7.0/gems/ruby-vips-2.0.17/lib/vips/operation.rb:355:in `block in call': unable to call VipsForeignSavePngFile: unknown option Q (Vips::Error)
(snip)

Did the key for the quality setting for PNG files change recently?

[jcupitt]

The Q parameter for pngsave was added in v8.7, back in 2017. Perhaps they have an older libvips?

It's not a very useful thing --- it sets quantization quality for palette encoding. I'm not sure I'd use palette PNGs unless I was certain the image was vector art and had no gradients.

[jcupitt]

I had a quick look at your plugin and it seems very nicely put together. I did notice one thing: you're not using thumbnail. This can give a really useful speedup for many image formats, and can boost quality significantly for things like PDF and SVG.

For example, here's roughly what you're doing now:

#!/usr/bin/ruby

require 'vips'

target_width = 200
image = Vips::Image.new_from_file ARGV[0]
scale_factor = target_width.to_f / image.width
image = image.resize scale_factor
image.write_to_file ARGV[1]

I see:

john@banana ~/try $ vipsheader x.jpg
x.jpg: 200x133 uchar, 3 bands, srgb, jpegload
john@banana ~/try $ /usr/bin/time -f %M:%e ./resize.rb ~/pics/nina.jpg x.jpg
189684:0.24

That's the fastest of five runs: 190mb of memory and 0.24s of elapsed time. thumbnail is open and resize in one operation, so you could write:

#!/usr/bin/ruby

require 'vips'

target_width = 200
image = Vips::Image.thumbnail ARGV[0], target_width
image.write_to_file ARGV[1]

I see:

john@banana ~/try $ /usr/bin/time -f %M:%e ./thumb2.rb ~/pics/nina.jpg x.jpg
60512:0.13

60mb and 0.13s. It's faster because it can exploit tricks like shrink on load. jp2 is even more dramatic:

john@banana ~/try $ /usr/bin/time -f %M:%e ./resize.rb ~/pics/4888.jp2 x.jpg
141468:1.04
john@banana ~/try $ /usr/bin/time -f %M:%e ./thumb2.rb ~/pics/4888.jp2 x.jpg
71532:0.13

8x faster, half the memory use.

[jcupitt]

ruby docs for thumbnail here:

https://www.rubydoc.info/gems/ruby-vips/Vips%2FImage.thumbnail

main docs with more detail here:

https://libvips.github.io/libvips/API/current/libvips-resample.html#vips-thumbnail

There's also thumbnail_source, which you can use to efficiently thumbnail directly from a URL, for example:

https://github.com/libvips/ruby-vips/blob/master/example/connection.rb

There's a blog post about it here: https://libvips.github.io/libvips/2019/12/11/What's-new-in-8.9.html

[rbuchberger]

That's all really helpful, thanks so much! And thanks for the kind words, though I'm not sure how much I deserve them. I only barely know what I'm doing 😄

I'll definitely be switching over to use thumbnail; I had steered away from it because of the name and description. Simpler code and better performance makes it an easy decision.

Regarding PNG quality, I'll just stop passing the option. Since it's breaking things for older versions of vips that we're often stuck with, and it's not all that useful to begin with, it's best to just forget about it.

You've been amazingly helpful, I really appreciate the time you've spent to walk me through all this. I'll let you know what the performance improvement is from switching to thumbnail.

[rbuchberger]

Weird, I'm seeing about a ~10% slowdown for the entire site build when using thumbnail instead of new_from_file + resize + smartcrop (~66s vs ~60s for my particular site.) I'm always passing a height: argument, and sometimes a crop but not usually. The write_to_file arguments don't change, but I'm passing a quality setting for all image formats that aren't gif or png.

The plugin's most common workflow is to take a source image and generate 3-5 sizes of that image in 2-3 formats. Inputs are usually jpg or png, outputs are usually webp and whatever the input is.

Looking at the rbspy flamegraph, most of that time is spent in write_to_file (vips_cache_operation_build), though I believe that libvips doesn't actually perform any operations until some form of write is called?

Here's the thumbnail implementation I came up with:

    def load_image
      Vips::Image.thumbnail source.name, @base.width, **load_opts
    end

    def write(image)
      case handler
      when :vips
        image.write_to_file(base.absolute_filename, **write_opts)
      when :magick
        image.magicksave(base.absolute_filename, **write_opts)
      end
    end

    def load_opts
      opts = { height: @base.height }
      # PictureTag.keep returns an interestingness setting for smartcrop.
      opts[:crop] = PictureTag.keep(@source.media_preset) if @source.crop?
      
      opts
    end

    def write_opts
      # Image specific write options, such as avif compression algorithm.
      opts = PictureTag.preset['image_options'][@base.format] || {}

      opts[:strip] = PictureTag.preset['strip_metadata']

      # quality_key switches beween Q: and quality: as appropriate. 
      opts[quality_key] = base.quality unless %w[gif png].include? base.format

      opts.transform_keys(&:to_sym)
    end

This is repeated for every output image. Is there something I can do better?

To be completely clear, this is still 5 times faster than our old imagemagick based implementation. If there's an easy performance gain here I'm interested, but if not it's no sweat.