compositing image

[Felix-N]

hi, i'm trying to create a big image out of several small images (about 250).

composite works fine for few images, but if the number gets bigger the ram usage explodes and it does not finish anymore.

i've tried to split it up into chaining several composite calls, but it appears that this does not work.

any hint how i could combine these images into a large one? most of the images do not overlap, though a few do.

[lovell]

Hi, does switching libvips' cache off via sharp.cache(false) help? Are you able to share a code sample?

[Felix-N]

Hi Lovell,
thanks for the quick reply, much appreciated.

I'm not sure if sharp.cache(false) helps - for sure it does not solve the issue.
A composite of 66 runs through, of 180 did not. Although the 66 was quite slow, and used about 10gigs of ram. I'm comparing to a mathlab solution, which takes about 10 seconds to finish.

I can share the code, no problem - how would it be best for you (I can share ?

Here the problem snippet

        sharp.cache(false);
        sharp({
          create: {
            width: 16123,
            height: 17973,
            channels: 3,
            background: { r: 0, g: 0, b: 0}
          }
        })
          .limitInputPixels(false)
          .composite(compLayer0)
          //.composite(compLayer1)
        .toFile(file,(err, info) => {
          if (err) {
            console.log(err);
          } else {
            console.log('COMPOSITE written');
          }
          });

compLayer0 and compLayer1 are both arrays of this shape:

     {
        input: e.image,
        // blend: 'source',
        top: config.trans_x,
        left: config.trans_y,
      }

If i use both composite layers, only the last layer is present in the output (trying with limited amounts of images, eg 5 each).

blend: 'source' did only result in one placed image, hence i commented it.

As a side note:
When I try to get a raw buffer instead of toFile, I'm getting an error:

Assertion failed: (length <= imp::kMaxLength && "too large buffer"), function NewBuffer, file ../node_modules/nan/nan.h, line 823.
Abort trap: 6

using:

        sharp.cache(false);
        sharp({
          create: {
            width: 16123,
            height: 17973,
            channels: 3,
            background: { r: 0, g: 0, b: 0}
          }
        })
          .limitInputPixels(false)
          .composite(compLayer0)
         // .composite(compLayer1)
           .raw()
           .toBuffer({ resolveWithObject: true })
           .then((obj) => {
             console.log(obj.info);
           })

[lovell]

Each layer is ~850MB, hence the rapid memory exhaustion. If there is no overlap then the possible future enhancement detailed in #1580 will help, otherwise you'll need to do this in sections, or a few layers at a time.

[Felix-N]

thanks, i naively was expecting the lib to merge down every image one after another.

how would i do it in segments? i thought i could do
.composite()
.composite()

but didnt get that to work. I guess i could get the toBuffer and create a new sharp object, but i'd like to avoid compression and decompression each time.

[lovell]

To avoid the (de)compression roundtrip try raw pixel input and output.

https://sharp.pixelplumbing.com/en/stable/api-constructor/#sharp
https://sharp.pixelplumbing.com/en/stable/api-output/#raw

[Felix-N]

the problem i have with raw is that it results in this error:

Assertion failed: (length <= imp::kMaxLength && "too large buffer"), function NewBuffer, file ../node_modules/nan/nan.h, line 823.
Abort trap: 6

[lovell]

An alternative might be to try an SVG with positioned, inline base64-encoded <image> elements - see https://raw.githubusercontent.com/lovell/sharp/master/test/fixtures/struct-image-04-t.svg for an example.

[Felix-N]

interesting idea, but unfortunately I need the end result as either a jpg or png for further processing.

I guess i'll look into moving that part over to python

[lovell]

You can convert from SVG to PNG using sharp(svgInput).toFile('out.png') - it uses librsvg under the hood.

[Felix-N]

sounds interesting, i'll give it a try - though first time using svg.

do you know how svg handles if those placed images overlap?

[Durss]

I'm also having huge memory issues with composite.
I want to build a grid of pictures (potentially thousands of 64x64px pictures) and quickly run out of memory.
I tried batching the composite into chunks of 100 pictures, but after 5 batches it crashes my node process.

I saw the arrayjoin() feature from VIPS that would perfectly fit my needs so i'm gonna give it a try although i like "zero dependency" of sharp ^^.

Thank you for your work !
It made my code way smaller/cleaner than with GraphicsMagick/ImageMagick !

Regards,
Durss

[Felix-N]

I'll give the svg a later try. The python variant works fine.

It feels as if composite currently places every image which needs placement into a new image of the target image size and in the end merges the complete stack.
Probably due to blending modes?

In my case the top layer just wins without any blending, hence just placing each image onto the target works fine, and does not blow up the memory usage. Numpy is currently fast enough as well

[lovell]

I'll close this as I'm not sure what else can be done for now but please do subscribe to #1580 for updates about the future possible feature that will definitely help.

[mgiraldo]

@lovell how would i go about creating the svg that i can composite tiles to? even with a 1024x1024 composite i'm getting crashes in node (expressjs).

i tried:

sharp('128_128.svg')
    .resize(1024, 1024)
    .composite(paths)
    .jpeg({ quality: 100 })
    .toBuffer()

and paths is a list of 700+ (each image is 32x32):

{
  input: inputImg,
  top: y,
  left: x
}

and this keeps crashing.