WrapMemory Pointer and Resizing

[ptasev]

I have really large image data (2 GB) in the form of span. I'm using the following code to do WrapMemory, but it seems the Resize mutate operation is incompatible. Is there any way to do the following without having to copy this massive image?

I don't have access to the bytes so I'm stuck with Span<L16> as the input. (using our own similar library where we MemoryMarshal.Cast our L16 pixel to ImageSharp's L16 pixel)

var stream = ...;
Span<SixLabors.ImageSharp.PixelFormats.L16> pixels = ...; // cast our image's pixel span to ImageSharp pixel type
var width = ...;
var height = ...;
unsafe
{
    fixed (void* pixPtr = pixels)
    {
        using var image=
            SixLabors.ImageSharp.Image.WrapMemory<SixLabors.ImageSharp.PixelFormats.L16>(pixPtr,
                width, height);

        image.Mutate(img => img.Resize(width / 4, height / 4, KnownResamplers.Bicubic));
        image.SaveAsJpeg(stream);
    }
}

[antonfirsov]

Internally, ResizeProcessor works by creating a new buffer so it can work as destination, then in the end swapping that buffer with the original one. The reason to do this is that otherwise certain cases would be really hard to implement. For example imagine, you are resizing the image into the bottom right corner of the same image as destination. Without copying you would overwrite the bottom right part of the source image while resizing the top rows.

What we could consider is to relax the following requirement to be >= instead of expecting strict equality, so a smaller buffer can be copied into a larger one when swapping is not possible:

ImageSharp/src/ImageSharp/Memory/DiscontiguousBuffers/MemoryGroup{T}.cs

Lines 189 to 193 in 08b0320

	if (target.TotalLength != source.TotalLength)
	{
	throw new InvalidMemoryOperationException(
	"Trying to copy/swap incompatible buffers. This is most likely caused by applying an unsupported processor to wrapped-memory images.");
	}

But note, that ResizeProcessor would still create a 2GB temporary buffer internally and do an extra copy in the end just to store the result pixels into your original buffer before you save the jpeg, so it's definitely not worth it in your case.

What I recommend instead, is to use Clone(...):

using var image= Image.WrapMemory<SixLabors.ImageSharp.PixelFormats.L16>(pixPtr, width, height);

// Only 2GB / (4*4) = 125 MB extra space is allocated, no unnecessary copies are done:
using Image resized = image.Clone(img => img.Resize(width / 4, height / 4, KnownResamplers.Bicubic));
resized.SaveAsJpeg(stream);

I think it's a pretty straightforward solution, and one of the main reasons for the Mutate / Clone API is to support such scenarios without extra copy.

[ptasev]

Thanks for the explanation! I wouldn't have thought about using Clone, but that makes sense since it's combined with mutate.