Memory Leak

[thiagopromano]

Hey, it seems there is a memory leak on Vix/Vips, probably on the Operation pipeline.

I've opened an issue on Image as I found the problem there initially, but after digging through the code, the problem is on Vix.

This is an example where the leak can be reproduced, it calculates the dhash of an image.

It can be saved to a file such as leak.exs on the root and run as mix run leak.exs.

Note that the memory usage increases and it isn't reported by the Erlang carriers, which points to a leak on the NIFs. It also greatly surpasses the reported cache size on Vix.Vips.tracked_get_mem/1.

Also, it is never freed until the BEAM stops.

hash = fn image ->
  hash_size = round(:math.sqrt(64))

  {:ok, conv_image_temp} = Vix.Vips.Image.new_matrix_from_array(2, 1, [[1.0, -1.0]])
  {:ok, convolution} = Vix.Vips.Operation.cast(conv_image_temp, :VIPS_FORMAT_FLOAT)

  {:ok, pixels} =
    image
    |> Vix.Vips.Operation.thumbnail_image!(hash_size + 1,
      size: :VIPS_SIZE_FORCE,
      height: hash_size
    )
    |> then(fn img ->
      if Vix.Vips.Image.has_alpha?(img) do
        Vix.Vips.Operation.flatten(img, [0, 0, 0])
      else
        img
      end
    end)
    |> Vix.Vips.Operation.colourspace!(:VIPS_INTERPRETATION_B_W)
    |> Vix.Vips.Operation.extract_band!(0)
    |> Vix.Vips.Operation.cast!(:VIPS_FORMAT_INT)
    |> Vix.Vips.Operation.conv!(convolution)
    |> Vix.Vips.Operation.extract_area!(1, 0, hash_size, hash_size)
    |> Vix.Vips.Operation.relational_const!(:VIPS_OPERATION_RELATIONAL_MORE, [0])
    |> Vix.Vips.Operation.linear!(Enum.map([255.0], &(1.0 / &1)), [0.0])
    |> Vix.Vips.Operation.cast!(:VIPS_FORMAT_UCHAR)
    |> Vix.Vips.Image.write_to_binary()

  dhash = for <<_::7, v::1 <- pixels>>, into: <<>>, do: <<v::1>>
  {:ok, dhash}
end

{:ok, im} = Vix.Vips.Image.new_from_file("test/images/puppies.jpg")

task = Task.async(fn ->
  _ = for _i <- 1..20000, do: hash.(im)
end)


Task.await(task, :infinity)

IO.puts("done, check memory usage")

Process.sleep(10000)

[akash-akya]

Hey @thiagopromano, thanks for the detailed issue, I'll try to check it this weekend

[akash-akya]

@thiagopromano I fixed the memory leak, if possible can you test fix-bug-169 branch?

{:vix, github: "akash-akya/vix", branch: "fix-bug-169"}

I think I introduced this bug during a recent change. I should invest sometime in creating automated checks for catching things like this in regression.

I'll merge and cut a release soon.

[thiagopromano]

Hey @akash-akya, we've tested it and it is definitely fixed.

Thank you so much for spending your time with this project!

[akash-akya]

@thiagopromano FYI, pushed a new release with the fix -- https://hex.pm/packages/vix/0.31.0