Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bitmap convolution operations #94

Open
noelwelsh opened this issue Jun 11, 2020 · 0 comments
Open

Add bitmap convolution operations #94

noelwelsh opened this issue Jun 11, 2020 · 0 comments

Comments

@noelwelsh
Copy link
Contributor

noelwelsh commented Jun 11, 2020

Overview

Convolution is useful for image processing tasks, such as smoothing and edge detection, as well as being a creative tool. An example of what can be achieved using convolutions, and other image processing effects, is given at https://www.smashingmagazine.com/2015/05/why-the-svg-filter-is-awesome/

The goal of this task is to add convolutions to Doodle. Both Java2D and SVG support convolutions, but in different ways. Thus this task requires some API design work. For example, it might make more sense to have different algebras for the different backends.

Details

Convolutions typically apply to rectangular bitmaps, while pictures in Doodle can have arbitrary boundaries and are usually represented a vector shapes, not bitmaps. This means there are several implementation options:

  1. Convolutions apply to an entire Canvas and cannot be applied to individual elements. A convolution is only applied once a Picture is rendered to screen and applies to everything that is rendered.
  2. Convolutions apply to individual elements (a Picture). On the Java2D backend this will require rendering an element to a bitmap buffer, performing the convolution, and then rendering the bitmap ("compositing") back into the main image. In SVG this is directly supported.
  3. There could be an explicit representation of a bitmap (see Refactor bitmap support #85, Convert Picture to BufferedImage #87, and Add "immediate mode" algebra ("Doodle Canvas") #93 for related issues) on which convolutions are applied, and then methods to transform this bitmap to and from a Picture. This achieves the same effect as 2 but is more explicit and will be difficult to support on SVG.

A convolution algebra may look something like the below. This example corresponds to option 3 above, with an explicit representation of a Bitmap.

final case class Kernel(/* Some representation here */)

trait Convolve[Bitmap]{
  def convolve(bitmap: Bitmap, kernel: Kernel): Bitmap
}

See backend specific details:

  • Java convolutions involve much ceremony, as is typical for Java libraries.
  • SVG has a fairly complex DSL for specifying image filters, of which convolution is one part. It might be interesting to model all of this in Doodle.
  • the HTML canvas doesn't directly support convolution as far as I know, but it is straightforward to implement convolutions in software.

A few notes:

  • Convolution operators compose and can be multiplied by a scalar.

See #85 for a related issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant