New example: content-aware image resizing with seam-carving #123
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Seam carving is an algorithm for "content-aware rescaling", allowing an image to be rescaled while somewhat preserving the interesting features of the image. It works by removing low-energy "seams" (vertical or horizontal paths of pixels) to reduce either the width or height of the image. For more info, see https://en.wikipedia.org/wiki/Seam_carving. This patch provides a parallel implementation. It can remove as many seams as desired, and can also generate an animation (as a .gif) of the process, showing how the image shrinks gradually as more seams are removed. It's super cool... check it out! Implementing seam carving efficiently turned out to be quite interesting. It's not as parallel as it seems (seams? haha). This is in part because removing seams is entirely sequential: we can't remove the Nth seam until we have removed the (N-1)th seam. But removing a single seam should be highly parallel though, right? Not exactly. Let's see some of the details... At the center of the algorithm is a dynamic-programming equation of the following form. (For vertical seams; horizontal are symmetric). M(i,j) = E(i,j) + min(M(i-1, j-1), M(i-1, j), M(i-1, j+1)) This equation at first appears highly parallel, because for example each row of the image can be processed entirely in parallel. However, in practice, it is difficult to extract this parallelism on multicore, because a typical image will only be at most a few thousand pixels wide. With proper granularity control of say up to 1000 pixels as a unit, there's not much parallelism left! So, row-major DP order isn't going to work. But this is not the only possible ordering. Here's a better one. Imagine breaking up the image into a bunch of strips, where each strip is then divided into triangles. If each triangle is processed sequentially row-wise from top to bottom, then we can compute a strip in parallel by first doing all of upper triangles (#), and then doing all of the lower triangles (.): +------------------------------------+ strip 1 -> |\####/\####/\####/\####/\####/\####/| |.\##/..\##/..\##/..\##/..\##/..\##/.| |..\/....\/....\/....\/....\/....\/..| strip 2 -> |\####/\####/\####/\####/\####/\####/| |.\##/..\##/..\##/..\##/..\##/..\##/.| |..\/....\/....\/....\/....\/....\/..| strip 3 -> | | Now, we can choose an appropriate grain size for a single triangle and still manage to fit a decent number of triangles horizontally. At the end of the day, however, this only gets us so far...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Seam carving is an algorithm for "content-aware resizing", allowing an image to be re-scaled while attempting to preserve the interesting features of the image. It works by removing low-energy "seams" (vertical or horizontal paths of pixels) to reduce either the width or height of the image. For more info, see https://en.wikipedia.org/wiki/Seam_carving.
This patch provides a parallel implementation. It can remove as many seams as desired, and can also generate an animation (as a .gif) of the process, showing how the image shrinks gradually as more seams are removed. It's super cool... check it out!
As part of this patch, I also implemented support for generating GIF files. This turned out to be an interesting problem of its own, because GIFs use a custom form of LZW compression. Very tricky to get the bits right...