Add support of segmentation mask in Augmix layer #1988
Conversation
cosmo3769
force-pushed
the
augmix-segmentation-mask
branch
from
d4ac7a5
to
9db35c8
Compare
|
I am getting this lint error here, though this error is not picked up in my local: Similar error I am getting in this #1991. |
I'm taking a look |
|
Lint error is unrelated and will be fixed by #1993 |
| def augment_segmentation_mask( | ||
| self, segmentation_masks, transformation=None, **kwargs | ||
| ): | ||
| chain_mixing_weights = self._sample_from_dirichlet( |
There was a problem hiding this comment.
Sampling these random values differently for the image augmentation and the mask augmentation will cause the image + mask to be augmented inconsistently with one another (your demo image shows, for example, that the dog mask is rotated differently than the dog itself).
We should implement and use get_random_transformation to get a single chain_mixing_weights and a single weight_sample that is used for both images and labels
|
Hey @cosmo3769 this PR should not auto-close #1992 Could you please edit that part out? |
I've updated the description |
|
Awesome! This looks better now -- I tinkered with the demo and the behavior seems correct |
| @@ -28,6 +28,7 @@ | |||
| def normalize(input_image, input_mask): | |||
| input_image = tf.image.convert_image_dtype(input_image, tf.float32) | |||
| input_image = (input_image - mean) / tf.maximum(std, backend.epsilon()) | |||
| input_image = input_image / 255 | |||
There was a problem hiding this comment.
Does this break other demos?
There was a problem hiding this comment.
Actually this seems incorrect -- we're already scaling by mean and stddev so this shouldn't be done
There was a problem hiding this comment.
No, I checked other demos as well.
There was a problem hiding this comment.
Why make this change though? It seems like it's not a reasonable transform given the existing input scale of the image
There was a problem hiding this comment.
In this, the value range of image is greater than 255. As we are setting Augmix value range to be [0, 255], so the demo does not work without making this change. Any better alternative? 🤔
There was a problem hiding this comment.
okay this seems more or less reasonable. However, this does make resize_demo.py strange, as that file already manually rescales by 255.0 on line 34. Let's get rid of that rescaling there and then this is fine.
|
/gcbrun |
* update augmix segmentation mask * fix * fix * added demo * add test * update readme * fix * fix * fix
What does this PR do?
Related: #1992
Add support of segmentation mask in Augmix layer. Here is the colab link for demo.
Output generated by demo:
Who can review?
@ianstenbit @jbischof