Colab Notebook for Normalizations

[Smokrow]

First draft of a Colab Notebook for Normalization layers.

I tried to formulate the description very basic and non technical for non ML experts while also including some more advanced topics like initialisers for example.

I will probably add a Recurrent example for one of the layers (Maybe layer normalization) as well.

[review-notebook-app]

Check out this pull request on ReviewNB: https://app.reviewnb.com/tensorflow/addons/pull/144

Visit www.reviewnb.com to know how we simplify your Jupyter Notebook workflows.

[armando-fandango]

Thanks @Smokrow Looks great. @seanpmorgan @facaiy should the ipynb files be in examples folder ? Because /docs folder would be overwritten everytime we run API docs generator.

[Smokrow]

Good point. I will move it

[seanpmorgan]

Hi @Smokrow this looks great and thanks for starting our push for examples. Comments noted above

[Smokrow]

@ppwwyyxx as promised the colab notebook we talked about #14 . Would you mind taking a look?

Do you know a good example we can add to showcase the axis parameter? (I tried using channels first/channels last but this will not run on colab since we have no GPU on there)

[seanpmorgan]

Hmmm.. Colab should have a GPU, but our pip package has not implemented GPU kernels yet (See #118). This is also a good time to make sure that axis parameter is intuitive because @ppwwyyxx expressed concerns about it not really making sense.

[seanpmorgan]

@Smokrow when time allows:

1. Remove the duplicate notebook
2. Move the notebook to just tensorflow_addons/examples
3. Rename the notebook to layers_normalizations
4. Edit the colab link to be the location the notebook will be at once it's merged.
5. Update the content now the LayerNorm has been removed
6. Provide an end to end example of GroupNorm vs. BatchNorm or LayerNorm from core

[Smokrow]

We mark this PR as WIP until tf core adds a normalization layer. After that I will change the links in the notebook since the comparison is not working right now

[seanpmorgan]

@Smokrow Just FYI this should be unblocked now that we are built against tf2-beta

[Smokrow]

@seanpmorgan
Everything seems to be working fine now
Feel free to merge.

Sry for the long wait

[seanpmorgan]

LGTM Thanks Moritz!