Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EnzymeRules for batchnorm #537

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Conversation

wsmoses
Copy link
Contributor

@wsmoses wsmoses commented Sep 25, 2023

Follow up to #536

I made this a separate PR, because for efficiency sake, it is better to have an inplace version of batchnorm, but that wasn't an exported API. Thus I implemented this on the inplace cuda one (which was the only implementation of batchnorm, in place or not).

If NNlib is later refactored to have batchnorm! this can go on the refactored rule.

@mashu
Copy link
Contributor

mashu commented Sep 25, 2023

Few suggestions ;)

  1. syntax: more than one semicolon in argument list around around line batchnorm.jl:288
  2. refactor into single line all || in if statements as starting new line with || fails compilation on 1.9.3

@CarloLucibello
Copy link
Member

There is a weird parsing error in the Code Coverage stage

@CarloLucibello
Copy link
Member

CarloLucibello commented Nov 7, 2023

Does anyone know about the ubuntu CI error

ERROR: LoadError: Base.Meta.ParseError("parsing error in ext/NNlibCUDACUDNNExt/batchnorm.jl:253: invalid identifier name \"||\"")
Stacktrace:
 [1] amend_coverage_from_src!(fc::FileCoverage)
   @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:186

?

@mashu
Copy link
Contributor

mashu commented Nov 7, 2023

Does anyone know about the ubuntu CI error

ERROR: LoadError: Base.Meta.ParseError("parsing error in ext/NNlibCUDACUDNNExt/batchnorm.jl:253: invalid identifier name \"||\"")
Stacktrace:
 [1] amend_coverage_from_src!(fc::FileCoverage)
   @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:186

?

I couldn't compile the branch on my local machine with Julia 1.9.3. I was also running into problems with parser not figuring out it's the new line, see my comment earlier. Basically code won't compile for me without fixing these.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants